It is difficult to pinpoint exactly when humans domesticated horses, but evidence shows that it might be as early as 6000 BC. Around 4200 BC, humans started riding atop horses in order to herd cattle and sheep better. This enabled them to take better care of livestock- so if you think about it, domestication of horses certainly contributed towards the creation of human settlement and improvement in comfort of life.
Horse riding was popular even before the invention of the wheel. Once the wheel was invented, for a long time, horses were also used to pull chariots, instead of having people directly ride of them. Of course, horse riders were used to deliver messages to different parts of the world, because of the fact that horses were the fastest domesticated animals.
Of course, one cannot consider history of horse riding without discussing the single biggest use of horse riders- war. Horses have been used in war for time immemorial. There are many written reports from ancient and medieval history that show that the cavalry (soldiers mounted on horses) were strategically important for armies. Mounted soldiers were faster than soldiers on foot, and their swiftness and stealth have been important to the creation of a number of war strategies throughout history.
One can presume that horse riding is just as old as the domestication. It seems like humans were interested in horse breeding, choosing the choice horses and betting on which one would win for quite a while. Horse racing and gambling have shared common roots since the ancient times, since one can find a lot of reports about thoroughbreds being raced for money. Rome used to have incredibly popular horse races and chariot races, and a considerable amount of money would ride on the favourites! As one can see, almost all these horse riding trends still continue- to this very day!