So accourding to Anouma, the puzzle solving starts from the beginning. You can approach areas in any order and from any direction.
Neat. Cool.
But what about travelling to these places? What if you're an unlucky bud like me and end up travelling to the place where you get Epona, or any other ride, or the teleportation device or sprint shoes or whatever like, halfway through the game or even later?
So to stop people from doing that, and from having to go through the hassle of travelling a huge open space without a suitable way to do so for long (I'm looking at YOU Hyrule Feild in the first half of Ocarina of Time) they will likely give you the horse early. Maybe at the beginning.