Part of it was with the influx of casual gamers. All the new gamers would be turned off by challenging games so they were made easier to be accessible to a wider audience, but in doing so require no skill and are easily beaten.
Also with older games, there wasn't much room to make a long games, so they just made them harder to justify your purchase. Gradually it seems games have gotten easier as the amount of content they can fit on the disc/cart increased.