How many yearly installment games are there? Handful of them. But you make it seem like it's the majority of the Western games.
This a supply and demand type business we're talking about. If the devs wanted to go through a different route probably most of them would be closed.
XBLA has the biggest selection in terms of quantity and quality of arcade games. And MS owns 80% of the heavy hitters.
When you say big companies only investing in AAA games, you're using the wrong figure of speech. Take a look at XBLA again if you own a XBOX(which I doubt) and see how many arcade titles they(EA,UBI,Activ) release on that platform.