It all comes down to the simple fact that development budgets shot up too far too fast in this generation, meaning that developers were forced to dilute proven game formulas to try to reach broader audiences because they quite frankly had to in order to hedge the costs of HD development. We've seen this problem become less of an issue over the course of the generation as costs have come down, such that the number of big-budget flops, so prolific in 2008, have dropped down, but in the case of some Japanese brands the damage has been done (namely Final Fantasy)
One also forgets that Japan pretty much owns the dedicated handheld space, an area where Western third parties have never seriously tried, and a space that for the 7th gen accounted for over 200 million units (dissolving overlap, you're still looking at at least 160 million unique individuals), and Japanese games have been more innovative and successful in these spaces (Monster Hunter, Level 5's stable, et al).
Japan shouldn't focus too much on the Japanese market (you see the problems with that phenomenon in anime currently, and it works to contract the market just as much as trying to please too many people does), but it is clear that the PC-development mentality that works for many Western developers does not work for the Japanese
Monster Hunter: pissing me off since 2010.
