Different categories but I won't complain about it. GTS was clearly going to flop relative to series' standards, it's one of the safest and most accurate predictions I made on VGChartz. Now, a proper GT7 as a launch or early PS5 title could easily break 9 million (On second thought, it may not be too easy anymore. The damage extent of all the missteps remains to be seen) but GTS never had a good shot, being a non-numbered entry with slim content and confusing marketing.
It's kind of weird to blame MK8's decline on install base and simultaneously ignore install base altogether when comparing GT4 to GT5 (which combined with digital, GT5 is actually ahead. FAR ahead if counting the prologue versions of both) The noticeable decline only occurred after Gran Turismo 6. That's 4.5 years ago, NOT 10! Granted the install base disparity ratio between PS2 and PS3 is a lot smaller than Wii to Wii U. But don't attempt to sell GT5's growth over GT4 as a decline!
GT4 had its "2nd game on the console" disadvantage while GT5 had the "smaller install base, challenge vs X360" disadvantage. Otherwise the GT brand remained massive in GT5's time.
Sorry but decline already started with GT5 as GT5 was the first title of past gen and should have done as much or even better as GT3 did on PS2. There is no point in denying that. Also the PS2 userbase was way lower than the PS3 one when GT3 was released in early 2001 vs. GT5 in late 2010. Timing of a release within a console lifecycle is essential in these kind of analysis. Would GT5 been released in the beggining of 2007 and GT6 in 2010/11, sales would have been completely different.
That said, let's not forget the decline in the genre itself. Racing simulator is not as hyped as it was 15 years ago.
Games (especially universally acclaimed games) releasing early in a console's life benefit from it in the long run because of the relatively weak competition and small selection of games early on. See Knack 1 vs Knack 2, Resistance 1 vs sequels, OoT vs Majora's Mask, Twilight Princess vs Skyward Sword, etc. Zelda Breath of the Wild wouldn't have been the sales phenomenon it is now if it were released at a time where Switch's game library is big, would have still sold ganbsusters but fallen maybe 2 or 3 million copies short of the current projection. Same applies to Super Mario 64 which went on to sell a whopping 12 million on N64's modest install base. The launch/early-release effect is an undeniable advantage. GT3 enjoyed that benefit, GT5 did not.
GT5 should have been expected to sell less than GT3 but more than GT4, and that's exactly where it landed. Again, counting the $60 Prologue, GT5's sales are actually head and shoulders above GT4's; it's 17.29M vs 13.6M as of October 2015.
If we speak strictly sales numbers, and disregard GT4/Prologues and everything in between, then yes of course GT5 declined (GT3's 14.89M vs GT5's 11.94M)