The main point is that around 25M sales of physical software on the Vita, and a good chunk of sales that were exclusive on the Playstation ecosystem will no longer be exclusive. Although Playstation 5 is assured AAA support of Japanese 3rd parties a lot of those efforts are focus on oversees audiences. While the AA efforts which are basically the vast majority of physical sales on Vita and a good chunk of 3rd party sales on Playstation 4 will now appear on the Switch, making the environment and competition Playstation 5 is entering far more difficult than what Playstation 4 faced.
So although the oversee audience is potentially big, a vast majority of these games were never popular oversees, to begin with. Also, AAA games on Playstation 5 in Japan will be hampered by the low audience. This is why you can expect a lot of cross-gen AAA games by 3rd Party Japanese companies.