It inevitably will. If Witcher 3 was held back by current gen consoles of the time, of course the Xbox one will hold back things. Even if they approached it as 2 completely optimised ports, thats a stretch on resources which otherwise could have been poured into series X games. Either getting them out quicker or improving their fidelity. Outside of that the SSD should be informing game design.
This isn't a problem right out the gate though, I also don't think most of these Playstation 5 exclusive are built for the ground up for PS5 even if they end up skipping PS4 come release date.