Considering we have games like Spider Man, TLOU and others that load under 10s, that comparison doesn't make much sense. None of those titles are really optimized or anything for a good load time, they barely improve against their gen8 version be it on Xbox or PS.
You are correct. They are not supposedly not optimized, but that actually makes things even weirder. In brute speed, the PS5's SSD is twice as fast as the one on the Series X. The means in theory the SSD on PS5 should offer roughly double the performance in unoptimized games.
It is like, you would never expect a 2.4GB memory offering better performance than a 5.5GB one on PC, given all scenarios are the same.