There was a big difference in games like the last of us between PS3 and PS4 version, HUGE difference, especially if you played the PS4/ps4 pro version in a 4K HDR tv.
Other notable mentions are Metal Gear Solid V, The evil within 1 ps3/360 to the evil within 2 ps4/Xbone.
So yeah, it will be a very noticeable difference in ps5/x series X compared to the current consoles.