I doubt losing a couple of inches in form factor every year is going to end generations as we know them.

Not true, with consoles you don't have to deal with limitations of last gen or deal with crazy fragmentation so it strikes a rather nice balance ... (Star Citizen is in many ways technically inspired by consoles to raise standards with higher minimum specs.) 

PS4 Pro didn't hurt anyone since no content was locked behind it and the vast majority of AAA developers can't afford to make different versions with different assets so we make due for generational updates instead since a lot of their customers find new PC hardware iterations to be toxic ... 

Well that's probably because most devs don't have a good LOD or streaming system in place. Ideally you wouldn't need a texture any bigger than the total size of the framebuffer itself ... (1:1 sampling ratio is a perfect match)

You only need 2048*2048 textures in worst case scenario if a player decides to do a close up on a 1080p screen and we even have texture compression and mipmapping to keep the storage problem tractable ... 

If we're doing 1080p then 16GB will be plenty and I imagine it will be enough for 1440p too but maybe not so much for 4K but I hope we go in the direction of real-time physically based global illumination instead of higher resolutions ...

PS2/Xbox to PS3/360 was the last big leap in my opinion.

Since then, the gaps have felt more like incremental steps than the quantum leaps of the past.

I honestly feel like there's not much progress to be made anymore. Realistic dynamic lighting can now be done. Textures can be taken from real life photographs of inanimate objects, and then processed in photoshop. Polycounts are high enough to afford to model small details like the buttons on your shirt, without losing performance.

What the heck else can be done?

I understand the "leap" from PS360 to current gen. Having just megabytes of ram to work with was a joke.

But now? Again, what is there to improve on?

Number of elements on screen and visual fidelity won't generate new experiences that are now possible.
I hope the extra power would be used on AI solutions.
If we have only visual improvements I would rather have a portable/hybrid switch like ps4 instead.

Well for one thing there's still a lot to be done in global illumination since games either bake it or omit it altogether compared to film rendering ... 

I agree with you on traditional displays. Gran Turismo Sport, Horizon, and Uncharted already look amazing. In all reality there is nothing new happening this gen on traditional displays, that could not have been on PS3/360 at lower resolutions and lower detail.

Where that extra power is massively important is in new experiences in the motion, VR, and AI spaces.

VR and Motion need much more CPU and GPU power for many reasons. Motion needs better tracking equipment and more CPU power to be able to accurately and reliably track both peripherals and full body tracking. This will add an incredible amount of freedom for new styles of gameplay, and allow current genres to better represented. On the VR side, we need both assive amounts of CPU power to increase the frame rates to 120fps, while having the GPU power to draw two screens at once at resolutions much higher than even 4K. 2K/1080P is fine on a 50" or below display, when you blow the image up to 300+ inches, and fill over 100° of your vision 2K looks like an old 480P flat screen.

AI has the ability to make huge advancement in the way stories are told, and relationships are built with digital characters. For this to happen, we need even more CPU power. This is power on top of what is needed to render real time effects on two screens at higher than 4K resolution, and a rock solid 120fps.

Therea many, many advancements that still need much, much more power to achieve. I believe we have hit somewhat of a sweet spot for traditional displays, and I think we are at the point where we can have expereinces on the level of GTA5, Horizan, Uncharted 4, ect. on mobile devices. VR is where the future of all things digital is heading, and that space is vibrant for more power.

UE4 already does that though. Are there improvements to be made, or are modern games not taking advantage of it yet? 

UE4 is trash and there are definitely very real improvements to be made in graphics since AAA games still aren't comparable to pre-rendered films ...

PS3/XBox360 to xbox one/ps4 was a pretty low jump in form of CPU

It was a good direction in terms of available ram, ram speed and GPU, same what xbox one x is doing right now.
Its all about how that power is used, the jump from xbox360 to xbox one and xbox one to xbox one x isn't a huge difference.

Next generation will have better CPU's thus making more instruction cycles possible, making better physics possible, making better ai possible, making higher framerates possible.

A cpu change will be a much bigger leap in terms of games and gameplay then we have seen this generation.

So I disagree.

*things about looking at destiny 2 running at 1080p/60fps on a $50 pc on digital froundry yesterday and YES 60fps destiny 2 is a massive jump from 30*