By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming - Nvidia reveals DLSS 5 , essentially applies AI filter to games in real time.

Eagle367 said:
Norion said:
It's not a potential thing as SC has gone into in other posts, it's already been helping out the medical field in the past few years. For what benefits to society can arise there's causing huge strides in medical research and scientific research in general, self driving cars becoming the norm and robots helping care for the elderly and disabled for a few big ones. Just those things alone will prevent ten's of millions deaths every year alongside a multitude of other huge benefits.
For banning it that's of course completely unrealistic but even disregarding that it's not a sensible position cause of those major applications. As I said in another post if you don't like ways it's being used right now then pushing for regulation is a far better approach. 

It being overall a good or bad thing is a different discussion though. My issue here was the implication in that post that the investment in AI is bringing no benefits when that is very clearly not the case. It'd be the same if someone said AI is causing no negatives, that would also be an absurd thing to say.

To get into that discussion though I do think there's a reasonable debate to be had over if the benefits or negatives are bigger right now so don't think it's as clear cut as you're saying. Me and SC have gone into various use cases in past posts in the thread and in general the benefits are already big and on my end the tech has overall improved my life so far. So many people are using it now to do things like help plan vacations and make their workflows more efficient so a lot of people are already getting a lot of value out of it.

Italicized: for self driving cars,  just use trains, buses and public trasnport. They are better, have been proven to be better for humans in general and are such a net benefit its not even funny. That will save many lives. For the old person care thing, i have a bridge to sell you. The real solution is not overworking people to the bone and having well funded human resources for the disabled and elderly. Its kot gonna be robots taking care of them, its gonna be humans. And for banning, it is possible if there is a will to do it. Its not as unrealistic as you think. You can ban gen AI slop and establishment of data centres, etc. There are a myriad of ways.

bolded: Increasing laziness of people as I said. Research has shown that Gen AI use is rotting the brains of so many people.

The cars vs public transport debate is a separate issue since public transport will also be automated in the future. Self driving vehicles are way safer is my point with that part. Do you think advanced robots are never gonna be a thing? Cause that's a matter of when, not if so they're obviously gonna be utilized in ways like that. People will still be involved since that'll be wanted but once there's enough of them it'll be a massive help in making sure that everyone gets the care they need. Like each patient could eventually have their own robot carer assigned to them that they could call on anytime they want and they'll be ready to immediately assist with something anytime if no people are currently available or if they don't wanna deal with other people.

For the banning part it is that unrealistic, the US and China are trying to be the first country to achieve AGI since that'll be a massive boon to whoever gets there first so neither of those countries are gonna do something that harmful to their national interests. In general the tech is so broad in terms of its applications and is already being used for so much that this is equivalent to the idea of banning the internet a couple decades or so ago. It's just a completely absurd idea so as I've said in past posts if someone doesn't like certain ways it's being used then the answer is pushing for regulation.

And for the last part can you share this research? Cause a big issue with studies about AI is the technology is changing so fast that even ones that are just a year old are already outdated so are worth way less compared to when they came out.

curl-6 said:
Norion said:

Mass layoffs only need to be a temporary issue. A world where all the work required to keep society running is being taken care of by AI systems and robots letting people do whatever they want with their time would make concepts like unemployment rate not matter any more. Governments are gonna need to be ready to handle the period between now and then though.

In general there are indeed future concerns like authoritarian regimes using the technology to crack down on dissent and entrench their rule but overall I very much expect the good to outweigh the bad since the good stuff that's already baked in is massive and that sort of world I mentioned is such a strong end goal that I really think something as disastrous as a Skynet sort of scenario would have to happen for the bad to outweigh the good long term.

Regimes are already using AI for surveillance, propaganda, misinformation, and control, and it's only early days for the technology; looking at the state of the world today with the far right in ascendency across the Western world and authoritarianism thriving across the globe, do we really think politicians and billionaires are going to deploy this technology for the good of the masses, when they're already weaponising it against us?

For democracies like ones in the Western world people can vote so if the masses feel like their governments are handling the issue badly they'll vote them out. A significant shift from current capitalistic systems in the next 10-20 years seems pretty likely to me. My biggest concern in the transition period is the phase when there is mass unemployment for white collar work but blue collar work isn't being impacted that much yet. Governments will need to make sure to help support their people through that.



Around the Network
Cerebralbore101 said:
sc94597 said:

Why "except Nintendo?" They're fully aligned with Nvidia. SW3 will almost certainly have a Nvidia chip and neural rendering features. It'll be interesting to see the Nintendo-primary users in this thread navigate this. I am sure there will be some arbitrary distinction of why it isn't gen-AI. 

I don't think Nintendo would be okay with just having their assets "drawn over" like all the pictures in this thread. At most they might make use of lighting and shadows being improved with AI, but they would say hell no to what this thread's examples are doing to characters and other assets. 

Nintendo also likes to recycle old assets to a hilarious level, so the longer they stay in the Wii U to Switch 1 level of graphics for their mainline games the faster it is for them to make. They are already about 13-15 years into making Switch 1 level assets. Switch 2 is mostly being used for higher resolution and other small additions. 

If it did happen though, I would just flat out stop buying Nintendo games. 

Even with the current version of DLSS 5 they could just mask their characters and have it only apply to environments. Nintendo's internal teams do have some counter-trending positions on graphics (they didn't use AA for the longest time as an example) but I don't think they will opt out of using a feature that is available to them. 

Also neural-rendering is more than just DLSS 5. Neural materials, neural shaders, neural radiance caches, etc. all work on the assets themselves, so there really isn't anything "drawn over." DLSS 5 probably will be old and deprecated by SW3 releases. 



Norion said:
curl-6 said:

Regimes are already using AI for surveillance, propaganda, misinformation, and control, and it's only early days for the technology; looking at the state of the world today with the far right in ascendency across the Western world and authoritarianism thriving across the globe, do we really think politicians and billionaires are going to deploy this technology for the good of the masses, when they're already weaponising it against us?

For democracies like ones in the Western world people can vote so if the masses feel like their governments are handling the issue badly they'll vote them out. A significant shift from current capitalistic systems in the next 10-20 years seems pretty likely to me. My biggest concern in the transition period is the phase when there is mass unemployment for white collar work but blue collar work isn't being impacted that much yet. Governments will need to make sure to help support their people through that.

The thing is though, sometimes democracies vote in authoritarianism; its not hard for popularist demagogues to persuade enough of the population to hand them power, as we've seen many times throughout history.

If anything, the last decade in particular has shown us that governments, big tech, and the elites don't give a shit about the wellbeing of the masses and will use every means they can including AI to monopolize power and wealth as the common people's expense.



sc94597 said:
Cerebralbore101 said:

I don't think Nintendo would be okay with just having their assets "drawn over" like all the pictures in this thread. At most they might make use of lighting and shadows being improved with AI, but they would say hell no to what this thread's examples are doing to characters and other assets. 

Nintendo also likes to recycle old assets to a hilarious level, so the longer they stay in the Wii U to Switch 1 level of graphics for their mainline games the faster it is for them to make. They are already about 13-15 years into making Switch 1 level assets. Switch 2 is mostly being used for higher resolution and other small additions. 

If it did happen though, I would just flat out stop buying Nintendo games. 

Even with the current version of DLSS 5 they could just mask their characters and have it only apply to environments. Nintendo's internal teams do have some counter-trending positions on graphics (they didn't use AA for the longest time as an example) but I don't think they won't use a feature that is available to them. 

Also neural-rendering is more than just DLSS 5. Neural materials, mega geometry, neural shaders, etc. all work on actual work on the assets themselves, so there really isn't anything "drawn over." DLSS 5 probably will be old and deprecated by SW3 releases. 

Can you explain exactly what neural rendering, Neural Materials, Mega Geometry, and Neural Shaders are? How are they different from regular rendering, regular materials, geometry and shaders? Is neural just a fancy way of saying "made by AI?"



Cerebralbore101 said:
sc94597 said:

Even with the current version of DLSS 5 they could just mask their characters and have it only apply to environments. Nintendo's internal teams do have some counter-trending positions on graphics (they didn't use AA for the longest time as an example) but I don't think they won't use a feature that is available to them. 

Also neural-rendering is more than just DLSS 5. Neural materials, mega geometry, neural shaders, etc. all work on actual work on the assets themselves, so there really isn't anything "drawn over." DLSS 5 probably will be old and deprecated by SW3 releases. 

Can you explain exactly what neural rendering, Neural Materials, Mega Geometry, and Neural Shaders are? How are they different from regular rendering, regular materials, geometry and shaders? Is neural just a fancy way of saying "made by AI?"

Neural rendering is just any sort of feature of graphics rendering that is assisted with a neural network. Technically this would include all DLSS version, really. 

As for the specific features, Nvidia explained them last year. 

Nvidia defines all of the different features here. 

https://developer.nvidia.com/blog/get-started-with-neural-rendering-using-nvidia-rtx-kit/

Name DescriptionStatus
RTX Neural Shaders (new)Train and deploy neural networks within shaders to unlock new compression and approximation techniques for next-generation asset generation.Available now
RTX Neural Materials (new)Use AI to compress shader code of complex multi-layered materials for up to 5X faster material processing to bring real-time performance to film-quality assets.Get notified when RTX Neural Materials is available 
RTX Neural Texture Compression (new)Use AI to compress textures with up to 8x VRAM improvement at similar visual fidelity to traditional block compression at runtime.Available now
RTX Texture Filtering (new)Randomly samples textures after shading and filters difficult volumes, reducing artifacts and improving image quality.Available now
RTX Mega Geometry (new)Accelerate BVH building for cluster-based geometry systems like Nanite for real-time path tracing of millions of triangles.SDK Available now

Get notified when it is available in the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Character Rendering (new)Set of tools to create path-traced hair and skin.Available now
DLSS 4 (new)Multi Frame Generation has been introduced for GeForce RTX 50 Series GPUs, allowing for generation of up to 3 frames per rendered frame.Available now through NVIDIA Streamline SDK and Unreal Engine 5 plugin
Reflex 2 (new)Reflex technologies optimize the graphics pipeline for ultimate responsiveness, providing faster target acquisition, quicker reaction times, and improved aim precision in competitive games. Reflex Low Latency available now

Get notified for Reflex Frame Warp availability
RTX Dynamic IlluminationLibrary of importance sampling algorithms that sample the most important lights in a scene and render them physically accurate. Provides implementations for three techniques: ReSTIR DI, GI and PT.Available now

ReSTIR PT coming soon to the SDK and the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Global IlluminationScalable solution to compute multi-bounce indirect lighting. Provides implementations of two techniques: Neural Radiance Cache (NRC) and Spatially Hashed Radiance Cache (SHARC).Available now (NRC is experimental)
RTX Path Tracing Reference solution for optimized real-time path tracing utilizing several RTX technologies.Available now
NVIDIA Real-Time Denoisers Library of denoisers designed to work with low ray-per-pixel signals.Available now
NVIDIA Opacity Micro-MapEfficiently map intricate geometries onto triangles and encode their opacity for better ray tracing performance.SDK available now

Coming soon to the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Memory UtilityCompaction and suballocation of acceleration structures to reduce memory consumption.Available now
SpatioTemporal Blue NoiseUtility tool containing some pregenerated blue noise textures and sample code for generating new blue noise texturesAvailable now
Shader Execution ReorderingPerformance optimization that unlocks the potential for better execution and memory coherence in ray tracing shaders. API available now for NVIDIA RTX 40 Series and later



Around the Network
Cerebralbore101 said:
sc94597 said:

Even with the current version of DLSS 5 they could just mask their characters and have it only apply to environments. Nintendo's internal teams do have some counter-trending positions on graphics (they didn't use AA for the longest time as an example) but I don't think they won't use a feature that is available to them. 

Also neural-rendering is more than just DLSS 5. Neural materials, mega geometry, neural shaders, etc. all work on actual work on the assets themselves, so there really isn't anything "drawn over." DLSS 5 probably will be old and deprecated by SW3 releases. 

Can you explain exactly what neural rendering, Neural Materials, Mega Geometry, and Neural Shaders are? How are they different from regular rendering, regular materials, geometry and shaders? Is neural just a fancy way of saying "made by AI?"

It's not rendering anything, it's just an AI algorithm trained to be able to write over things like faces, lighting, materials, etc. and to quickly be able to alter the 2D image and "overhaul" the final output image you see on the screen.

It's probably more accurate to call it "image manipulation done at extreme speed" (30 fps+).  

Last edited by Soundwave - 20 hours ago

Cerebralbore101 said:
sc94597 said:

Why "except Nintendo?" They're fully aligned with Nvidia. SW3 will almost certainly have a Nvidia chip and neural rendering features. It'll be interesting to see the Nintendo-primary users in this thread navigate this. I am sure there will be some arbitrary distinction of why it isn't gen-AI. 

I don't think Nintendo would be okay with just having their assets "drawn over" like all the pictures in this thread. At most they might make use of lighting and shadows being improved with AI, but they would say hell no to what this thread's examples are doing to characters and other assets. 

Nintendo also likes to recycle old assets to a hilarious level, so the longer they stay in the Wii U to Switch 1 level of graphics for their mainline games the faster it is for them to make. They are already about 13-15 years into making Switch 1 level assets. Switch 2 is mostly being used for higher resolution and other small additions. 

If it did happen though, I would just flat out stop buying Nintendo games. 

I do wonder if a future version of DLSS5 (I really wish they had given this a unique name) if you'll be able to feed it data on what you want the final result to look like to iron out that problem. 

Like if the the algorithm could understand that you want Donkey Kong to look like this (which is art from your own artists). 

Particularly with the fur, then maybe it can understand and zero in on doing that up from a model that actually has a rendering cost of this:



I just realized a major problem with this from corporate perspective:
Generative AI images aren´t copyrightable, so the game IP owner wouldn´t have ownership over game visuals.
Now there is other related IP (character identity etc) but losing copyright over game visuals seems big deal.
It also opens them up to lawsuits over any other IP that may have been violated via the generative AI.
(i.e. it´s training set etc)



mutantsushi said:

I just realized a major problem with this from corporate perspective:
Generative AI images aren´t copyrightable, so the game IP owner wouldn´t have ownership over game visuals.
Now there is other related IP (character identity etc) but losing copyright over game visuals seems big deal.
It also opens them up to lawsuits over any other IP that may have been violated via the generative AI.
(i.e. it´s training set etc)

I believe the law is the image has to be fully created by AI. Characters assisted by AI but still done by humans can be copyright. 



sc94597 said:
Cerebralbore101 said:

Can you explain exactly what neural rendering, Neural Materials, Mega Geometry, and Neural Shaders are? How are they different from regular rendering, regular materials, geometry and shaders? Is neural just a fancy way of saying "made by AI?"

Neural rendering is just any sort of feature of graphics rendering that is assisted with a neural network. Technically this would include all DLSS version, really. 

As for the specific features, Nvidia explained them last year. 

Nvidia defines all of the different features here. 

https://developer.nvidia.com/blog/get-started-with-neural-rendering-using-nvidia-rtx-kit/

Name DescriptionStatus
RTX Neural Shaders (new)Train and deploy neural networks within shaders to unlock new compression and approximation techniques for next-generation asset generation.Available now
RTX Neural Materials (new)Use AI to compress shader code of complex multi-layered materials for up to 5X faster material processing to bring real-time performance to film-quality assets.Get notified when RTX Neural Materials is available 
RTX Neural Texture Compression (new)Use AI to compress textures with up to 8x VRAM improvement at similar visual fidelity to traditional block compression at runtime.Available now
RTX Texture Filtering (new)Randomly samples textures after shading and filters difficult volumes, reducing artifacts and improving image quality.Available now
RTX Mega Geometry (new)Accelerate BVH building for cluster-based geometry systems like Nanite for real-time path tracing of millions of triangles.SDK Available now

Get notified when it is available in the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Character Rendering (new)Set of tools to create path-traced hair and skin.Available now
DLSS 4 (new)Multi Frame Generation has been introduced for GeForce RTX 50 Series GPUs, allowing for generation of up to 3 frames per rendered frame.Available now through NVIDIA Streamline SDK and Unreal Engine 5 plugin
Reflex 2 (new)Reflex technologies optimize the graphics pipeline for ultimate responsiveness, providing faster target acquisition, quicker reaction times, and improved aim precision in competitive games. Reflex Low Latency available now

Get notified for Reflex Frame Warp availability
RTX Dynamic IlluminationLibrary of importance sampling algorithms that sample the most important lights in a scene and render them physically accurate. Provides implementations for three techniques: ReSTIR DI, GI and PT.Available now

ReSTIR PT coming soon to the SDK and the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Global IlluminationScalable solution to compute multi-bounce indirect lighting. Provides implementations of two techniques: Neural Radiance Cache (NRC) and Spatially Hashed Radiance Cache (SHARC).Available now (NRC is experimental)
RTX Path Tracing Reference solution for optimized real-time path tracing utilizing several RTX technologies.Available now
NVIDIA Real-Time Denoisers Library of denoisers designed to work with low ray-per-pixel signals.Available now
NVIDIA Opacity Micro-MapEfficiently map intricate geometries onto triangles and encode their opacity for better ray tracing performance.SDK available now

Coming soon to the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Memory UtilityCompaction and suballocation of acceleration structures to reduce memory consumption.Available now
SpatioTemporal Blue NoiseUtility tool containing some pregenerated blue noise textures and sample code for generating new blue noise texturesAvailable now
Shader Execution ReorderingPerformance optimization that unlocks the potential for better execution and memory coherence in ray tracing shaders. API available now for NVIDIA RTX 40 Series and later

So after looking into it, so much of this stuff isn't needed for Nintendo 1st party games. Cartoon looking graphics don't need ray tracing at all. I could see benefits from neural textures but it's one thing to tell an AI model to make a photorealistic stone texture on the fly, and asking it to texture anything organic and complex. Yeah take 1000 pics of a glass or stone or wood object to train AI to make a texture for a material in game. That's a good use of AI. Same goes for saving ram space. Or figuring out how to light a jewel in real time. 

But having it do what it does to RE Requeim or Hogwarts characters is horrible. And for Nintendo, their asset library is massive already. Why use AI to make a hyper realistic brick texture when some artist did it good enough for a cartoon artstyle in 2016 and you still have that asset?