Nvidia reveals DLSS 5 , essentially applies AI filter to games in real time.

[Cerebralbore101]

sc94597 said:

Cerebralbore101 said:

Can you explain exactly what neural rendering, Neural Materials, Mega Geometry, and Neural Shaders are? How are they different from regular rendering, regular materials, geometry and shaders? Is neural just a fancy way of saying "made by AI?"

Neural rendering is just any sort of feature of graphics rendering that is assisted with a neural network. Technically this would include all DLSS version, really. 

As for the specific features, Nvidia explained them last year. 

Nvidia defines all of the different features here. 

https://developer.nvidia.com/blog/get-started-with-neural-rendering-using-nvidia-rtx-kit/

Name	Description	Status
RTX Neural Shaders (new)	Train and deploy neural networks within shaders to unlock new compression and approximation techniques for next-generation asset generation.	Available now
RTX Neural Materials (new)	Use AI to compress shader code of complex multi-layered materials for up to 5X faster material processing to bring real-time performance to film-quality assets.	Get notified when RTX Neural Materials is available
RTX Neural Texture Compression (new)	Use AI to compress textures with up to 8x VRAM improvement at similar visual fidelity to traditional block compression at runtime.	Available now
RTX Texture Filtering (new)	Randomly samples textures after shading and filters difficult volumes, reducing artifacts and improving image quality.	Available now
RTX Mega Geometry (new)	Accelerate BVH building for cluster-based geometry systems like Nanite for real-time path tracing of millions of triangles.	SDK Available now Get notified when it is available in the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Character Rendering (new)	Set of tools to create path-traced hair and skin.	Available now
DLSS 4 (new)	Multi Frame Generation has been introduced for GeForce RTX 50 Series GPUs, allowing for generation of up to 3 frames per rendered frame.	Available now through NVIDIA Streamline SDK and Unreal Engine 5 plugin
Reflex 2 (new)	Reflex technologies optimize the graphics pipeline for ultimate responsiveness, providing faster target acquisition, quicker reaction times, and improved aim precision in competitive games.	Reflex Low Latency available now Get notified for Reflex Frame Warp availability
RTX Dynamic Illumination	Library of importance sampling algorithms that sample the most important lights in a scene and render them physically accurate. Provides implementations for three techniques: ReSTIR DI, GI and PT.	Available now ReSTIR PT coming soon to the SDK and the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Global Illumination	Scalable solution to compute multi-bounce indirect lighting. Provides implementations of two techniques: Neural Radiance Cache (NRC) and Spatially Hashed Radiance Cache (SHARC).	Available now (NRC is experimental)
RTX Path Tracing	Reference solution for optimized real-time path tracing utilizing several RTX technologies.	Available now
NVIDIA Real-Time Denoisers	Library of denoisers designed to work with low ray-per-pixel signals.	Available now
NVIDIA Opacity Micro-Map	Efficiently map intricate geometries onto triangles and encode their opacity for better ray tracing performance.	SDK available now Coming soon to the NVIDIA RTX Branch of Unreal Engine 5 (NvRTX)
RTX Memory Utility	Compaction and suballocation of acceleration structures to reduce memory consumption.	Available now
SpatioTemporal Blue Noise	Utility tool containing some pregenerated blue noise textures and sample code for generating new blue noise textures	Available now
Shader Execution Reordering	Performance optimization that unlocks the potential for better execution and memory coherence in ray tracing shaders.	API available now for NVIDIA RTX 40 Series and later

So after looking into it, so much of this stuff isn't needed for Nintendo 1st party games. Cartoon looking graphics don't need ray tracing at all. I could see benefits from neural textures but it's one thing to tell an AI model to make a photorealistic stone texture on the fly, and asking it to texture anything organic and complex. Yeah take 1000 pics of a glass or stone or wood object to train AI to make a texture for a material in game. That's a good use of AI. Same goes for saving ram space. Or figuring out how to light a jewel in real time. 

But having it do what it does to RE Requeim or Hogwarts characters is horrible. And for Nintendo, their asset library is massive already. Why use AI to make a hyper realistic brick texture when some artist did it good enough for a cartoon artstyle in 2016 and you still have that asset?