Currently Online 
Soundwave said:
It's not rendering anything, it's just an AI algorithm trained to be able to write over things like faces, lighting, materials, etc. and to quickly be able to alter the 2D image and "overhaul" the final output image you see on the screen. It's probably more accurate to call it "image manipulation done at extreme speed" (30 fps+). |
DLSS 5 is indeed a post-process effect, but all of these other features are directly part of the render pipeline and aren't post-process effects. In the case of neural materials, as an example, small MLPs tweak the material properties of specific materials to maintain a high quality without performance penalties. This happens far before the image is finally processed and output to the screen.
https://research.nvidia.com/labs/rtr/neural_appearance_models/
|
Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior—transformation of directions into learned shading frames—facilitates accurate reconstruction of mesoscale effects. The second prior—a microfacet sampling distribution—allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation. |
The input is the actual sampled mesh and shading data and the output are material properties.
