Pemalite said:
I would argue Unreal Engines TSR offers the better option currently.
It's a good idea to watch the PC space as that is where the innovation is occurring on this front, with A.I being the hot buzz word, Intel and AMD rolling out A.I capabilities in their CPU's, we are in for an interesting few years. |
I've been tracking Nvidia's white-papers (and the research in general), and there are a lot of features of Visual Deep-Learning that they haven't yet added to current "DLSS" versions, but they're being developing in the field.
For example, this paper where they introduce compact-NGP, which is an even more efficient NGP (Neural Graphics Primitives) model than Instant NGP.
https://research.nvidia.com/labs/toronto-ai/compact-ngp/
NGPs can eventually be used for better ray-construction and lighting in general, dynamic real-time asset generation, efficient physics simulations, smoother animations, etc. In the paper they're initially using them for less-lossy texture-compression, which can make LOD-management even more seamless, but the possibilities are pretty diverse.
"Compact neural graphics primitives (Ours) have an inherently small size across a variety of use cases with automatically chosen hyperparameters. In contrast to similarly compressed representations like JPEG for images (top) and masked wavelet representations [Rho et al. 2023] for NeRFs [Mildenhall et al. 2020] (bottom), our representation neither uses quantization nor coding, and hence can be queried without a dedicated decompression step. This is essential for level of detail streaming and working-memory-constrained environments such as video game texture compression. The compression artifacts of our method are easy on the eye: there is less ringing than in JPEG and less blur than in Rho et al. (though more noise). Compact neural graphics primitives are also fast: training is only 1.2-2.6x slower (depending on compression settings) and inference is faster than Instant NGP because our significantly reduced file size fits better into caches."
Here is another paper (not from Nvidia) that goes through the details of how NeRF's (which NGPs are a generalization of) can be used in rasterization pipelines. There are some example assets on their website.
https://mobile-nerf.github.io/