Can someone explain to me what the difference is between DLSS to normal resolution pixels? And am I correct in assuming DLSS is a more efficient way to have better visuals, which means weaker hardware can have better visuals, or is that wrong?
You are correct in assuming DLSS is a more "efficient" way to have better visuals while using the same processing power.
But I want to explain much deeper than that. I'm going to simplify and overlook a lot of complexities and nuances for the sake of making this more digestible for people who aren't into tech. I'll also explain a bit about how game processing works beyond just DLSS so other stuff the likes of CGI and Pemalite explain around here makes more sense in the future for many. This post isn't just for you so I'm sorry if you already understand this stuff beyond the question you asked. I really don't mean to be talking down to you if it comes off that way. I just want to make it more general for anyone interested.
Let's break how a game processes a frame down into 2 things
- Logic - What's happening in the game. Where objects are located, what they're doing, the implication of what objects are doing has on other objects (i.e. physics), and where objects are going. The result of logic processing is a few wireframes (skeletons) of the game world at a specific point of time. These wireframes represent different types of information about the game world like the depth of objects in relation to one another and "motion vectors" which represents which directions objects are moving in. This typically happens 30 or 60 times a second for console games. And this typically takes place on the CPU.
- Image - Taking all the wireframe information from a particular camera view and then painting it in with colour, textures, shadows, and effects. If you want to really simplify it, you can think of it as painting made up of dots. The higher the number of dots (pixels) you paint an image with, the more work it is. And each dot takes a bunch of information about light, texture, shadows, object, and depth to figure out what exactly needs to be painted for each dot. The more effects you are using, the more time each dot takes to paint. And this process typically takes place on the GPU.
The general aim of upscaling techniques is to paint with less dots and then use math to fill in more dots (the missing information) to make it fit the wanted output resolution. The math that is used is what separates different forms of upscaling and DLSS is a very advanced way of filling in that extra information.
For example, the easiest form of upscaling an image is to just duplicate information that is already there. Going from 1080p to 4K is just doubling the information of a pixel both horizontally and vertically. So the game engine processes the image for a 1080p image and then before outputting it to the TV, makes each dot/pixel take up 4 dots instead in a square fashion.
Checker-boarding is much more advanced than that. It renders the initial image at a lower resolution and then uses information from previous frames and motion vectors (how the objects are moving), along with other information, to help fill in the gaps of this lower resolution image before outputting it to the TV. It also applies additional image filters like sharpening to focus on smoothing edges in the image (where colours change drastically between neighbouring pixels - also known as high frequency areas in image processing). But that leads to a different but related discussion of anti-aliasing.
DLSS takes this process of filling in the image with more information one step further. With checker-boarding and upscaling, the same math model is applied to all images and all pixels pretty much the same way. But now imagine you could learn "best case scenarios" and know how to treat different situations with different methods of upscaling. This is what DLSS does. Roughly the process is:
- An AI (created by Nvidia) is shown how a game should look at a very high resolution.
- Then the AI is fed lower resolution images/videos of similar scenes and it works out how to apply math to the image to make it look like the high resolution one.
- This process is iterated on a few times to improve the math. The machine is also given different scenes from the game to help improve the math.
- The machine spits out a bunch of math that can be used to take a lower resolution image and make it look higher resolution. The math involves existing image data, data from past images, and more and is even more context aware based on what might be happening in the game. The process from steps 1 to 4 is known as machine learning, or "deep-learning"; which is where DLSS comes from. DLSS = Deep Learning Super Sampling
- The math is included in the game files or the driver/code used to operate the GPU.
- Now when the game is running the initial image is rendered at a lower resolution but then upscaled using the fancy math produced by the AI to make an image much closer to (and in some cases even better than) an image that would require much more processing power at a higher resolution.
For those who were interested, I hope this layman explanation helped
EDIT: And now I see I was beaten to the punch with a much more succinct but still good explanation. Haha! Back to work I go.
Last edited by trasharmdsister12 - on 09 October 2020