Remij is a fucking nerd - the thread (GFX technology discussion thread)

[Remij 4,679]

Long post is long  

 

First off.. I get that people here don't care about this.. I'm fine with that.  I love you all.   Anyway, I'm just posting this shit because it's super fucking interesting to me, and maybe some of you will also find it a bit interesting as it really does have implications of the future of 3D rendering and image processing.  I made posts before about Ray Tracing, but the Turing architecture is so much more than just that.. which people don't seem to realize.

Nvidia unveils a new alternate shader pipeline with Turing

This series of GPU absolutely IS Nvidia's biggest and most significant architectural advancement since the introduction of programmable shaders.  If I wasn't convinced before, I'm convinced now. There's so much new technology in this GPU that it's kinda crazy that we're getting all of this at once.  Not only does the new family of GPUs have all this new Ray Tracing and AI technology, but the entire shader pipeline is undergoing changes that are going to help with massive amounts geometry that future games will have by reducing the amount of stress processing that amount of geometry puts on the CPU. 

Advanced Shading Technologies

 

Mesh Shaders:

The biggest change in the pipeline is what are called "Mesh shaders".  These introduce two new steps in the pipeline, a "Task Shader" and a "Mesh Shader".

Before, the CPU was responsible for determining LODs and Object List Processing, as well as the draw calls per object... then the Vertex Shaders would produce the polygons for those objects to either be tessellated or submitted directly to be rasterized and then pixel shaded.  Now instead of the CPU having to calculate LODs and Object List Processing, the task can become bundled as shader which can be efficiently parallelized on the GPU called a "Task Shader". This essentialy combines those steps of deciding what needs to be rendered or not.. and then passes that to the Mesh Shaders.  The Mesh Shader is then responsible for deciding the level of detail those objects should be rendered at.  Meaning how many polygons to generate/display for any given geometry.  Not only that.. but developers can segment objects into smaller meshes (called meshlets) and optimize the amount of verticies of each segment individually based on their needs.

 

 

So for example a large object with tons of individual parts connecting and moving.. like say a car engine.. can be segmented into these meshlets, and then the number of verticies (triangles) per meshlet can be optimized reducing the amount of data fetching required.  Essentially, they can process more geometry while at the same time reducing the bandwidth required to do so.

 

 

This is a HUGE advancement in the efficiency of the pipeline, and incredibly important in reducing the amount of work submitted and performed by the CPU... which is a BIG bottleneck on games with tons of unique geometry... which games of the future most certainly will be.

 

Variable Rate Shading:

 

The goal of this technology is to reduce the amount of unnecessary shading work done by the GPU that doesn't visibly improve the image quality of the final rendered frame.  Most of the time, there are various elements to a scene which don't need the same level of quality as other elements which are in focus.  There have been methods of reducing the work done by the GPU in this regard already.  As I said before that intelligent methods of rendering for VR would trickle over to standard gaming.  We've had multi resolution shading.. where the periphery of an image could be rendered at a fixed lower resolution in a fixed portion of the screen..improving performance.  But this new Variable Rate Shading method takes that same basic principle and makes it much more flexible.

 

 

VRS can now be fully dynamic.  Every 16x16 set of pixels can be shaded at different rates.  This means that they can take a scene, segment it into 16x16 pixel boxes, and then shade those pixels in any fashion they want completely separate from the others... A single triangle can actually be shaded using multiple rates across the grid.. This is some crazy shit!

 

The example they gave was Forza Horizon 3. (just an image not an actual functioning example) You can see the different colored segments which are shaded at different rates.. Important elements such as the Car, and the environment on the horizon appearing at full rate, while the sides of the roads and the road closest to the camera are shaded at a much lower rate due to the fact that they are high motion elements of the scene.    As the car slows down or stops, the scene dynamically adjusts the shading to full. This comes at no noticeable visual degredation to the player.

 

It's not just racing games.. but all games could utilize this for elements which don't require full shading for every pixel.

 

There's 3 notable algorithms for VRS:

 

Content Adaptive:  Which basically adjusts the shading rate of slowly changing detailed elements.  Say the sky, or walls, things of similar colors... or anything that basically doesn't change rapidly.  This can be used to tell the next frame that it doesn't have to be shaded at full resolution again because it already has relevant information from the previous frame. Saving performance.  Developers can essentially dictate tiny details that require full shading rate, and separate them from others which aren't as detailed and don't require it.

 

 

Motion Adaptive:  The way our eyes work and perceive motion is linear.. we see the same detail in motion as we do when objects are stationary.  Display technology doesn't work like that.  An image is scanned to the screen and updated at a variable/fixed rate.  As the image is drawn top to bottom, left to right... the object motion shifts causing blurring between frames.  The resolution of a display may be 4K or 1080p or 1440p.. but when it motion, the percieved resolution is lower.  Detail gets blurred the faster the motion... it's unavoidable.  Higher refresh-rates can help.. and black frame insertion can greatly improve this...  (You may have heard of LightBoost or ULMB) but it's never going to be perfect.  So in that case, it's wasteful to shade fast moving objects at full resolution.  This can be used in conjunction with Content Adaptive shading... which the Forza example above would make use of.

 

Foveated Rendering:  This is basically the example of what I said above where the periphery, or the outer elements of an image can be rendered at a lower resolution due to it not being in the focus center of the image.  Some VR games utilize this technology, and even though this mostly makes sense for VR, Shadow Warrior 2 actually makes use of this kind of technology already.

 

 

Developers are able to create their own algorithms which work best for their own use cases and can use the technology how they see fit.  Apparently Wolfenstein 2 will utilize Content Adaptive Shading and the press who saw a demo said it looked damned fine with no perceptual change in quality.  The demo let you toggle it on and off and Nvidia said it can boost frame rate by 20fps or more when targeting 60fps on a mainstream GPU.  Impressive stuff.

That's not it for Turing though, then there's the AI side of the GPU.. which isn't just about being wasted when Ray-tracing isn't involved...

 

NGX Technology (Neural Graphics Accelleration)

We already know that huge performance gains can be had by intelligently rendering computer graphics.  Whether it's checkerboard rendering, temporal injection.. whatever the case may be.  The goal is to improve performance by rendering less and intelligently determining the values of neighboring pixels without having to expensively render them.  In many cases, the visual reduction is negligible, especially when considering that the performance improvements can be massive.  Nobody will argue that these technologies will be important for the future.  AMD, Sony, and MS are working on their own solutions, developers have their own solutions.. and Nvidia of course, has solutions of their own.  All of these solutions can co-exist and regardless of how it happens or how it's implemented, the goal of increasing performance with as little impact to visual quality is universal.. and we should all be excited for what it means in the future.

 

Nvidia has a few new technologies for improving performance and image quality that they are introducing with Turing, which take advantage of their Neural Network services and utilizes thier efficient Tensor cores to accelerate those operations for the end user.

 

DLSS (Deep Learning SuperSampling)

 

DLSS actually has two different modes.  There's "DLSS" which is the standard mode which uses supercomputers to train neural networks how to generate a full target resolution image by rendering an image with half the pixels.  They train the network by gathering images of extremely high quality from developers for their games.  These "ground truth" images use 64x supersampling, which essentially creates an image with perfect detail and anti-aliasing quality. At the same time they take "raw" versons of those images actually rendered by the GPU at half the resolution, then run it through their supercomputer, asking it to match the target image.  They weigh the difference each time and the network adjusts its algorithm until it keeps getting closer and closer to the target image. Eventually it matches resulting in an image rendered at basically half resolution with presumably very little to no discernable difference from the full 4K resolution image with TAA.. and at the same time has the added benefit of reducing/eliminating the ghosting of TAA.

 

The second mode is "DLSS 2X" which instead of having the goal to improve performance, is meant to purely improve image quality further.  This mode has the GPU render the full resolution image and then combines that with DLSS to produce an image which is essentially a match for those high sampled 64xSS images.  So this is like rendering with 64x the samples meaning that the ghosting and loss of detail with TAA is virtually non existent.  TAA blurs when in movement because it utilizes the motion vectors of the moving objects from the previous frame.  This causes high movement scenes to lose detail.  DLSS 2X eliminates that and avoids the issue altogether.  It provides a much more temporally stable image while retaining clarity.

 

 

DLSS is gaining a fair bit of traction.  With over 25 confirmed games already and Nvidia has stated that it will be possible for previously released games to implement this technology relatively quickly.  It remains to be seen just how easy Nvidia can make this for developers... but considering the fact that most of the heavy work is done by Nvidia's super computers and their drivers.. and that it's completely free, means it has a good chance at being highly adopted.  It's not something only big AAA developers have the capability to implement.. thankfully.

 

So you have these new technologies, along with the fact that the CUDA shaders are ~50% more performant than the Pascal architecture, and memory bandwidth has been improved by ~50% as well, means that I think Nvidia was quite right to introduce RT cores and AI cores onto the chip and make up that difference and more with advancements in their shading architecture and memory subsystem.

 

Yes, performance takes a massive hit with Ray Tracing.. but the technology is undeniably pushing real-time graphics forward.. allowing things never before possible.  The thing is,  Nvidia didn't skimp on general raster performance just to introduce this.  The new GPUs are still massive leaps over their predecessors in standard raster graphics.

 

This just isn't possible elsewhere atm.  MS and Sony are undoubtedly working on this stuff too.  I think we could all be surprised where this goes.

 

 

 

 

[-GD-X★ 7,738]

i didn't read any of it, but it seems like a godly thread  

[kaz 2,439]

i need more pixel

[kaz 2,439]

i didn't read any of it either, looks boring as hell

[kaz 2,439]

yea yea maybe i read it later

[-GD-X★ 7,738]

Just now, kaz said:

yea yea maybe i read it later

same. it looks informative. my adhd is through the roof today. wait, what thread am i in?

[-GD-X★ 7,738]

the foveated rendering and dlss seem pretty awesome, and clever ways to boost visuals.

[Remij 4,679]

7 minutes ago, -GD- said:

the foveated rendering and dlss seem pretty awesome, and clever ways to boost visuals.

Yea.  Everything here is extremely clever.  Developers and these scientists that create this shit are fucking real life wizards.. seriously.  

[-GD-X★ 7,738]

Just now, Remij_ said:

Yea.  Everything here is extremely clever.  Developers and these scientists that create this shit are fucking real life wizards.. seriously.  

for real! i have a lot of respect for them. hopefully, these techniques can be incorporated into programs like cinema4d. having to use a render farm blows...

[Remij 4,679]

7 minutes ago, -GD- said:

for real! i have a lot of respect for them. hopefully, these techniques can be incorporated into programs like cinema4d. having to use a render farm blows...

Cinema4D will undoubtedly have plugins to take advantage of RTX.  

[Remij 4,679]

Here's a video showing the Variable Rate Shading in action... VERY cool tech.

 

 

One of the cool things about this as well is that while the rate of the shading changes, rasterization is still full resolution meaning polygon edges still stay sharp.  So in motion the difference is essentially unnoticeable.

 

 

[Remij 4,679]

Here's the new shader pipeline in action... holu FUCK!  

 

Standard

 

Mesh Shader

 

Standard

48fps

Asteriods: 253,153

Drawn Triangles: 93,126,232

Max LOD Triangles: 2,788,080,746,496

 

Mesh Shader

51fps

Asteriods: 253,153

Drawn Triangles: 34,091,756

Max LOD Triangles: 2,788,080,746,496

 

So the same number of asteroids, almost 1/3 of the triangles... and it looks better while running a few fps faster... like FUCK.  They could literally throw 3x more objects on the screen and keep the same fps!  

[-GD-X★ 7,738]

that mesh shader is really cool. i think the engine that really blew me away was the id tech 4. the way normal mapping and shadows were done - mindblowing.  then the megatexture with rage was sick!

[The Mother Fucker 26]

Non gamers like me want to know how many MH /s can it produce?