Nvidia introduced a new texture compression method this week that offers 4x higher resolution than traditional block truncation coding (BTC, BC) methods while having similar storage requirements. The core concept of the proposed approach is to compress multiple material textures and their mipmap chains together and decompress them using a neural network trained for a specific decompression pattern. In theory, this method could also impact future GPU architectures. However, for now, this method has limitations.
new requirements
Recent advances in real-time rendering for video games have enabled the use of techniques such as physically-based shading, ray tracing, path tracing for photorealistic modeling of materials, and denoising for accurate global illumination. Approaching cinematic visual quality. Texturing technology, on the other hand, hasn’t really progressed at the same pace, as texture compression techniques have remained essentially the same as they were in his late 1990s.
This is because GPUs still rely on block-based texture compression methods. These techniques have highly efficient hardware implementations (because the fixed-function hardware that supports them has evolved over 20 years), random access, data locality, and near-lossless quality. However, they are designed for moderate compression ratios between 4x and 8x and are limited to a maximum of 4 channels. Modern real-time renderers often require more material properties, requiring multiple textures.
Nvidia way
Here is Nvidia Random Access Neural Compression of Material Textures (opens in new tab) (NTC) will appear. Nvidia’s technology allows for two additional levels of detail (16x more texels, or 4x the resolution) while maintaining storage requirements similar to traditional texture compression methods. This means you can now run optimized compressed textures per material at resolutions up to 8192 x 8192 (8K).
To do so, NTC takes advantage of redundancy spatially, between mipmap levels, and between different material channels. This preserves texture detail when the viewer is close to the object. This is not possible with modern methods.
Nvidia claims that NTC textures are decompressed using matrix multiplication hardware such as tensor cores working in SIMD cooperative fashion. That means the new technology doesn’t require any special purpose hardware and can be used with virtually any modern Nvidia GPU of his. But perhaps the biggest concern is that every texture must decompress its own optimized neural network. This places an additional burden on game developers.
According to Nvidia, the texture quality obtained at these aggressively low bitrates is said to match or even exceed modern image compression standards such as AVIF and JPEG XL. These standards are not designed for real-time decompression with random access anyway.
Practical Advantages and Disadvantages
In fact, the images shown by Nvidia clearly show that NTC is superior to traditional block coding-based technologies. However, Nvidia admits that its method is slower than traditional methods (1.15ms on GPU to render a 4K image with NTC textures, 1.15ms to render a 4K image with BC textures). It took 0.49 ms), giving 16 times more texels despite using stochastic filtering. .
Although NTC is more resource intensive than traditional hardware-accelerated texture filtering, the results show that NTC provides high performance and is suitable for real-time rendering. Additionally, in complex scenes using a full-featured renderer, the ability of the GPU to hide latency allows the cost of NTC to be partially offset by concurrent execution of other tasks (such as ray tracing).
Rendering with NTC, on the other hand, can be faster with newer hardware architectures, more dedicated matrix multiplication units that may be used, larger cache sizes, and register usage. In fact, some optimizations can be done at the programmable level.
Nvidia also notes that NTC is not a completely lossless texture compression method, causes visual degradation at low bitrates, and has several limitations, including sensitivity to channel correlation, uniform resolution requirements, and limited benefit at long camera distances. We do admit that there are some limitations. Additionally, the benefits are proportional to the number of channels and may not be as significant with fewer channels. Also, NTC is optimized for material textures and always decompresses all material channels, so it may not be suitable for use in various rendering contexts.
The advantage of NTC is that it does not use fixed-function texture filtering hardware to produce good results, but this is also an important drawback. The cost of texture filtering is computationally expensive, so anisotropic filtering with NTC is currently not suitable for real-time rendering. On the other hand, probabilistic filtering can introduce flickering.
However, despite its limitations, NTC’s multiple channels and mipmap-level compression produce results that exceed industry standards. Nvidia researchers believe the approach paves the way for cinema-quality visuals with real-time rendering, making it practical for memory-constrained graphics applications. However, it has a small timing overhead compared to his simple BTC algorithm, which impacts performance.
