Syllogi GrafiKon

Introduction This is an implementation of Neural Texture Compression (NTC) which is based on the NVIDIA research paper: Random-Access Neural Compression of Material Textures. I was at the point where I wanted to do texture compression for my custom engine, and after skimming through this research, I was impressed with the compression-quality results. So I went down the rabbit hole of machine learning. This post will serve as extended documentation of MetalNTC. It focuses on the details that decide performance and visual fidelity when NTC has to work inside a Metal renderer. I will discuss positives/negatives and fail cases, as well as use cases of NTC, since for now it can’t entirely replace traditional texture sampling. Moreover, we will see how much the newly added neural accelerators on M5 chip help with real-time inference and latency hiding as well as how NTC can provide value even on an M1 air. ...

Introduction I’ll walk through my implementation of the Single Pass Downsampler (SPD) algorithm for Apple’s Metal API. SPD is an algorithm for generating mipmaps in a single compute shader pass, as the FidelityFX SPD by AMD. Full source code at the end of the post. What is a Single Pass Downsampler? Traditional mipmap generation requires multiple passes, where each mip level is computed in a separate dispatch with synchronization points between them. This means the GPU must wait for each level to complete, write results to device memory, and then read them back for the next. This results in both memory bandwidth costs and CPU-GPU synchronization overhead from multiple API calls. SPD eliminates this overhead by using threadgroup memory to keep intermediate downsampling results local to the compute threadgroup, allowing all mip levels to be generated without round trips to device memory. SPD can be generalized to be used for any texture, however I implemented it to generate a depth pyramid for occlusion culling. ...