Carzy Zarx’s PC Gaming Emporium - Catch Up on All the Latest PC Gaming Related News

[JEMC]

NVIDIA Might End the ‘CUDA Moat’ With Its Latest Update, Says Chip Architect Jim Keller, Who Claims It Will Now Be Easier to Port Kernels
https://wccftech.com/nvidia-might-end-the-cuda-moat-with-its-latest-update-says-jim-keller/
(...)

"It would not be incorrect to say that, so far, with the 'AI frenzy', CUDA has managed to hold everything for NVIDIA when it comes to providing developers with specific libraries and frameworks for creating AI-infused workflows. To the best of our knowledge, no other company has successfully replicated such a software stack. Building upon this, NVIDIA has introduced a major update to CUDA, called CUDA Tile, which is a shift from the traditional SIMT to a tile-based approach. We'll discuss the update in-depth later on, but the chip architect Jim Keller believes that it might mark the end of the CUDA moat.

Now, before this update, CUDA gave the leverage to programmers when it comes to fine-tuning parameters like tile sizes, data to load in shared memory, and the compute resources required for execution on the GPU, but with CUDA Tile, the dynamics completely change. NVIDIA has introduced a tile-based programming model and a new low-level VM called Tile IR, which views the GPU as a tile processor. This allows programmers to focus more on core logic, rather than GPU complexities."

(...)

"One of the reasons why Jim Keller believes that CUDA Tile will make porting the code onto other GPUs, such as those from AMD, much easier is that the tiling approach has been widely common in the industry, being used by frameworks such as Triton, so it ultimately raises the possibility that porting CUDA code to Triton, then to AMD's AI chips, will become a lot more easier. Secondly, when you raise the abstraction level, developers wouldn't need to worry about writing architecture-specific CUDA code anymore, so it is supposed that code porting would become a lot easier.

However, building on this argument, CUDA Tile actually strengthens NVIDIA's moat, in my opinion, mainly because the proprietary technology behind it, such as Tile IR, is optimized for NVIDIA hardware semantics. Hence, while porting might become easier, implementation would remain complex. Ultimately, by making CUDA programming easier, NVIDIA is actually consolidating its control over the CUDA stack, which is one of the reasons why the newest update is being touted as a 'revolution' in the realm of GPU programming."

On one hand, I trust Keller's knowledge on the matter more than a journalist from a site that happily posts all kinds of rumors. But on the other hand, Keller's own company, Tenstorrent, is going after Nvidia's AI pie, so he has an ulterior motive to say that.