On: AMD's MI300X Outperforms Nvidia's H100 for LLM Inf...

[logicchains]

That's a pretty big claim, that Microsoft and Meta have their own proprietary cuda-replacement stack. Do you have any evidence for that claim?

[versteegen → logicchains]

I'm guessing what they meant is that they use toolchains that are retargetable to other GPUs (and typically compile down to PTX (nVidia assembly language) on nVidia GPUs rather than go through CUDA source -- GCC and clang can both target PTX). For example XLA and most SYSCL toolchains support much more than nVidia.

[sigmoid10 → versteegen]

Even then it's an insanely bold assumption that a company other than Nvidia could build a better framework than CUDA for compiling PTX. Especially since CUDA is already so much C-like. I've never seen anyone go deeper than that outside of academia.

[dexterac → sigmoid10]

How many customers/consumers will care about services are be built with CUDA?

If they need a ChatBot that uses a model with same accuracy and performance as on non-CUDA hardware, would they still want CUDA based hardware?

[sigmoid10 → dexterac]

Who is going to build the architecture and compile the device specific kernels? You have to pay those people as well and you can save tons of money and time if you do it with cuda.