OpenCL is an alternative to CUDA just like Legos are an alternative to bricks. The problem with OpenCL isn't even the performance, it's everything. If OpenCL were any good, people could use it to build similarly powerful applications on cheaper AMD GPUs.
OpenCL is just a spec. It's up to companies to implement it in a successful way or not. There is no reason in and of itself that OpenCL can't compete with CUDA on performance. The fact that Apple's Metal, which is pretty good, is actually implemented with a private OpenCL system is proof that the spec is not to blame.
Well, performance isn't the issue. Like the parent said, the problem is mostly that CUDA is such a radically mature API and everything else isn't. You might be able to reimplement all the PCIe operations using OpenCL, but how long would that take and who would sponsor the work? Nvidia simply does it, because they know the work is valuable to their customers and will add to their vertical integration.
OpenCL isn't bad, and I'd love to see it get to the point where it competes with CUDA as originally intended. The peaceable sentiment of "let's work together to kill the big bad demon" seems to be dead today, though. Everyone would rather sell their own CUDA-killer than work together to defeat it.
For similar reasons why Microsoft created DirectX. It allows them to have a system where the software and hardware are more tightly integrated than using a cross-platform spec. It also allows them to situate the API within the context of the operating system and other languages that are used on Apple platforms, making things easier on developers. And at least in that regard, they certainly succeeded. Metal is probably the easiest GPU API offered on any platform. Not necessarily the most powerful, but it’s almost trivial to hand write a compute kernel and spin it off.
AMD’s first attempt at displacing CUDA-as-a-runtime was called AMD APP (advanced parallel processing) so this layer did indeed exist. It just sucked as badly as the rest of AMD’s slop.
Bonus points: the rest of the software libraries intended to compete with the CUDA ecosystem are still online in the “HSA Toolkit” GitHub repo. Here’s their counterpart to the Thrust library (last updated 10 years ago):
