Glad to see more support for OpenGL, but I really hope we'll soon move to a compute-only way of handling graphics. The overhead of vulkan is absolutely insane (and not warranted, in my opinion), and OpenGL is on its last legs.
Things like Nanite spark a little hope, since they've shown that software-rasterization via compute can be faster than the standard graphics pipeline with the hardware rasterizer for small, dense triangles. Seems like a matter of time until everything goes compute, even large triangles. Maybe the recent addition of work-graphs in DirectX is one step in that direction?
> Vulkan and OpenGL both already support mesh shaders, which is a compute-oriented alternative to the traditional rasterization pipeline.
Mesh shaders are a step in the right direction, but they are still embedded in all that unnecessary Vulkan fluff. I would want these things in CUDA because it does the opposite approach of Vulkan - it makes the common things easy, and the hard/powerful things optional. Just let me draw things directly in CUDA, and maybe give access to the hardware rasterizer via a CUDA call.
Yes, sorry for the confusion. I'd just rather have an API where the common things are easy, and the super powerful low-level optimizations are optional.
except the easy common things are very slow, and the optimizations which are basically required become a careful dance of understanding the underlying hardware interface anyway (without any real documentation) and figuring out how to map that onto the high-level interface.
What happened is that Khronos Group banked on third parties writing OpenGL-esque "middleware libraries" that bridged the gap between the hopelessly complex Vulkan API and people who want to write simple 3D graphics. Unfortunately, Khronos learnt the hard way that you can't just expect people to do work for you without a sufficient incentive, since after all, OpenGL still exists for anyone who wishes to have an easy API for graphics programming, and anyone who doesn't want to do that just uses an engine or high-level graphics API.
Of course I'm not saying that these libraries don't exist, but they haven't taken off in the way they expected
You could always just use a library that simplifies Vulkan or keep some skeleton apps around. When you're past all of the verbose set up, the actual meat of it isn't too bad.
* ... a compute-oriented continuation of the practice where they take the average of Nvidia and AMD's implementations, and then force the developer to deal with the overhead of harmonizing the details without going insane.
GPU compute is awful, almost as bad as GPU rendering.
I know it won't happen overnight, but since it's already possible to do software rasterization for small triangles faster than hardware, having a graphics API framework starts losing its purpose. After all, we want to have the detail of small triangles anyway. Just let us draw to the screen in CUDA without the need for OpenGL/Vulkan interop, and I believe we'll soon see a shift to serious compute-based real-time rendering.
Basically, instead of graphics being a framework, I want graphics to be a straightforward library you include and use in your CUDA/HIP/SYCL/OpenCL code.
> let us draw to the screen in CUDA without the need for OpenGL/Vulkan interop
how would that work? like GPU frameworks would just be compute (like cuda) and some small component of it would just allow to write the end result to a buffer which would be displayed or smth?
Exactly. Things like that already work with a workaround: You can use Cuda-OpenGL interop to expose an OpenGL framebuffer in CUDA, then you can simply write into that framebuffer from your CUDA kernel, and afterwards you get back to OpenGL to display it on screen. Just directly integrate that functionality in CUDA by providing a CUDA native framebuffer and a present(buffer) or buffer swap functionality.
Only if you take a very narrow view of the gaming industry that only includes high end consoles and PCs.
The Nintendo Switch and PCs support OpenGL and Vulkan. But those are both dwarfed by mobile gaming which I think makes up more than half of the entire gaming market, and both iPhone and Android support OpenGL. Although I don't know if the majority of mobile games use it or not.
While switch supports OpenGL and Vulkan, in practice very little games use it. The native API on Switch is a proprietary API called NVN.
OpenGL on Apple is on life support to support existing software, Apple platforms are all in on Metal now.
The only platforms where OpenGL and Vulkan are first class citizens is Linux, Android and Windows (barely). And Vulkan is still second fiddle to DirectX on Windows.
All fair points. However, I was taking issue specifically with the claim that the "Gaming industry is mostly using DirectX or Gnm/PSSL".
It would be hard to quantify exactly how much of it is using OpenGL, as a percentage.
However, for a very rough upper bound I quickly looked up these:
* Mobile gaming apparently makes 52% of the entire gaming market in 2023
* Android is 78% of mobile gaming
That could mean OpenGL makes up to 40% of the entire gaming market just from Android.
Although I have no idea what percentage of Android games use OpenGL and wouldn't know how to look that up. I also don't know how accurate those percentages are. I got them from brief Google queries and got both the 52% and 78% figures from the featured snippet.
But even if that 40% is much too high, we still need to add switch and PC games and I would find a figure of 30% of the gaming industry using openGL to be believable. That would put OpenGL as the largest API which I didn't expect, even as a possibility.
Just to reiterate on this point: Pretty much every Vulkan game on Windows forgoes using Vulkan Swapchains in favor of presenting from compute and using native DXGI Swapchains. If you are shipping a production quality game or application, you really want to use the platforms native APIs to get the full capabilities.
I meant the development/learning overhead. With Vulkan you can do incredible low-level optimizations to squeeze every last bit of performance out of your 3D application, but because you are basically mandated to do it that way, you have a very harsh learning curve and need lots of code for the simplest tasks. I'd rather prefer approaches that make the common things that everyone wants to do easy (draw your first simple scenes), and then optionally gives you all the features to sqeeze out performance where you really need to. Because at least for me, I don't work with massive scenes with millions of instances of thousands of different objects. I do real-time graphics research, mostly with compute, and I'd just like to present the triangles I've created or the framebuffers I created via compute (like shadertoy).
I'll readily admit, I'm neither smart nor patient enough for Vulkan so I quickly gave up and learned CUDA instead, because it was way easier to write a naive software-rasterizer for triangles in CUDA, than it was to combine a compute shader and a vertex+fragment shader in Vulkan. I'm just rendering a single buffer with ~100k compute-generated triangles, and learning Vulkan for that just wasn't worth it.
Sure, the implementations today are thin layers over D3D12, Vulkan and Mantel. But it is a proper API, cross platform and without the implementation overhead.
Next time I need to draw some graphics I will definitely use WebGPU instead of OpenGL or Vulkan.
PS. Don't let the name fool you. It is a proper render API, Mozilla has written their implementation in rust and Google in C++.
Would not be surprised if WebGPU do get native support by the drivers in the future.
Yeah, but OpenGL doesn't get updates anymore. My timeline goes like: I needed pointers(and pointer casting) for my compute shaders so I checked the corresponding GLSL extension, which was only available in Vulkan so I tried switching from OpenGL to Vulkan. After a week I gave up - the pointer/Buffer reference extension did not look promising anyway - and I tried out CUDA instead. That's when I found out that CUDA is the greatest shit ever. That's what I want graphics programming to be like. Since then I just render all the triangles and lines in CUDA, because it easily handles hundreds of thousands of them in real-time with a naive, unoptimized software-rasterizer, and that's all I need. In addition to the billion points you can also render in real-time in CUDA with atomics.
> That's when I found out that CUDA is the greatest shit ever. That's what I want graphics programming to be like.
Something which requires you to buy new hardware from a specific brand, and load an out-of-tree binary-only module on your kernel? That's not what I want graphics programming to be like.
The Vulkan API might be clunkier (I don't know, I haven't looked at the CUDA API, since I don't have the required hardware), but at least it can work everywhere.
It's the exact reason why I've avoided CUDA for years, but I hit a dead end with OpenGL and Vulkan, and CUDA happened to be a fantastic, easy and fast solution. Of course I don't want graphics programming to be NVIDIA-only, but I want it to be like CUDA, just for all platforms.
I'm confused by what you mean here by "software-rasterizer" when you are compiling it to run on the GPU. What do you think the graphics driver is doing when you program it with OpenGL? It's doing more or less the same thing under the hood.
I guess you can technically call that software rasterization now that GPUs are very programmable. But it's not how the word is usually used.
Not necessarily. I'm working on an application that does exactly that, as we are rendering spheres. It ends up being much more efficient to rasterize the sphere in a combination of vertex and fragment shaders than to instantiate triangles.
It's not just micro-triangles, it works for triangles that span multiple pixels. And that's not a special case, that's the standard nowadays, except for games targeting very low-end devices.
The dedicated hardware is nice for general-purpose support for arbitrary triangle-soups. But if you structure triangles a certain way and have a certain amount of density (which you want for modern games), you can specifically optimize for that and beat the general-purpose hardware rasterizer.
I mean, about 5 years ago there were many folks that were trying to do raytracing using GPU compute rather than the current methods, it was essentially treating the CPU as a giant parallel software rendered. The results were pretty good even then.
The standard rendering pipeline is still fairly fixed and mandates shaders for vertices and fragments. With software rasterization, there are no more vertex or fragment shader, only compute. And you don't use the hardware rasterization units - you rasterize the triangles yourself and write the results to the framebuffers with atomic-min/max operations. You decide for yourself how and when you compute the shading in your compute shader. This can be multiple times faster for small triangles, and 10-100 times faster for points. And once you do things that way, there isn't much point for graphics APIs anymore - everything is just buffers and functions that process them.
Why is there a pipeline at all? Give me a framebuffer, a place to upload some CUDA or equivalent, and some dumb data pipe between the host program and GPU program.
But that is one generation of GPUs away when they will make some computational shortcut for small triangles, which to me seems to be rather trivial to implement.
Not really trivial if you have to support any input set of triangles and don't know much about them. Software rasterizion exploits things like localized chunks of triangles, but the hardware rasterizer does not know about that in advance. Also, these kinds of software rasterization algorithms add limitations, which aren't much of an issue with your own rendering pipeline that specifically knows how to deal with these limitations and works with them.
For small triangles, they use a software rasterizer into the V-buffer. Obviously for 1px triangles since they don't want to waste quad overdraw, but I think they found it's faster up to 12px/tri or so on AMD.
Part of Nanite is software rasterization by rendering triangles with 64 bit atomics. You can simply draw the closest fragment of a triangle to screen via atomicMin(framebuffer[pixelID], (depth << 32) | triangleData).
No, not even close. On Intel in some chipset(s) Vulkan is not as performant as OpenGL 4.6.
Gallium3D with Iris it's the future against the old pipeline, tho. For instance, I get OpenGL 2.1 in my netbook with the new drivers, 1.4 with the old ones instead.
The big players will probably switch once we get to the point that nothing still running on OpenGL needs optimally efficient usage of the GPU (either because it's old or because it was never performance-critical to start with, such as a student project).
I don't mean to detract from the technical merits of this at all, but for those not aware you might be interested to know that ImaginationTech is a Chinese-owned firm.
It's nice to see Chinese tech embrace existing standards.
There are a lot of chips that are very poorly documented and supported by abysmal proprietary SDK. The software part of hardware is so often an afterthought yet it is also what makes or breaks the product.
Is ImgTec back as a real going concern? Last I heard (back in 2016 or so) they'd basically lost the Apple GPU work (Apple opened up an office one town across, poached a load of engineers and bought the design work in house), and since then have been desperately trying to pivot to vision or VR of AI, but I've not heard of them having much traction.
They're closed source. Just parts of the driver are opensource.
Note that these GPU's are widely used in (mostly low end) phones, and I think power the UI in some other household items with screens. They provided iPhone GPU's till 2017 (and there is debate if Apples new GPU might contain some imagination tech)
Crazy is that they were aggressively anti-foss even with their drivers not too long ago. Intel graphics usually has good Linux support, except for GMA500 which was based on Imagination tech. N900 was the Linux darling phone, and the graphics drivers where god-awful buggy blob from Imagination. And so on..
For a company who historically has mostly made GPUs for phones and set-top boxes, I don't think that's true. People don't usually program these devices at this level. Especially in the case of all the hardware they did for Apple, no requirements to be open there.
Maybe these days that's changing but it's a big change for a company like this to embrace.
Oomph-wise, they claim a good 4x over what the Raspberry Pi 4/400 have. Note that rPi's is known to be anemic due to serious memory bandwidth bottlenecks, barely able to keep up with filling a 1080p screen.
Currently they do not support the specific variant used in JH7110, but they seem to both be variants of the same architecture.
JH7110 has the BXE-4-32MC1[0], whereas the driver currently supports the BXS-4-64-MC1[1].
No idea about compute, and AIUI openCL suport in mesa3d is still a disaster for all drivers.
Things like Nanite spark a little hope, since they've shown that software-rasterization via compute can be faster than the standard graphics pipeline with the hardware rasterizer for small, dense triangles. Seems like a matter of time until everything goes compute, even large triangles. Maybe the recent addition of work-graphs in DirectX is one step in that direction?