There are two steps in the task of hw assisted video playback: 1) video decoding, i.e. going from compressed stream to uncompressed, offscreen video frame, and 2) compositing video with all the other surfaces the video playback application uses -- the decoded frames are usually put into OpenGL (or Vulkan, whatever) texture and then composed with the rest.
When you do 1) on the GPU (well, not exactly GPU, but video decode block on the videocard, but that's not really important now), you end up with decoded frame in VRAM. Reading it back to system RAM, just to push it back to VRAM elsewhere is expensive (if the graphic is not UMA, then you go via PCIE back and forth) and unnecessary in the end, it is more efficient to have some way to directly share content from 1) into 2) already in VRAM.
For that, both subsystems must support some way of sharing memory buffers. VA-API (for 1) and Mesa (for 2) support DMA-BUF, so that's the reason why it is used here.
Though Vulkan has video decoding built in, so eventually everything will hopefully unify to use that and we can avoid needing multiple separate subsystems depending on vendor...
It looks like [0] there currently exist extensions for H.264 and H.265. I don't know anything about Khronos politics, but I expect there's nothing that would stop e.g. Google from proposing similar extensions for VP9, AV1, etc., besides the need to actually get it implemented. (The H.264 and H.265 extensions have only AMD, Intel, and NVIDIA employees listed as Contributors.)
Video playback is accelerated in many apps, for example VLC 3.x defaults to it.
In web browsers, things are more delicate because videos live within the complex context that is a webpage -with its own layer of hardware acceleration!-, and failures can be brutal depending on your browser, hardware and driver quality (crashes, video corruption, etc).
The pain-in-the-ass-ness to enable it on a given browser varies with time and browser/OS and along regressions, stack changes, and maintainer priorities. From personal experience during the past years, it has never been great on Linux:
- In Firefox, regressions have been frequent.
- In Chrome, the feature is here but Google disables it at build time in their official builds (my personal intuition is that they do enable it for their Chromebooks, but they don't want to support the wild west of broader Linux configuration and old/broken driver fun). However, out of official Chrome builds, several distributions builds (Arch, Debian) of Chromium now enable it at build time, so it's possible to try to use it.
So, it's finnicky to enable, but sometimes feasible, until it's no longer :D . To give it a try on a Linux box, the well-maintained Arch wiki is what you want (and these sections are mostly not specific to Arch):
Bang on! This just summarizes the experience of browser video hardware acceleration on Linux. Marred with bugs and incompatibility issues. Regressions are the norm for Firefox in this regard according to my experience using an Intel card.
Since announcing support for HW video initially on Wayland, Firefox has been off to the races on this [0]. Despite requiring lots of hacks to enable flags as you linked on the Arch Wiki, one of which now suggests disabling the sandbox for content processes and a major red flag, it somehow worked.
For one reason or another [1], Firefox chose to lock the video drivers to the old i965 drivers instead of choosing the newer iHD driver. A weird decision that still applies to current builds.
There have been incompatibility issues with video codecs. In my experience I can play h264 videos with hardware acceleration but not vp9 with the flatpak distribution. With vanilla package, both h264 and vp9 video is hw accelerated.
I could also have sworn that video acceleration support has gotten worse over time since the introduction.
I now get a weird artifact at the bottom half of the screen only when watching YouTube videos. [2]
Video hardware acceleration is not entirely doomed on Linux. Even if Chromium does not want to make any move toward this direction in this century. Gnome web or rather Epiphany has stellar video acceleration support by using Gstreamer plugins. So it is possible to get hw video working on a browser.
