The X protocol is from a time before proprietary/differentiated GPU devices. Network transparency looks very different when you're doing simple stuff like "render a line" or "render a box". Nowadays even 2D applications are essentially sending arbitrary programs to a standalone accelerator (the GPU) which they expect to have incredible bandwidth.
In the simple case of the past, it made sense to send the primitives. In the complex now it's way more robust to stuff the bitmap over the wire.
Much of the reason RDP and WebGL work so well is that they send the raw command stream/assets to the rendering side rather than continuously sending a stream of compressed frame buffer data. RDP, for example, can send the raw video stream being played, or recompress it, and send it rather than trying to render it to the screen and then recompress the resulting output, which is what you get with something like VNC. Thats why its completly possible to stream video over RDP on links that choke up pretty much everything else.
For an incredibly wide range of things, high level GUI command streams are going to be significantly less data intensive than the resulting rendered and compressed video stream. AKA the GUI command stream is itself an applicaiton specific compresion algorithm. A draw line command can affect tens of thousands of pixels around the line due to antialiasing, and that won't ever compress as well as the drawing (x,y,brush). So while sending a bunch of textures, shaders, and the like might be a huge initial overhead, its going to quickly pay for itself over the lifetime as used in something like a game/etc, particularly at high resolution and refresh rates.
Never mind, of course, the overhead of doing a bunch of local rendering, compressing it, sending it over the wire, and doing video playback. If it weren't for hardware video encoding/decoding, it would essentially be a case of pegged CPUs on both sides just doing graphical updates.
This may just be one of the issues with Wayland compositing; over time, I've become more convinced that the loss of standard GUI widgets and a serializable drawing stream might be a huge mistake.
And we do have hardware encoding / decoding. The latency is quite reasonable, like 15ms in total, so modern solutions like parsec work for any app, including games and CAD.
That stuff was just appearing when I last worked on an RDP client for a mobile platform. At the time, it depended on the application/client region and the bandwidth/capability requests from the client. AKA, it would kick in for an application that was just dumping a predrawn buffer but not for applications that were still using the Win32 API extensively unless there were additional graphical operations going on (e.g., transparent windows).
And that is part of the fun of RDP, it has an immense number of different ways of accomplishing the end goal, and newer methods were being added to compensate for the way applications themselves were being written while the older ones remained. It was a long road of dealing with edge cases, with certain applications not rendering properly.
AKA, use a 3rd party library to draw all the GUI widgets and windows didn't have any alternative but to come up with a better way to compress and send those updates. And the last time I ran an RDP client to Windows, this stuff was still visible; older win32 applications were fast/responsive, and things like Firefox would just chug every time it hit a web page that was doing any kind of dynamic content (think scrolling an ad).
There are two seemingly irreconcilable camps at this point as to whether X itself is fundamentally sound technology for remote display or any kind of display technology going into the future.
There are many reasons hashed out to death about this. Yes you could continue to add new X extensions, and the second system effect is a real consideration, but neither of those are absolute arguments against starting fresh without legacy baggage both in design and implementation.
X earned its place in the UHH over 30 years ago, at least let’s not pretend this was some universally beloved technology either.
> There are two seemingly irreconcilable camps at this point as to whether X itself is fundamentally sound technology for remote display or any kind of display technology going into the future.
Are there? I don't think anyone seriously thinks X is the way forward do they? They just don't like some of Wayland's poor choices, which I think is fair.
>I don't think anyone seriously thinks X is the way forward do they?
I think "forward" is the wrong mindset here. X works. Plenty of tooling is just done, lots of people have their workflow just perfect, the whole ecosystem is mature, and at worst the Wayland devs decided to uproot the whole thing in the name of "progress".
Wayland could be nice, but its compositors require 10x the code that a WM would (which is why there aren't that many Wayland "WMs") and its supposed benefits aren't better enough to justify the ambitious rewrite. I could imagine a universe where Wayland did the minimum to replace X's jank and make the thing maintainable, and nobody would have objected to that IMO.
Those goddamn Wayland devs and their individual free will. How dare they not continue to maintain this miserable, crusty old spec and continue to make their marketable skills less valuable.
> Wayland could be nice, but its compositors require 10x the code that a WM would (which is why there aren't that many Wayland "WMs"
Nah, this is an ecosystem maturity problem that is improving rapidly.
There will always be a few hold outs for the old way, but people will and are moving on.
It’s silly to compare the number of anything with a project that had a 30 year head start.
> Wayland did the minimum to replace X's jank and make the thing maintainable
That’s called Xorg. It still exists and you are free to do whatever you wish, even invest in its development,
Wayland didn’t “uproot” anything.
One of the biggest pieces of “jank” replaced is the nonexistent security model. You can research why the proposed SECURITY extension was unworkable and smarter people decided it was better to start from scratch. This benefit alone is better enough for most people with a stake to justify a rewrite.
In pure economic terms there is certainly a price where Xorg can continue to remain viable - this isn’t some ancient artifact lost forever. In this lens it’s hard not to see these comments as anything but unproductive bitterness at not being able to provide or raise these funds. It doesn’t help that part of this price is a direct result of the qualities of the thing you’re trying to save.
>Those goddamn Wayland devs and their individual free will.
That's not the problem. The problem is all the heckling for everyone to switch to Wayland, and to make it default. And also pretending that reversing the "mechanism, not policy" wasn't a fundamental change of philosophy and was just "progress".
And to be clear, by "at worst" I meant "this is the least-charitable interpretation".
>Nah, this is an ecosystem maturity problem that is improving rapidly.
TinyWM is 50LOC and wlroots's TinyWL is 900LOC. This hasn't changed. Writing Wayland compositors is a pain in the ass compared to WMs.
Wayland started in 2008 IIRC, so here in 2024 Wayland is 16 years old. It doesn't have teething issues, it just has issues.
Plenty of WM makers have just straight-up said they won't ever port their WM to Wayland (so talking about time and how "X is older" is irrelevant here), because Wayland's opinionation breaks too many things.
>One of the biggest pieces of “jank” replaced is the nonexistent security model.
You mean like how X clients can read keyboard inputs? There's so much FUD around this. /dev/keyboard does that already, you need to sandbox every app anyway - at which point your sandbox should just interdict the X interface. Security is the worst argument. Wayland isn't necessary for security, and for the longest time it's been locking the door and leaving the window open. Portals.
> The problem is all the heckling for everyone to switch to Wayland
No one’s making you switch and Xorg is as open as it ever was. X is dying, slowly, that’s just a simple fact of the universe.
> Plenty of WM makers have just straight-up said they won't ever port their WM to Wayland
There are plenty of up and coming WM makers willing to produce suitable stacking and tiling WM/compositors. Active projects exist today.
> You mean like how X clients can read keyboard inputs?
No.. how every X client has global access to all of the X state of every client and how the only response to this are extensions that no one really uses for well established reasons.
> you need to sandbox every app anyway - at which point your sandbox should just interdict the X interface.
As someone who actually researched X I don't really see the problems with it. Xorg has some, but the X11 protocol seems to have fewer problems than the Wayland protocol.
For instance it already supports mixing windows with different colour modes, which can be used for HDR.
X11 the base protocol is wholly unsuited to the needs of modern display systems, full stop. Moreover, the base security profile is also completely outdated. Now you can always extend the protocol, and continue to use the myriad extensions over the years like XRENDER, DBE, DnD, etc, etc (or paper over deficiencies with contraptions like NX/x2go and Xpra). The question is what useful benefit is being provided by that miniscule still relevant base that justifies its existence along with the other legacy cruft and baggage.
"X11 seems to have fewer problems than the Wayland protocol" is too vague to be cogent so I will not address it. But the argument is more than just X11 vs Wayland, as Wayland isn't the only alternative display system nor is it the only answer to network transparent remote display technology either. "Wayland sucks" really is not a valid response to "X I don't really see the problems with it."
True, X11 the wire protocol isn't the most horrible thing in a world where SOAP exists, but in practice the latency story is overall bad. Yes you can pipeline with xcb and not with Xlib, so a few of the rehashed latency issues by the peanut gallery are false attribution, but the core protocol still makes many basic operations inherently synchronous and strictly ordered, many just a consequence of how the X server manages state. There are fundamental architectural issues.
I don't think that using extensions to support new features is much of a problem - if you disagree with that then you'll have to admit that Wayland has an even bigger problem because it requires even more extensions.
X11 doesn't inherently require that many round trips. Xlib does, because Xlib is bad. But, for example, clients choose their own object IDs, so they don't need round trips to find the IDs of newly created objects. Of course it requires a few round trips to do anything, but that is also true of Wayland. The complaint was about excessive round trips, not a few.
> I don't think that using extensions to support new features is much of a problem
You are arguing issues that were not even raised. Read 1st para more carefully, especially the last sentence.
> X11 doesn't inherently require that many round trips. The complaint was about excessive round trips,
Why do NX, x2go, and xpra exist if the X11 protocol over high latency networks is such a peach? It's lovely that clients don't need to round trip for IDs, but this is like saying the Trabant is one the best cars ever made because it possesses a steering wheel (and fwiw Wayland has an even more flexible client side ID allocation). As already said, just because something isn't horrible doesn't mean it's any good either.
There is so much other typical crap that has to go on that is suboptimal: property requests, chatty window management, inefficient frame synchronization, inefficient event handling.
Even if certain operations don't require a lockstep roundtrip, overall chattiness also contributes to latency.
> Of course it requires a few round trips to do anything, but that is also true of Wayland
The Wayland project is not perfect (it is implementing de facto X12 and a large undertaking in a computing world much more complicated both technically and financially than 1986), but it's patently ridiculous and just misinformed to imply the core protocol doesn't address some fundamental shortcomings of X11.
Meanwhile I thought it should be obvious that dispensing with the baked in network transparency of X11 was done as a conscientious choice not because a bunch of smart people just "forgot".
"So here's the thing: X works extremely well for what it is, but what it is is deeply flawed. There's no shame in that, it's 33 years old and still relevant, "... "Though the code happens to implement an unfortunate specification,"
X being network transparent hasn’t been true for decades — no one uses xmotif and similar GUI libs, they just grab a buffer and paint stuff themselves, making the X protocol an inefficient format to carry bitmaps around.
Local-first is the correct approach as the end computers are significantly more powerful than 30 years ago. Plus, we can transport bitmaps far more efficiently with compression, so it’s quite obviously an improvement on every metric, it is pretty shortsighted to note it down as circular development.
"no one uses xmotif and similar GUI libs, they just grab a buffer and paint stuff themselves, making the X protocol an inefficient format to carry bitmaps around."
While true for things like firefox, and GUI libs that think they know better than the host, its not ideal, and entirely dependent on the language/toolkit being used. And it's a side effect of X being too low level and not having a standard widget toolkit as one gets on Windows/mac. On those platforms, it's not just the widgets but file open dialogs provided by the OS, skinned consistently across applications, and things like global keyboard shortcuts (ex, copy/paste) tend to work close to 100% of the time.
If one goes back to the win3.x/macos7/etc timeframe it was almost unheard of for applications not to follow system wide color themes. WHich is why "dark mode" 1995 worked more reliably than it does on any platforms today. Today, doing something like inverting to a "dark" theme is suddenly something that each application hacks in, and when the OS provides a theme option, the applications frequently don't follow it correctly/etc.
I'm glad I found this thread, firefox really jumped the shark with remote x at some point, it always confused me why this happened all of a sudden. All I could find out from googling was the wayland thing and all of a sudden how x over the network wasn't cool anymore lol
I'm not sure this is true for apps that commonly forwarded though? My impression is tk for example uses xlib heavily and so has much better behaviour over the network than GTK, and the majority of apps I've seen where the audience uses X forwarding commonly seem to be using tk.
Somebody in a thread the other day lamented how almost always X clients are sensitive to network drops. And the suggestion is to use Xpra! (It’s “like tmux”)
Xpra doesn’t forward X though, it is a rootless X server that then forwards everything as compressed bitmaps like VNC. It’s a nice homage to how awful X is even at its one oft purported benefit around here.
X11 network transparency is a pretty poor technology, one can go through each of the “Fallacies of Distributed Computing” document to understand why.
and instead of x12 modernizing the network and access layer, breaking backwards compatibility with x11, they dropped those two features, and rewrote everything else which worked, and broke backwards compatibility.
It’s not a circle, it’s plain stupidity. Running stuff over the network has always been a requirement and somehow wayland people decided so ignore that requirement, producing an half-assed spec and quarter-assed implementations.
All this could have been avoided, and yet here we are…
What a strange comment. I was running applications over the network years ago using Wayland. Just because a new tool exists that you can do it with doesn't mean it doesn't work.
Running software locally has been a requirement for at least a few years by my calendar, yet I don't see you complaining about their network-first-patch-local-after approach.
What I never understood is how Spice is not elevated to a separate product. It does a fantastic job handling the graphical, audio, and peripheral connections to VMs, so how come there isn't a Spice server that I can just install to whatever Linux system and remote to it with the Spice clients that are already there?
Also, I'd like to plug Sunshine - I think it works on Wayland, and from the tests I did it does a fantastic job on a local network to make a desktop usable, even from an Android device. The downside is that whatever happens on the remote screen, it must happen on the host screen as well, so you can't have the host locked while you do gaming on it for example.
EDIT: just an FYI for everyone interested, here's a bit of info about how Proxmox is handling it. TL;DR Spice is still at least maintained until 2029, so it's not going to just go away anytime soon.
I looked at the Sunshine / Moonlight combination at one point as I thought something focused on latency would also be great for interactive desktop applications. It turns out though that clipboard synchronization isn't even supported so that was a complete non-starter for me.
Is it better than xrdp though? I found xrdp last week and it's stellar, basically a microsoft-compatible RDP server for Linux. It's a complete solution, no drama and simply amazing.
Every other Linux remote desktop solution I tried was a mess, including Xpra (which was quite involved to even get installed and Debian, and then a letdown, yikes).
I wonder what happens with Wprs when you try to launch virt-manager, which requires root.
For me, definitely yes, because it's rootless. Being able to stream individual windows is an absolutely killer feature. Having to deal with non-native window management is a nightmare, not to mention that it leaves you pulling down more pixels than you need to.
Microsoft has similar functionality but it's gated behind enterprise licenses. I ran Xpra for free when I was a kid and it was magical; many years later I've still never played with Windows' Terminal Services, which is the MS thing that can do rootless IIRC. The way MS gates the necessary windowing features is why to this day you can't do rootless with X2Go or Xpra with normal Windows clients.
Is there special reason the usual GUI PolKit privilege elevation wrappers can't work with a remote X session? I think I used to be able to use kdesu and similar just fine over Xpra. Note that 'rootless' in this context (windowing) means 'without a root window', i.e., without rendering a desktop. It has nothing to do with the usage in projects like podman where 'rootless' means 'not requiring elevated privileges'.
Rootless is a good UX especially if you use a tiling window manager locally, because of the cognitive dissonance when one of your tiled windows is a remote fvwm desktop or similar.
Likewise, I think the overwhelming majority of tiling window managers are very keyboard-oriented, so if you use the same or a similar one on a remote rooted vnc session, there will be conflicts between local and remote keybindings unless you're really careful about it. Better to just let the local WM manage everything.
Xpra can also stream rootless single windows, to html5 or other Xpra clients. I found it annoying with a bunch of sharp edges compared to getting a full desktop, though. You must have a different use-case :)
Can xrdp open windows in single apps? This is my main use case for using waypipe (mostly) and xpra. I don't want to show a full desktop nor having to deal with keyboard shortcuts highjack between host and remote.
I use remmina for that. It's simpler to configure. For example, it can export a local Linux directory to a Windows machine as a network share without messing with command line options. Not that CLI is bad, far from me, but it saves the time to learn the options for xrdp.
Can this be used to switch back and forth between different computers? If I remember correct Xpra supports this.
E.g. (1) open a session on desktop with firefox, thunderbird, vscode. (2) Then grab laptop and go to a coffee shop, connect with wayland-tranpositor to the desktop to resume the session with those programs where you left off with them. (3) Then go home and resume from the desktop.
I guess for (1) and (3) you are just running both the server and client on the desktop, and then for (2) you close the client on the desktop and run it on the laptop?
It would be cool if it could do shadowing, or have multiple clients running at the same time.
There's a second crucial element to this: can it lock the local screen while you do this?
I did this a whole lot in the lab with Windows - walk away, and log in remotely and monitor my session, then head back in and log back on locally.
I never found a way to do this with Linux that just restores the session the way Windows seems able to. Instead all I ever found was "the remote screen is unlocked and the cursor is moving" or whatever.
Yes. A wprs session is persistent on the server (currently a single session per user, we're going to support multiple sessions for a user when we get to it), so you can connect/reconnect from different client computers.
If you start the application in a local wprsc connecting to a local wprsd instead of in the local compositor directly, you can later connect to the session with a remote wprsc.
Locking the screen (the other subthread in here) is not applicable to this, applications are drawn wherever the wprsc is running and only one wprsc can be connected at a time.
Is there any open-source remote desktop that forwards the full desktop experience (not just pixels, but also audio, input devices beyond keyboard & mouse, with high refresh rates and low latency)?
wprs:
* does audio by forwarding a pulseaudio/pipewire socket, but it doesn't work so well for remote networks. I'd like to do something better in the future.
* defaults to 60fps and can support significantly higher refresh rates, depend on your cpu and bandwidth.
* aims to be low latency but prioritizes quality and so requires a good cpu and network connection to be low latency
* should be able to handle (possibly with some additional code) any input devices wayland supports
pixels, audio, input devices should work these days.
additional devices could be forwarded with usbip
high refresh rates and low latency are non-trivial problems because they require realtime video encoding which is less bandwidth efficient than offline encoding which means you need a network transport which can shovel high bandwidth data at low latency and gracefully handle packet losses. if you do it over the open internet you also need to adapt to variable bandwidth. all these things have been done, but someone would still need to tie all of it together in an open source solution.
There's the sunshine/moonlight combo for remote game streaming, but it lacks other features for remote desktop use.
wprs doesn't do its own rendering, it passes through already-drawn image buffers from applications on the server side to the local compositor on the client side, which then composits/renders them however it wishes to.
"Here's an optimization for when the X display is the same machine as the process."
"We've built Wayland. We all run applications on the same machine as the display and X gets in the way."
"We'd like to run applications from machines over the internet."
"We've built Wprs. It allows you to run graphical applications over a network."
That's the circle of life.....