I've never had any reason to use Wayland or desire to learn much about it, so I can't tell you anything technical or from first hand experience.
But I think we have X-Windows to thank for the fact that most Unix programmers use Macs now.
And it's way too late for Wayland to change that fact. Especially with the release of the M1 Max. That ship has sailed.
It didn't matter how much better NeWS was than X-Windows -- it still lost despite all its merits.
And I don't see Wayland as being any more better than X-Windows, than NeWS was better, decades ago.
So simply "being better than X" is not sufficient to displace it, and Wayland isn't even that much better than X, and is even worse in some ways.
The fact that it didn't occur to the Wayland designers to make it extensible with an embedded scripting language like PostScript in NeWS or Lisp in Emacs or Python in Blender or Lua in Redis or JavaScript in web browsers means that it was obsolete from the start, and its designers should have considered the design and architecture of NeWS and Emacs and Blender and Redis and web browsers before designing Yet-Another-X-Windows-Clone. It's not like those ideas were secret, or patented, or hard to find, or wrong.
The world moved up the stack a layer to the web browser, and that's where all the excitement's happening these days, not in the window system layer.
Why have X-Windows or Wayland at all, when you could just run the web browser itself directly on the hardware, as efficiently and flexibly as possible, and implement all your user interface, window management and desktop stuff with modern open standard JavaScript / WebAssembly / JSON / XML / HTML / SVG / CSS / Canvas / WebGL / WebGPU / HTTPS / WebRTC?
I've written about this numerous times before, but I'll transclude and reorganize some of it with checked and updated archive urls to save you the pointing and clicking:
>colanderman>And this is my primary issue with Wayland. I cannot fathom why anyone would think it's a sound design decision to bundle a hardware-independent component (the window manager) with a hardware-dependent component (the compositor). This hearkens back to the days of DOS video games – what fun it was to implement support for everyone's sound card! Instead now we'll get to support KMS, Quartz, whatever-the-heck BSD uses, etc.
>Just put a JavaScript (or whatever) interpreter in the window server, and program the window manager locally in that. Then you aren't fucked by synchronization issues. James Gosling did something like that with PostScript many years ago, an alternative to X11, which was then merged with X11, and it was called NeWS (and later X11/NeWS or OpenWindows):
>I've written several window managers / user interface toolkits / tabbed window frames / menu system in PostScript for NeWS. We even wrote an X11 window manager in PostScript, complete with rooms, scrolling virtual desktop, tabbed windows, pie menus, and seamless integration of X11 and NeWS windows.
>Having the window manager running as an outboard process, communicating via an asynchronous protocol, is a terrible idea, but is baked into X11 at a very low level -- with the ICCCM thrown in as a kludgy afterthought. The X11 protocol has hacks and band-aids like "mouse button grabs" and "server grabs" to mitigate the problems, but they cause a lot of other problems, complexity and inefficiencies on their own.
>That's exactly the point of using HyperLook as a window manager, which we were discussing at Sun in 1991 before they canceled NeWS.
>HyperLook, which was inspired by HyperCard, but written in PostScript for the NeWS window system, let you build specialized task-oriented user interface by assembling and customizing and scripting together components and applets into their own "stacks".
>HyperLook (which ran on NeWS, and was like a networked version of Hypercard based on PostScript) was kind of like looking outwards through windows, in the way Rob Pike described the Blit (although it was architecturally quite different).
>I think his point was that window frames should be like dynamically typed polymorphic collections possibly containing multiple differently typed components that can change over time (like JavaScript arrays), not statically typed single-element containers that are permanently bound to the same component type which can't ever change (like C variables).
>With X11, the client comes first, and the window manager simply slaps a generic window frame around it, subject to some pre-defined client-specified customizations like ICCCM properties to control the window dressing, but not much more, and not under the control of the user.
>With HyperLook, the "stack/background/card" or window frame came first, and you (the user at runtime, not just the developer at design time) could compose any other components and applications together in your own stacks, by copying and pasting them out of other stacks or warehouses of pre-configured components.
>For example, I implemented SimCity for HyperLook as a client that ran in a separate process than the window server, and sent messages over the local network and drew into shared memory bitmaps. SimCity had its own custom frames ("screw-up windows") that looked more "mayoral" than normal windows.
>There were multiple map and editor views that you could copy and paste into other stacks while they were running, or you could copy widgets like buttons or drawing editors into the SimCity stack while it was running. So you could copy and paste the "RCI" gauge into your graphics editor window to keep an eye on it while you worked, or paste a clock into your SimCity window to see when it was time to stop playing and get some sleep. And you could even hit the "props" key on the clock to bring up a PostScript graphics editor that let you totally customize how it looked!
>It had "warehouse" stacks containing collections of pre-configured component prototypes (including widgets, applets, and window management controls like close buttons, resize corners, navigation buttons, clocks, graphics editors, etc) that you could copy and paste into your own stacks, and which would automatically be listed in the "New Object" menu you get in edit mode, to create them in place without opening up the warehouse:
>Normal X11 window managers put the cart before the donkey. The window frames are generic, stupid and unscriptable, and can't even be customized by the user, or saved and restored later. The client comes first, with just one client per generic frame. You can't move or copy a widget or panel from one frame to another, to use them together in the same window. That's terribly limited and primitive.
>We implemented a more-or-less traditional (but better than normal) ICCCM X11 window manager in PostScript for X11/NeWS, which had tabbed windows, pie menus, rooms, scrolling virtual desktop, etc, uniformly for all NeWS apps and X11 clients. But the next step (not NeXT Step) was to use HyperLook to break out of the limited one-client-per-frame paradigm.
>(It's interesting to note that David Rosenthal, the author of the ICCCM specification, was also one of the architects of NeWS, along with James Gosling.)
>Our plan was to use HyperLook to implement a totally different kind of integrated scriptable X11 (and NeWS) window manager, so you could compose multiple X11 clients into the same frame along with HyperLook widgets and scripts. You could integrate multiple X11 clients and NeWS components into fully customizable seamless task oriented user interfaces, instead of switching between windows and copying and pasting between a bunch of separate monolithic clients that don't know about each other. But unfortunately Sun canceled NeWS before we could do that.
>Here's some more stuff I've written about that stuff, and how window managers should be implemented today in JavaScript:
>That's how I implemented tabbed windows in 1988 for the NeWS window system and UniPress Emacs (aka Gosling Emacs aka Evil Software Hoarder Emacs), which supported multiple windows on NeWS and X11 long before Gnu Emacs did.
That seemed to me like the most obvious way to do it at the time, since tabs along the top of bottom edges were extremely wasteful of screen space. (That was on a big hires Sun workstation screen, not a tiny little lores Mac display, so you could open a lot more windows, especially with Emacs.)
>It makes them more like a vertical linear menu of opened windows, so you can real all their titles, fit many of them on the screen at once, and you instantly access any one and can pop up pie menus on the tabs to perform window management commands even if the windows themselves are not visible.
>But now that there's another Web Browser layer slapped on top of X-Windows (and the terms "client" and "server" switched places), you're stuck with two competing mutually incompatible window managers nested inside of each other, one for the outside frames around apps, and one for the inside tabs around web pages, that don't cooperate with each other, and operate uncannily differently.
>So none of your desktop apps benefit from any of the features of the web browser (unless they include their own web browser like Electron).
>And you can't write lightweight apps in a couple pages of script like "Big Brother Eyes" that run in the shared window server, without requiring their own separate external app with a huge runtime like Electron.
>Nor can you easily integrate desktop applications and widgets and web pages together with scripts, the way HyperCard, OpenDoc, CyberDog and HyperLook enabled.
>>Now Firefox and Chrome still need decent built-in universally supported and user customizable pie menus, but unfortunately the popup window extension API is inadequate to support them, because there's no way to make them pop up centered on the cursor, or control the popup window shape and transparency. Unfortunately they were only thinking of drop-down linear menus when they designed the API. (Stop thinking inside that damn rectangular box, people!)
>>But I remain hopeful that somebody will eventually rediscover pie menus in combination with tabbed window for the web browser, and implement them properly (not constrained to pop up inside the browser window and be clipped by the window frame, and not just one gimmicky hard coded menu that user's can't change and developers can't use in their own applications). But the poorly designed browser extension APIs still have a hell of a lot of catching up to do with what it was trivial to do in NeWS for all windows 30 years ago.
>And this is why a modern window manager should be written in JavaScript, leverage HTML, Canvas and WebGL, and support Accessibility APIs, as well as screen scraping, pattern recognition, screen casting, virtual desktops, overlays, drawing and image composition, input event synthesis, journaling, macros, runtime user customization and scripting, programming by demonstration, tabbed windows, pie menus, etc:
>[...] About 5 years ago I opened this issue, describing an experiment I did making the web browser in a topmost window with a transparent background to implement user interface overlays scripted in HTML.
>WebView window for HTML user interfaces like pie menus to Slate. #322:
>Slate used a hidden WebView for its scripting engine. So I made it un-hidden and float on top of all the other windows, and was easily able to use it to draw any kind of user interface stuff on top of all the other Mac windows. And I could track the position of windows and draw a little clickable tab next to or on top of the window title bar, that you could click on to pop up a pie menu.
>It actually worked! But I didn't take it much further, because I never got any feedback on the issue I opened, so gave up on using Slate itself, and never got around to starting my own JavaScript window manager myself (like you did!). I opened my issue in June 2013, but the last commit was Feb 2013, so development must have stopped by then.
>[...] Think of it like augmented reality for virtualizing desktop user interfaces and web pages.
>[...] So I adapted the "uwm" window manager to support pie menus, so you could define your own pie menus and linear menus in your .uwmrc file. Because a window manager could really benefit from pie menus: lots of the commands are spatially oriented and can be arranged in appropriate mnemonic directions, and they have a fixed set of common window management commands that can be thoughtfully arranged into a set of efficient pie menus, as well as a menu definition language to enable users to create custom menus for running apps, connecting with remote hosts, etc.
>[...] I had been using Mitch Bradley's "Forthmacs", which was a very nice Forth system for the 68k. (It eventually evolved into the Sun 4 Forth boot ROMS, OpenFirmware, and the OLPC boot ROMs). It was capable of dynamically linking in C code (well not dll's or shared libraries, but it would actually call the linker to relocate the code to the appropriate place in Forth's address space, read in the relocated code, and write Forth wrappers, so you could call C code from Forth, pass parameters, etc -- SunOS 4.2 didn't have relocatable shared libraries or light weight threads back then, so Forth had to do a lot of the heavy lifting to plug in C code. But Forth was a really great way to integrate a library into an interactive extension language, call it from the Forth command line, build on top of C code in Forth, call back and forth between C and Forth, play around with it from the Forth command line, etc).
>Inspired by HyperCard, we (old Sun NeWS hands) also pleaded until we were blue in the face to make HyperLook (a NeWS/PostScript/network based reinterpretation of HyperCard) the window manager / editable scriptable desktop environment!
>“I thought NeWS was ‘the right way to go’ (except it missed the live system underneath). It was also very early in commercial personal computing to be able to do a UI using Postscript, so it was impressive that the implementation worked at all.” -Alan Kay
>What’s the Big Deal About HyperCard?
>"I thought HyperCard was quite brilliant in the end-user problems it solved. (It would have been wonderfully better with a deep dynamic language underneath, but I think part of the success of the design is that they didn’t have all the degrees of freedom to worry about, and were just able to concentrate on their end-user’s direct needs."
>"HyperCard is an especially good example of a system that was “finished and smoothed and documented” beautifully. It deserved to be successful. And Apple blew it by not making the design framework the basis of a web browser (as old Parc hands advised in the early 90s …)" -Alan Kay
>HyperLook was so far ahead of its time in 1989, that there still isn't anything quite like it for modern technology. Since we developed HyperLook and SimCity at the same time, that forced us to eat our own dog food, and ensure that HyperLook supported everything you needed to develop real world applications. (Not to imply that SimCity is a real world! ;)
>This is a discussion of ICCCM Window Management for X11/NeWS. One of the horrible problems of X11/NeWS was window management. The X people wanted to wrap NeWS windows up in X frames (that is, OLWM). The NeWS people wanted to do it the other way around, and prototyped an ICCCM window manager in NeWS (mostly object oriented PostScript, and a tiny bit of C), that wrapped X windows up in NeWS window frames.
>Why wrap X windows in NeWS frames? Because NeWS is much better at window management than X. On the surface, it was easy to implement lots of cool features. But deeper, NeWS is capable of synchronizing input events much more reliably than X11, so it can manage the input focus perfectly, where asynchronous X11 window managers fall flat on their face by definition.
>Our next step (if you'll pardon the allusion) was to use HyperNeWS (renamed HyperLook, a graphical user interface system like HyperCard with PostScript) to implemented a totally customizable X window manager!
>>You can't do opengl efficiently over the network. At least xorg can't. Most applications uses opengl these days.
>Sure you can, it's just called WebGL! ;)
>They just added another layer, flipped the words "server" and "client" around, and added more hardware.
>Now you run the web browser client on top of the local window system server, through shared memory, without using the network. And both the browser and GPU are locally programmable!
>And then the local web browser client accesses remote web servers over the network, instead of ever using X11's networking ability.
>One way of looking at it in the X11 sense is that a remote app running in the web server acts as a client of the local window server's display and GPU hardware, by downloading JavaScript code to run in the web browser (acting as a programmable middleman near the display), and also shader code to run in the window server's GPU.
>Trying to pigeonhole practices like distributed network and GPU programming into simplistic dichotomies like "client/server," or partition user interface programming into holy trinities like "model/view/controller," just oversimplifies reality and unnecessarily limits designs.
>Imglorp> Hi Don! Tabs aside, NeWS got so many things right, and not just at the UI but the underlying idea of code instead of bitmaps. I wonder if we were going to start afresh with Wayland/X11 now, bare hardware, what place do you think the NeWS lessons would have? Would a modern NeWS be on top of PS or is there something better now?
>Hi Imglorp!
>Simply use a standard JavaScript / WebAssembly / WebGL / Canvas / HTML based web browser as the window system itself! And use WebSocket/SocketIO/RTP/HTTP instead of the X-Windows or VNC protocols.
>Microsoft kinda-sorta did a half-assed inside-out version of that with Active Desktop, but Internet Explorer wasn't powerful or stable enough to do it right, and it didn't eliminate and replace the whole Win32 / MFC layer, which misses the main point.
>There was recently this discussion about a browser based Mac window manager project (now offline), and there have been others like it (Slate), but the ideal goal is to completely eliminate the underlying window system and just use pure open web technologies directly on the metal:
>Show HN: Autumn – A macOS window manager for (Type|Java)Script hackers (sephware.com):
>I was also quite inspired by Slate. Unfortunately there hasn't been any activity with it for about 5 years or so. It's great you're picking up the mantel and running with it, because the essential idea is great!
>It would also be great to flesh out the accessibility and speech recognition APIs, and make it possible to write all kinds of intelligent application automation and integration scripts, bots, with nice HTML user interfaces in JavaScript. Take a look at what Dragon Naturally Speaking has done with Python:
>I would like to discuss how we could integrate Prefab with a Javascriptable, extensible API like aQuery, so you could write "selectors" that used prefab's pattern recognition techniques, bind those to JavaScript event handlers, and write high level widgets on top of that in JavaScript, and implement the graphical overlays and gui enhancements in HTML/Canvas/etc like I've done with Slate and the WebView overlay.
>Video: Prefab: What if We Could Modify Any Interface? Target aware pointing techniques, bubble cursor, sticky icons, adding advanced behaviors to existing interfaces, independent of the tools used to implement those interfaces, platform agnostic enhancements, same Prefab code works on Windows and Mac, and across remote desktops, widget state awareness, widget transition tracking, side views, parameter preview spectrums for multi-parameter space exploration, prefab implements parameter spectrum preview interfaces for both unmodified Gimp and Photoshop:
>PDF: A General-Purpose Target-Aware Pointing Enhancement Using Pixel-Level Analysis of Graphical Interfaces. Morgan Dixon, James Fogarty, and Jacob O. Wobbrock. (2012). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '12. ACM, New York, NY, 3167-3176. 23%.
>Video: Content and Hierarchy in Prefab: What if anybody could modify any interface? Reverse engineering guis from their pixels, addresses hierarchy and content, identifying hierarchical tree structure, recognizing text, stencil based tutorials, adaptive gui visualization, ephemeral adaptation technique for arbitrary desktop interfaces, dynamic interface language translation, UI customization, re-rendering widgets, Skype favorite widgets tab:
>PDF: Content and Hierarchy in Pixel-Based Methods for Reverse-Engineering Interface Structure. Morgan Dixon, Daniel Leventhal, and James Fogarty. (2011). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '11. ACM, New York, NY, 969-978. 26%.
>Video: Sliding Widgets, States, and Styles in Prefab. Adapting desktop interfaces for touch screen use, with sliding widgets, slow fine tuned pointing with magnification, simulating rollover to reveal tooltips:
>PDF: Prefab: Implementing Advanced Behaviors Using Pixel-Based Reverse Engineering of Interface Structure. Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 1525-1534. 22%
>PDF: Prefab: What if Every GUI Were Open-Source? Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 851-854.
>Today, most interfaces are designed by teams of people who are collocated and highly skilled. Moreover, any changes to an interface are implemented by the original developers and designers who own the source code. In contrast, I envision a future where distributed online communities rapidly construct and improve interfaces. Similar to the Wikipedia editing process, I hope to explore new interface design tools that fully democratize the design of interfaces. Wikipedia provides static content, and so people can collectively author articles using a very basic Wiki editor. However, community-driven interface tools will require a combination of sophisticated programming-by-demonstration techniques, crowdsourcing and social systems, interaction design, software engineering strategies, and interactive machine learning.
I've never had any reason to use Wayland or desire to learn much about it, so I can't tell you anything technical or from first hand experience.
But I think we have X-Windows to thank for the fact that most Unix programmers use Macs now.
And it's way too late for Wayland to change that fact. Especially with the release of the M1 Max. That ship has sailed.
It didn't matter how much better NeWS was than X-Windows -- it still lost despite all its merits.
And I don't see Wayland as being any more better than X-Windows, than NeWS was better, decades ago.
So simply "being better than X" is not sufficient to displace it, and Wayland isn't even that much better than X, and is even worse in some ways.
The fact that it didn't occur to the Wayland designers to make it extensible with an embedded scripting language like PostScript in NeWS or Lisp in Emacs or Python in Blender or Lua in Redis or JavaScript in web browsers means that it was obsolete from the start, and its designers should have considered the design and architecture of NeWS and Emacs and Blender and Redis and web browsers before designing Yet-Another-X-Windows-Clone. It's not like those ideas were secret, or patented, or hard to find, or wrong.
The world moved up the stack a layer to the web browser, and that's where all the excitement's happening these days, not in the window system layer.
Why have X-Windows or Wayland at all, when you could just run the web browser itself directly on the hardware, as efficiently and flexibly as possible, and implement all your user interface, window management and desktop stuff with modern open standard JavaScript / WebAssembly / JSON / XML / HTML / SVG / CSS / Canvas / WebGL / WebGPU / HTTPS / WebRTC?
I've written about this numerous times before, but I'll transclude and reorganize some of it with checked and updated archive urls to save you the pointing and clicking:
https://news.ycombinator.com/item?id=5844345
>colanderman>And this is my primary issue with Wayland. I cannot fathom why anyone would think it's a sound design decision to bundle a hardware-independent component (the window manager) with a hardware-dependent component (the compositor). This hearkens back to the days of DOS video games – what fun it was to implement support for everyone's sound card! Instead now we'll get to support KMS, Quartz, whatever-the-heck BSD uses, etc.
>Just put a JavaScript (or whatever) interpreter in the window server, and program the window manager locally in that. Then you aren't fucked by synchronization issues. James Gosling did something like that with PostScript many years ago, an alternative to X11, which was then merged with X11, and it was called NeWS (and later X11/NeWS or OpenWindows):
http://en.wikipedia.org/wiki/NeWS
>I've written several window managers / user interface toolkits / tabbed window frames / menu system in PostScript for NeWS. We even wrote an X11 window manager in PostScript, complete with rooms, scrolling virtual desktop, tabbed windows, pie menus, and seamless integration of X11 and NeWS windows.
https://www.youtube.com/watch?v=tMcmQk-q0k4
https://news.ycombinator.com/item?id=5845119
>Having the window manager running as an outboard process, communicating via an asynchronous protocol, is a terrible idea, but is baked into X11 at a very low level -- with the ICCCM thrown in as a kludgy afterthought. The X11 protocol has hacks and band-aids like "mouse button grabs" and "server grabs" to mitigate the problems, but they cause a lot of other problems, complexity and inefficiencies on their own.
https://news.ycombinator.com/item?id=20182294
>That's exactly the point of using HyperLook as a window manager, which we were discussing at Sun in 1991 before they canceled NeWS.
>HyperLook, which was inspired by HyperCard, but written in PostScript for the NeWS window system, let you build specialized task-oriented user interface by assembling and customizing and scripting together components and applets into their own "stacks".
https://news.ycombinator.com/item?id=19007885
>HyperLook (which ran on NeWS, and was like a networked version of Hypercard based on PostScript) was kind of like looking outwards through windows, in the way Rob Pike described the Blit (although it was architecturally quite different).
>I think his point was that window frames should be like dynamically typed polymorphic collections possibly containing multiple differently typed components that can change over time (like JavaScript arrays), not statically typed single-element containers that are permanently bound to the same component type which can't ever change (like C variables).
>With X11, the client comes first, and the window manager simply slaps a generic window frame around it, subject to some pre-defined client-specified customizations like ICCCM properties to control the window dressing, but not much more, and not under the control of the user.
>With HyperLook, the "stack/background/card" or window frame came first, and you (the user at runtime, not just the developer at design time) could compose any other components and applications together in your own stacks, by copying and pasting them out of other stacks or warehouses of pre-configured components.
https://donhopkins.medium.com/hyperlook-nee-hypernews-nee-go...
>For example, I implemented SimCity for HyperLook as a client that ran in a separate process than the window server, and sent messages over the local network and drew into shared memory bitmaps. SimCity had its own custom frames ("screw-up windows") that looked more "mayoral" than normal windows.
https://cdn-images-1.medium.com/max/2560/0*ZM8s95LNxemc5Enz....
>There were multiple map and editor views that you could copy and paste into other stacks while they were running, or you could copy widgets like buttons or drawing editors into the SimCity stack while it was running. So you could copy and paste the "RCI" gauge into your graphics editor window to keep an eye on it while you worked, or paste a clock into your SimCity window to see when it was time to stop playing and get some sleep. And you could even hit the "props" key on the clock to bring up a PostScript graphics editor that let you totally customize how it looked!
https://cdn-images-1.medium.com/max/600/0*oHtC0F5qK83ADw1H.g...
>It had "warehouse" stacks containing collections of pre-configured component prototypes (including widgets, applets, and window management controls like close buttons, resize corners, navigation buttons, clocks, graphics editors, etc) that you could copy and paste into your own stacks, and which would automatically be listed in the "New Object" menu you get in edit mode, to create them in place without opening up the warehouse:
https://cdn-images-1.medium.com/max/600/0*sHClGU8ALljuRQKb.g...
https://cdn-images-1.medium.com/max/600/0*QwIQ_GLxQl1v968F.g...
https://cdn-images-1.medium.com/max/600/0*aWbuo6k_eJuZnUmV.g...
https://cdn-images-1.medium.com/max/600/0*zya4vNBP3libpNSA.g...
https://cdn-images-1.medium.com/max/600/1*G0LWky2iejYm4IGBsU...
>Normal X11 window managers put the cart before the donkey. The window frames are generic, stupid and unscriptable, and can't even be customized by the user, or saved and restored later. The client comes first, with just one client per generic frame. You can't move or copy a widget or panel from one frame to another, to use them together in the same window. That's terribly limited and primitive.
>We implemented a more-or-less traditional (but better than normal) ICCCM X11 window manager in PostScript for X11/NeWS, which had tabbed windows, pie menus, rooms, scrolling virtual desktop, etc, uniformly for all NeWS apps and X11 clients. But the next step (not NeXT Step) was to use HyperLook to break out of the limited one-client-per-frame paradigm.
http://www.art.net/~hopkins/Don/unix-haters/x-windows/i39l.h...
>(It's interesting to note that David Rosenthal, the author of the ICCCM specification, was also one of the architects of NeWS, along with James Gosling.)
>Our plan was to use HyperLook to implement a totally different kind of integrated scriptable X11 (and NeWS) window manager, so you could compose multiple X11 clients into the same frame along with HyperLook widgets and scripts. You could integrate multiple X11 clients and NeWS components into fully customizable seamless task oriented user interfaces, instead of switching between windows and copying and pasting between a bunch of separate monolithic clients that don't know about each other. But unfortunately Sun canceled NeWS before we could do that.
>Here's some more stuff I've written about that stuff, and how window managers should be implemented today in JavaScript:
https://news.ycombinator.com/item?id=18837730
>That's how I implemented tabbed windows in 1988 for the NeWS window system and UniPress Emacs (aka Gosling Emacs aka Evil Software Hoarder Emacs), which supported multiple windows on NeWS and X11 long before Gnu Emacs did. That seemed to me like the most obvious way to do it at the time, since tabs along the top of bottom edges were extremely wasteful of screen space. (That was on a big hires Sun workstation screen, not a tiny little lores Mac display, so you could open a lot more windows, especially with Emacs.)
>It makes them more like a vertical linear menu of opened windows, so you can real all their titles, fit many of them on the screen at once, and you instantly access any one and can pop up pie menus on the tabs to perform window management commands even if the windows themselves are not visible.
https://news.ycombinator.com/item?id=18865242
>But now that there's another Web Browser layer slapped on top of X-Windows (and the terms "client" and "server" switched places), you're stuck with two competing mutually incompatible window managers nested inside of each other, one for the outside frames around apps, and one for the inside tabs around web pages, that don't cooperate with each other, and operate uncannily differently.
>So none of your desktop apps benefit from any of the features of the web browser (unless they include their own web browser like Electron).
>And you can't write lightweight apps in a couple pages of script like "Big Brother Eyes" that run in the shared window server, without requiring their own separate external app with a huge runtime like Electron.
https://www.donhopkins.com/home/archive/news-tape/fun/eye/ey...
>Nor can you easily integrate desktop applications and widgets and web pages together with scripts, the way HyperCard, OpenDoc, CyberDog and HyperLook enabled.
https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-g...
>Here's how the web browser and window manager should work together seamlessly:
https://news.ycombinator.com/item?id=18837730
>>Now Firefox and Chrome still need decent built-in universally supported and user customizable pie menus, but unfortunately the popup window extension API is inadequate to support them, because there's no way to make them pop up centered on the cursor, or control the popup window shape and transparency. Unfortunately they were only thinking of drop-down linear menus when they designed the API. (Stop thinking inside that damn rectangular box, people!)
>>But I remain hopeful that somebody will eventually rediscover pie menus in combination with tabbed window for the web browser, and implement them properly (not constrained to pop up inside the browser window and be clipped by the window frame, and not just one gimmicky hard coded menu that user's can't change and developers can't use in their own applications). But the poorly designed browser extension APIs still have a hell of a lot of catching up to do with what it was trivial to do in NeWS for all windows 30 years ago.
>And this is why a modern window manager should be written in JavaScript, leverage HTML, Canvas and WebGL, and support Accessibility APIs, as well as screen scraping, pattern recognition, screen casting, virtual desktops, overlays, drawing and image composition, input event synthesis, journaling, macros, runtime user customization and scripting, programming by demonstration, tabbed windows, pie menus, etc:
https://news.ycombinator.com/item?id=18797587
>[...] About 5 years ago I opened this issue, describing an experiment I did making the web browser in a topmost window with a transparent background to implement user interface overlays scripted in HTML.
>WebView window for HTML user interfaces like pie menus to Slate. #322:
https://github.com/jigish/slate/issues/322
>Slate used a hidden WebView for its scripting engine. So I made it un-hidden and float on top of all the other windows, and was easily able to use it to draw any kind of user interface stuff on top of all the other Mac windows. And I could track the position of windows and draw a little clickable tab next to or on top of the window title bar, that you could click on to pop up a pie menu.
>It actually worked! But I didn't take it much further, because I never got any feedback on the issue I opened, so gave up on using Slate itself, and never got around to starting my own JavaScript window manager myself (like you did!). I opened my issue in June 2013, but the last commit was Feb 2013, so development must have stopped by then.
>[...] Think of it like augmented reality for virtualizing desktop user interfaces and web pages.
https://news.ycombinator.com/item?id=5861229
>[...] So I adapted the "uwm" window manager to support pie menus, so you could define your own pie menus and linear menus in your .uwmrc file. Because a window manager could really benefit from pie menus: lots of the commands are spatially oriented and can be arranged in appropriate mnemonic directions, and they have a fixed set of common window management commands that can be thoughtfully arranged into a set of efficient pie menus, as well as a menu definition language to enable users to create custom menus for running apps, connecting with remote hosts, etc.
>[...] I had been using Mitch Bradley's "Forthmacs", which was a very nice Forth system for the 68k. (It eventually evolved into the Sun 4 Forth boot ROMS, OpenFirmware, and the OLPC boot ROMs). It was capable of dynamically linking in C code (well not dll's or shared libraries, but it would actually call the linker to relocate the code to the appropriate place in Forth's address space, read in the relocated code, and write Forth wrappers, so you could call C code from Forth, pass parameters, etc -- SunOS 4.2 didn't have relocatable shared libraries or light weight threads back then, so Forth had to do a lot of the heavy lifting to plug in C code. But Forth was a really great way to integrate a library into an interactive extension language, call it from the Forth command line, build on top of C code in Forth, call back and forth between C and Forth, play around with it from the Forth command line, etc).
https://news.ycombinator.com/item?id=16839825
>Inspired by HyperCard, we (old Sun NeWS hands) also pleaded until we were blue in the face to make HyperLook (a NeWS/PostScript/network based reinterpretation of HyperCard) the window manager / editable scriptable desktop environment!
https://news.ycombinator.com/item?id=18314265
https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-g...
>Alan Kay on NeWS:
>“I thought NeWS was ‘the right way to go’ (except it missed the live system underneath). It was also very early in commercial personal computing to be able to do a UI using Postscript, so it was impressive that the implementation worked at all.” -Alan Kay
>What’s the Big Deal About HyperCard?
>"I thought HyperCard was quite brilliant in the end-user problems it solved. (It would have been wonderfully better with a deep dynamic language underneath, but I think part of the success of the design is that they didn’t have all the degrees of freedom to worry about, and were just able to concentrate on their end-user’s direct needs."
>"HyperCard is an especially good example of a system that was “finished and smoothed and documented” beautifully. It deserved to be successful. And Apple blew it by not making the design framework the basis of a web browser (as old Parc hands advised in the early 90s …)" -Alan Kay
https://news.ycombinator.com/item?id=8546507
>HyperLook was so far ahead of its time in 1989, that there still isn't anything quite like it for modern technology. Since we developed HyperLook and SimCity at the same time, that forced us to eat our own dog food, and ensure that HyperLook supported everything you needed to develop real world applications. (Not to imply that SimCity is a real world! ;)
http://www.art.net/~hopkins/Don/unix-haters/x-windows/i39l.h...
>Who Should Manage the Windows, X11 or NeWS?
>This is a discussion of ICCCM Window Management for X11/NeWS. One of the horrible problems of X11/NeWS was window management. The X people wanted to wrap NeWS windows up in X frames (that is, OLWM). The NeWS people wanted to do it the other way around, and prototyped an ICCCM window manager in NeWS (mostly object oriented PostScript, and a tiny bit of C), that wrapped X windows up in NeWS window frames.
>Why wrap X windows in NeWS frames? Because NeWS is much better at window management than X. On the surface, it was easy to implement lots of cool features. But deeper, NeWS is capable of synchronizing input events much more reliably than X11, so it can manage the input focus perfectly, where asynchronous X11 window managers fall flat on their face by definition.
>Our next step (if you'll pardon the allusion) was to use HyperNeWS (renamed HyperLook, a graphical user interface system like HyperCard with PostScript) to implemented a totally customizable X window manager!
https://news.ycombinator.com/item?id=13785384
>>You can't do opengl efficiently over the network. At least xorg can't. Most applications uses opengl these days.
>Sure you can, it's just called WebGL! ;)
>They just added another layer, flipped the words "server" and "client" around, and added more hardware.
>Now you run the web browser client on top of the local window system server, through shared memory, without using the network. And both the browser and GPU are locally programmable!
>And then the local web browser client accesses remote web servers over the network, instead of ever using X11's networking ability.
>One way of looking at it in the X11 sense is that a remote app running in the web server acts as a client of the local window server's display and GPU hardware, by downloading JavaScript code to run in the web browser (acting as a programmable middleman near the display), and also shader code to run in the window server's GPU.
>Trying to pigeonhole practices like distributed network and GPU programming into simplistic dichotomies like "client/server," or partition user interface programming into holy trinities like "model/view/controller," just oversimplifies reality and unnecessarily limits designs.
https://news.ycombinator.com/item?id=20183838
>Imglorp> Hi Don! Tabs aside, NeWS got so many things right, and not just at the UI but the underlying idea of code instead of bitmaps. I wonder if we were going to start afresh with Wayland/X11 now, bare hardware, what place do you think the NeWS lessons would have? Would a modern NeWS be on top of PS or is there something better now?
>Hi Imglorp!
>Simply use a standard JavaScript / WebAssembly / WebGL / Canvas / HTML based web browser as the window system itself! And use WebSocket/SocketIO/RTP/HTTP instead of the X-Windows or VNC protocols.
>Microsoft kinda-sorta did a half-assed inside-out version of that with Active Desktop, but Internet Explorer wasn't powerful or stable enough to do it right, and it didn't eliminate and replace the whole Win32 / MFC layer, which misses the main point.
https://en.wikipedia.org/wiki/Active_Desktop
>There was recently this discussion about a browser based Mac window manager project (now offline), and there have been others like it (Slate), but the ideal goal is to completely eliminate the underlying window system and just use pure open web technologies directly on the metal:
>Show HN: Autumn – A macOS window manager for (Type|Java)Script hackers (sephware.com):
https://news.ycombinator.com/item?id=18794928
>Site archive:
https://web.archive.org/web/20190101121003/https://sephware....
Comments:
https://news.ycombinator.com/item?id=18797587
>I was also quite inspired by Slate. Unfortunately there hasn't been any activity with it for about 5 years or so. It's great you're picking up the mantel and running with it, because the essential idea is great!
https://news.ycombinator.com/item?id=18797818
>Here are some other interesting things related to scriptable window management and accessibility to check out:
>aQuery -- Like jQuery for Accessibility
https://web.archive.org/web/20180317054320/https://donhopkin...
>It would also be great to flesh out the accessibility and speech recognition APIs, and make it possible to write all kinds of intelligent application automation and integration scripts, bots, with nice HTML user interfaces in JavaScript. Take a look at what Dragon Naturally Speaking has done with Python:
https://github.com/t4ngo/dragonfly
>Morgan Dixon's work with Prefab is brilliant.
>I would like to discuss how we could integrate Prefab with a Javascriptable, extensible API like aQuery, so you could write "selectors" that used prefab's pattern recognition techniques, bind those to JavaScript event handlers, and write high level widgets on top of that in JavaScript, and implement the graphical overlays and gui enhancements in HTML/Canvas/etc like I've done with Slate and the WebView overlay.
>Web Site: Morgan Dixon's Home Page.
http://morgandixon.net/
Web Site: Prefab: The Pixel-Based Reverse Engineering Toolkit.
https://web.archive.org/web/20130104165553/http://homes.cs.w...
>Video: Prefab: What if We Could Modify Any Interface? Target aware pointing techniques, bubble cursor, sticky icons, adding advanced behaviors to existing interfaces, independent of the tools used to implement those interfaces, platform agnostic enhancements, same Prefab code works on Windows and Mac, and across remote desktops, widget state awareness, widget transition tracking, side views, parameter preview spectrums for multi-parameter space exploration, prefab implements parameter spectrum preview interfaces for both unmodified Gimp and Photoshop:
http://www.youtube.com/watch?v=lju6IIteg9Q
>PDF: A General-Purpose Target-Aware Pointing Enhancement Using Pixel-Level Analysis of Graphical Interfaces. Morgan Dixon, James Fogarty, and Jacob O. Wobbrock. (2012). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '12. ACM, New York, NY, 3167-3176. 23%.
https://web.archive.org/web/20150714010941/http://homes.cs.w...
>Video: Content and Hierarchy in Prefab: What if anybody could modify any interface? Reverse engineering guis from their pixels, addresses hierarchy and content, identifying hierarchical tree structure, recognizing text, stencil based tutorials, adaptive gui visualization, ephemeral adaptation technique for arbitrary desktop interfaces, dynamic interface language translation, UI customization, re-rendering widgets, Skype favorite widgets tab:
http://www.youtube.com/watch?v=w4S5ZtnaUKE
>PDF: Content and Hierarchy in Pixel-Based Methods for Reverse-Engineering Interface Structure. Morgan Dixon, Daniel Leventhal, and James Fogarty. (2011). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '11. ACM, New York, NY, 969-978. 26%.
https://web.archive.org/web/20150714010931/http://homes.cs.w...
>Video: Sliding Widgets, States, and Styles in Prefab. Adapting desktop interfaces for touch screen use, with sliding widgets, slow fine tuned pointing with magnification, simulating rollover to reveal tooltips:
https://www.youtube.com/watch?v=8LMSYI4i7wk
>Video: A General-Purpose Bubble Cursor. A general purpose target aware pointing enhancement, target editor:
http://www.youtube.com/watch?v=46EopD_2K_4
>PDF: Prefab: Implementing Advanced Behaviors Using Pixel-Based Reverse Engineering of Interface Structure. Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 1525-1534. 22%
https://web.archive.org/web/20150714010936/http://homes.cs.w...
>PDF: Prefab: What if Every GUI Were Open-Source? Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 851-854.
https://web.archive.org/web/20150714010936/http://homes.cs.w...
Morgan Dixon's Research Statement:
https://web.archive.org/web/20160322221523/http://morgandixo...
>Community-Driven Interface Tools
>Today, most interfaces are designed by teams of people who are collocated and highly skilled. Moreover, any changes to an interface are implemented by the original developers and designers who own the source code. In contrast, I envision a future where distributed online communities rapidly construct and improve interfaces. Similar to the Wikipedia editing process, I hope to explore new interface design tools that fully democratize the design of interfaces. Wikipedia provides static content, and so people can collectively author articles using a very basic Wiki editor. However, community-driven interface tools will require a combination of sophisticated programming-by-demonstration techniques, crowdsourcing and social systems, interaction design, software engineering strategies, and interactive machine learning.