The fundamental challenge of GUIs is that they have state as a core concern. Unlike most systems, state is not an implementation detail that can be refactored away. It is central to the problem domain. The end user sees and cares about having a stateful system.
The hardest part of dealing with state in a complex system is maintaining consistency in different places. Some instances of this this can be avoided by creating single-sources-of-truth, but in other cases you can't unify your state, or you're dealing with an external stateful system (like the DOM), and you have no choice but to find a way to keep separate pieces of state in sync.
You could conceive of a GUI as merely a projection of internal state. If you do, then a GUI could be a completely stateless layer that exists solely to render visual content parametrically.
I've been developing a convention in React over the last year that uses this idea and it's very very nice. I'm also trying to write a platform-agnostic UI language specifically for designers that makes this a first class concept.
You're just redefining "GUI" as "the MVC View layer". It's good to make the View a pure projection - this is what React approximates - but the state doesn't stop being a concern of the GUI, it just gets pushed to other places in the system (the internal state of HTML elements, the app's store whether that's in Redux or otherwise, even the server if we're talking about a server-rendered app (I would actually argue the main benefit of a client-rendered app is that it's much less onerous to add GUI state; of course the flip-side is that you end up with a lot more GUI state because it's so easy to add))
While I agree with that to a point, I'd like to point out that the post you're replying to also just describes "refactoring state away" (like what you alluded to in your original post.
So, I think that you're both not wrong: State tends to be one of the (if not the) fundamental problems in most of programming. Often, looking at things from another perspective is very helpful. And we should be uncovering new strategies to do that for all places where we haven't, including the GUI.
Of course, that also means that we arrive at the meta-problem of synchronizing the state backing various systems at one point. Which mostly encapsulates Karlton's "two hard things": Cache invalidation and naming things. :)
> The fundamental challenge of GUIs is that they have state as a core concern. Unlike most systems, state is not an implementation detail that can be refactored away.
I do talk about "unifying state"; eliminating "accidental" state. But any UI that allows for interaction also has essential state which can't be refactored away.
> You could conceive of a GUI as merely a projection of internal state.
That mental model has been the most natural for me as well. It really clicked for me when reading Trygve Reenskaug's descriptions of MVC [1], particularly this paragraph (emphasis mine) and the illustration that follows:
> The essential purpose of MVC is to bridge the gap between the human user's mental model and the digital model that exists in the computer. The ideal MVC solution supports the user illusion of seeing and manipulating the domain information directly. The structure is useful if the user needs to see the same model element simultaneously in different contexts and/or from different viewpoints.
What you describe is the model and view part of the MVC pattern: The view is just a projection of the model.
How does the UI get notified of changes? (like the article discusses some changes might come from different part of the UI, or might even come from external like in a chat client)
How do you handle actions of the user? (Of course in the controller in MVC, but how does it work exactly?)
Thanks for pointing that out... Everything old is new again.
Note that in this case, most modern framework actually focus on what Microsoft called MVVM in Silverlight (Model View View-Model), a lesser strict "clean controller" approach where view and controller are tangled in each other since it's throw away code for the most part, while the model stays clean.
Unless you rewrite the content of every input and textarea and reposition the cursor at every input event, or monitor which element has focus at any given time, you have some DOM components with an internal state.
Focus is the hardest one, because that can move around really erratically. But even that sometimes has to be done in state. IE11 doesn’t have descendant focus selectors so at my last job I wrote a hook for our dropdown children to dump their focus state into a React context so our renderers could stay pure and not rely on CSS and therefore DOM state.
Just last week I implemented a hook to reposition a tooltip based on the mouse cursor.
You do need to think about DOM state a little when doing these things. But I would argue that’s somewhat separate from the activity of building a React UI. It’s pretty rare you can’t just render purely based on the React state, using off-the-shelf hooks.
I have built a Qt app using this approach. I created a vdom for Qt to allow for the view to be a pure function of state. However, the app also has some extensive animation, and it runs on a slow embedded processor. I needed an escape hatch to not recompute the entire view on every animation frame on some screens. I think using something like MobX computed expressions could provide the necessary performance to avoid needing the escape hatch. However, the code base is now large and dynamically typed and extensive change is now hard.
In general this approach involves recomputing the entire UI and diffing against the previous UI on every state change or maybe every 33ms. If the above takes too long, then it’s a problem.
In cases where you're working with primitives/immutable values (constant-time comparable for change detection), you can use plain-old memoization to avoid redundant work. If you're familiar with React, this is basically what PureComponent tries to do. The limitation, of course, is that this doesn't work with arbitrarily-deep, arbitrarily-mutable state objects. That's where really MobX shines.
That's essentially declarative programming for GUIs, like, say, Flutter does. You have some state provider, and the GUI is fully re-rendered when it is notified of a state change. There's many different actual implementations of this paradigm, but that's it in a nutshell. I've been having lots of fun learning this, coming from a more traditional MVC pattern in Qt and the likes.
>> You could conceive of a GUI as merely a projection of internal state.
Absolutely. The error is in trying to treat GUI elements as an actual data model. That and premature optimization trying to only update things as they change.
Once you take that approach the only remaining hard part is that for complex applications, events can trigger state dependent behavior. But that should be at a layer under the GUI toolkit.
The way Tcl/Tk dealt with this was nice: You could watch the value in a variable, getting notified when the variable changed. GUI controls could therefore exactly reflect the content of variables. Of course this comes with a certain hidden overhead.
This complete example will toggle the checkbox every second (1000ms), or the user can click to update the variable. The checkbox watches variable "v".
#!/usr/bin/wish
checkbutton .button -variable v -text "Click me"
pack .button
proc run {} {
global v
after 1000 run
set v [expr !$v]
}
run
MobX is the equivalent for the space of reactive web frameworks, and I agree it's a fantastic model. The cost is that it requires a little "magic", but the benefit is that the entire question of synchronizing state virtually disappears, leaving you to focus on modeling and controlling state (which are still nontrivial)
No. Just no. You simply can't use "MobX" and "fantastic" in the same sentence.
There are just so many problems with MobX. For example reactions - ever tried figuring out why some value changes in a bigger app? Not to mention abysmal tooling (compared to Redux DevTools). But the biggest problem is that everything is... sprinkled with magic. Just give me Redux(-toolkit), thank you very much, I can understand these much more deeply. /rant
If I sound confrontational, sorry about that... I just had the misfortune of inheriting a MobX app. Magic indeed.
I introduced MobX (along with HTML5 UI, React, TypeScript, etc) to a large team and it went swimmingly. Debugging/stepping-through was no more tedious than regular Chrome DevTools debugging.
I recommend using MobX’s strictest setting (where changing reactive objects can only occur in named actions), and restricting reactivity to a monolithic “model” (mdl) object and its children.
It really does make writing fast, correct & good UI a breeze, in my experience.
The thing about MobX's magic is that it is small, and simple, and easy to learn predictably. Once you understand it, it's only "magic" in the sense that it's automatic, not in the sense that it's inscrutable or unpredictable.
This was the "two-way data binding" model that Angular used and React famously knocked down. It's been part of UI toolkits for a long time. Remember Object.observe?
Funny you mention this.
I used to think the same way, and would constantly trip up managing state, until a co-worker who had worked in video games for two decades introduced me to the concept of the Immediate Mode GUI (IMGUI).
In around 2013, Omar Cornut wrote an incredibly high quality practical implementation of Muratori's concept called _Dear ImGui_:
https://github.com/ocornut/imgui
Using both Cornut's library and Muratori's mindset is incredibly powerful and liberating. Things that would require days of work and multiple files in Qt or Cocoa can be finished in four hours and a couple of hundred lines of IMGUI code. It also uses an order of magnitude less CPU/memory resources (which was an issue for us as we were rendering and displaying RAW video footage in real-time).
I find it amazing that this way of thinking hasn't completely dominated frontend programming by now. There is literally no downside to using the IMGUI concept - entire classes of bugs disappear, and your code is simultaneously smaller, easier to maintain and more logical in flow.
It's also a shitload more fun to write (something I think that SWE culture overlooks too much) - you spend the majority of your time writing code that directly renders graphics to the screen, rather than fixing obscure bugs and dealing with the specifics of subclassing syntax.
> There is literally no downside to using the IMGUI concept
There's a big downside, the performance penalty. For a GUI that renders moderately complex objects, the cost of not caching becomes overwhelming, the equivalent of losing 20 years in GL architecture advances. Pushing the VBO to the GPU each frame is the same as losing indexing. My own application doesn't render at 60 fps in immediate mode.
I find that people who believe that IMGUI is somehow faster than RMGUI are game developers who have been taken in by marketing, because basic knowledge of GPU programming (i.e. what is indexed rendering) is enough to see that this couldn't possibly be true. Their UIs are usually simple enough that the performance penalty is not important. And many RMGUIs have heavy styling, visually incomparable to their IMGUI counterparts, which makes the average RMGUI far heavier.
I'm not a graphics programmer, and don't understand the specifics of how a modern GPU works.
I can state from experience that using Dear ImGui over Qt massively sped up the application we developed (Media player with non-trivial UI that decoded RAW video in real-time using CUDA/Metal/OpenCL).
Framerate increased by around 10%, and there was no longer any significant difference between the consumer-oriented software and an internal engineering tool that was literally nothing but an SDL window and some direct API calls.
Perhaps caching was performed driver-side, or perhaps maintaining indexing is not a huge issue on modern GPUs. I couldn't tell you, honestly, but I saw the performance gains with my own eyes. Entire UIs would chew up less resources than a blank QWindow.
I think "media player" is one of the best cases for ImGUI's, as they whole frame changes every time.
The RMGUIs (should?) be much more efficient when the contents are both hard to render and don't change much -- think large amount of styled text, or detailed SVG diagram, or a graph.
(and blank QWindow consumes 0% CPU when there are no screen updates -- this would be hard to beat with IMGUI's even with the best GPU)
> I saw the performance gains with my own eyes. Entire UIs would chew up less resources than a blank QWindow.
I already mentioned this: "many RMGUIs have heavy styling, visually incomparable to their IMGUI counterparts, which makes the average RMGUI far heavier". Dear IMGUI will be faster than the average RMGUI implementation. It's doing way less work. In addition to being cleaner internally.
For simple use cases, Dear IMGUI behaves like an RMGUI internally. (flohofwoe mentions this in another thread.) So its only performance impact is some minor caching overhead, which is far smaller than extra GPU work.
I agree that immediate-mode is a pleasant mental model, though I've heard mixed things about performance. It does a ton of potentially redundant work (though maybe memoization could reduce that), but it removes so much complexity that there's a genuine trade-off. I'm not sure how things shake out in practice; I haven't used it for anything real.
Though the bigger reality for most front-end devs in 2021 is that most of us are building for the web, and the web doesn't really allow for immediate-mode
In practice, performance is excellent - Dear ImGui's OpenGL implementation trounces most other GUI libraries in terms of performance (we're talking 2 orders of magnitude faster than Qt, let alone Electron et al).
I was wondering why this was all so much easier back in VB than it is in the browser, and I think you're right: we've got to keep the state in the DOM and the "useful" state in the program in sync. Whereas in VB there was only one program so only one source of truth (though it was possible to get your GUI desynced from your database, you kinda had to work hard to do that).
I think this is what Elm solved (and then by plagiarising the same approach, React and Vue). Make the interface declarative rather than imperative and voila - state is automatically in sync.
The hardest part of dealing with state in a complex system is maintaining consistency in different places. Some instances of this this can be avoided by creating single-sources-of-truth, but in other cases you can't unify your state, or you're dealing with an external stateful system (like the DOM), and you have no choice but to find a way to keep separate pieces of state in sync.
I should probably write a blog post on this