The fundamental challenge of GUIs is that they have state as a core concern. Unlike most systems, state is not an implementation detail that can be refactored away. It is central to the problem domain. The end user sees and cares about having a stateful system.
The hardest part of dealing with state in a complex system is maintaining consistency in different places. Some instances of this this can be avoided by creating single-sources-of-truth, but in other cases you can't unify your state, or you're dealing with an external stateful system (like the DOM), and you have no choice but to find a way to keep separate pieces of state in sync.
You could conceive of a GUI as merely a projection of internal state. If you do, then a GUI could be a completely stateless layer that exists solely to render visual content parametrically.
I've been developing a convention in React over the last year that uses this idea and it's very very nice. I'm also trying to write a platform-agnostic UI language specifically for designers that makes this a first class concept.
You're just redefining "GUI" as "the MVC View layer". It's good to make the View a pure projection - this is what React approximates - but the state doesn't stop being a concern of the GUI, it just gets pushed to other places in the system (the internal state of HTML elements, the app's store whether that's in Redux or otherwise, even the server if we're talking about a server-rendered app (I would actually argue the main benefit of a client-rendered app is that it's much less onerous to add GUI state; of course the flip-side is that you end up with a lot more GUI state because it's so easy to add))
While I agree with that to a point, I'd like to point out that the post you're replying to also just describes "refactoring state away" (like what you alluded to in your original post.
So, I think that you're both not wrong: State tends to be one of the (if not the) fundamental problems in most of programming. Often, looking at things from another perspective is very helpful. And we should be uncovering new strategies to do that for all places where we haven't, including the GUI.
Of course, that also means that we arrive at the meta-problem of synchronizing the state backing various systems at one point. Which mostly encapsulates Karlton's "two hard things": Cache invalidation and naming things. :)
> The fundamental challenge of GUIs is that they have state as a core concern. Unlike most systems, state is not an implementation detail that can be refactored away.
I do talk about "unifying state"; eliminating "accidental" state. But any UI that allows for interaction also has essential state which can't be refactored away.
> You could conceive of a GUI as merely a projection of internal state.
That mental model has been the most natural for me as well. It really clicked for me when reading Trygve Reenskaug's descriptions of MVC [1], particularly this paragraph (emphasis mine) and the illustration that follows:
> The essential purpose of MVC is to bridge the gap between the human user's mental model and the digital model that exists in the computer. The ideal MVC solution supports the user illusion of seeing and manipulating the domain information directly. The structure is useful if the user needs to see the same model element simultaneously in different contexts and/or from different viewpoints.
What you describe is the model and view part of the MVC pattern: The view is just a projection of the model.
How does the UI get notified of changes? (like the article discusses some changes might come from different part of the UI, or might even come from external like in a chat client)
How do you handle actions of the user? (Of course in the controller in MVC, but how does it work exactly?)
Thanks for pointing that out... Everything old is new again.
Note that in this case, most modern framework actually focus on what Microsoft called MVVM in Silverlight (Model View View-Model), a lesser strict "clean controller" approach where view and controller are tangled in each other since it's throw away code for the most part, while the model stays clean.
Unless you rewrite the content of every input and textarea and reposition the cursor at every input event, or monitor which element has focus at any given time, you have some DOM components with an internal state.
Focus is the hardest one, because that can move around really erratically. But even that sometimes has to be done in state. IE11 doesn’t have descendant focus selectors so at my last job I wrote a hook for our dropdown children to dump their focus state into a React context so our renderers could stay pure and not rely on CSS and therefore DOM state.
Just last week I implemented a hook to reposition a tooltip based on the mouse cursor.
You do need to think about DOM state a little when doing these things. But I would argue that’s somewhat separate from the activity of building a React UI. It’s pretty rare you can’t just render purely based on the React state, using off-the-shelf hooks.
I have built a Qt app using this approach. I created a vdom for Qt to allow for the view to be a pure function of state. However, the app also has some extensive animation, and it runs on a slow embedded processor. I needed an escape hatch to not recompute the entire view on every animation frame on some screens. I think using something like MobX computed expressions could provide the necessary performance to avoid needing the escape hatch. However, the code base is now large and dynamically typed and extensive change is now hard.
In general this approach involves recomputing the entire UI and diffing against the previous UI on every state change or maybe every 33ms. If the above takes too long, then it’s a problem.
In cases where you're working with primitives/immutable values (constant-time comparable for change detection), you can use plain-old memoization to avoid redundant work. If you're familiar with React, this is basically what PureComponent tries to do. The limitation, of course, is that this doesn't work with arbitrarily-deep, arbitrarily-mutable state objects. That's where really MobX shines.
That's essentially declarative programming for GUIs, like, say, Flutter does. You have some state provider, and the GUI is fully re-rendered when it is notified of a state change. There's many different actual implementations of this paradigm, but that's it in a nutshell. I've been having lots of fun learning this, coming from a more traditional MVC pattern in Qt and the likes.
>> You could conceive of a GUI as merely a projection of internal state.
Absolutely. The error is in trying to treat GUI elements as an actual data model. That and premature optimization trying to only update things as they change.
Once you take that approach the only remaining hard part is that for complex applications, events can trigger state dependent behavior. But that should be at a layer under the GUI toolkit.
The way Tcl/Tk dealt with this was nice: You could watch the value in a variable, getting notified when the variable changed. GUI controls could therefore exactly reflect the content of variables. Of course this comes with a certain hidden overhead.
This complete example will toggle the checkbox every second (1000ms), or the user can click to update the variable. The checkbox watches variable "v".
#!/usr/bin/wish
checkbutton .button -variable v -text "Click me"
pack .button
proc run {} {
global v
after 1000 run
set v [expr !$v]
}
run
MobX is the equivalent for the space of reactive web frameworks, and I agree it's a fantastic model. The cost is that it requires a little "magic", but the benefit is that the entire question of synchronizing state virtually disappears, leaving you to focus on modeling and controlling state (which are still nontrivial)
No. Just no. You simply can't use "MobX" and "fantastic" in the same sentence.
There are just so many problems with MobX. For example reactions - ever tried figuring out why some value changes in a bigger app? Not to mention abysmal tooling (compared to Redux DevTools). But the biggest problem is that everything is... sprinkled with magic. Just give me Redux(-toolkit), thank you very much, I can understand these much more deeply. /rant
If I sound confrontational, sorry about that... I just had the misfortune of inheriting a MobX app. Magic indeed.
I introduced MobX (along with HTML5 UI, React, TypeScript, etc) to a large team and it went swimmingly. Debugging/stepping-through was no more tedious than regular Chrome DevTools debugging.
I recommend using MobX’s strictest setting (where changing reactive objects can only occur in named actions), and restricting reactivity to a monolithic “model” (mdl) object and its children.
It really does make writing fast, correct & good UI a breeze, in my experience.
The thing about MobX's magic is that it is small, and simple, and easy to learn predictably. Once you understand it, it's only "magic" in the sense that it's automatic, not in the sense that it's inscrutable or unpredictable.
This was the "two-way data binding" model that Angular used and React famously knocked down. It's been part of UI toolkits for a long time. Remember Object.observe?
Funny you mention this.
I used to think the same way, and would constantly trip up managing state, until a co-worker who had worked in video games for two decades introduced me to the concept of the Immediate Mode GUI (IMGUI).
In around 2013, Omar Cornut wrote an incredibly high quality practical implementation of Muratori's concept called _Dear ImGui_:
https://github.com/ocornut/imgui
Using both Cornut's library and Muratori's mindset is incredibly powerful and liberating. Things that would require days of work and multiple files in Qt or Cocoa can be finished in four hours and a couple of hundred lines of IMGUI code. It also uses an order of magnitude less CPU/memory resources (which was an issue for us as we were rendering and displaying RAW video footage in real-time).
I find it amazing that this way of thinking hasn't completely dominated frontend programming by now. There is literally no downside to using the IMGUI concept - entire classes of bugs disappear, and your code is simultaneously smaller, easier to maintain and more logical in flow.
It's also a shitload more fun to write (something I think that SWE culture overlooks too much) - you spend the majority of your time writing code that directly renders graphics to the screen, rather than fixing obscure bugs and dealing with the specifics of subclassing syntax.
> There is literally no downside to using the IMGUI concept
There's a big downside, the performance penalty. For a GUI that renders moderately complex objects, the cost of not caching becomes overwhelming, the equivalent of losing 20 years in GL architecture advances. Pushing the VBO to the GPU each frame is the same as losing indexing. My own application doesn't render at 60 fps in immediate mode.
I find that people who believe that IMGUI is somehow faster than RMGUI are game developers who have been taken in by marketing, because basic knowledge of GPU programming (i.e. what is indexed rendering) is enough to see that this couldn't possibly be true. Their UIs are usually simple enough that the performance penalty is not important. And many RMGUIs have heavy styling, visually incomparable to their IMGUI counterparts, which makes the average RMGUI far heavier.
I'm not a graphics programmer, and don't understand the specifics of how a modern GPU works.
I can state from experience that using Dear ImGui over Qt massively sped up the application we developed (Media player with non-trivial UI that decoded RAW video in real-time using CUDA/Metal/OpenCL).
Framerate increased by around 10%, and there was no longer any significant difference between the consumer-oriented software and an internal engineering tool that was literally nothing but an SDL window and some direct API calls.
Perhaps caching was performed driver-side, or perhaps maintaining indexing is not a huge issue on modern GPUs. I couldn't tell you, honestly, but I saw the performance gains with my own eyes. Entire UIs would chew up less resources than a blank QWindow.
I think "media player" is one of the best cases for ImGUI's, as they whole frame changes every time.
The RMGUIs (should?) be much more efficient when the contents are both hard to render and don't change much -- think large amount of styled text, or detailed SVG diagram, or a graph.
(and blank QWindow consumes 0% CPU when there are no screen updates -- this would be hard to beat with IMGUI's even with the best GPU)
> I saw the performance gains with my own eyes. Entire UIs would chew up less resources than a blank QWindow.
I already mentioned this: "many RMGUIs have heavy styling, visually incomparable to their IMGUI counterparts, which makes the average RMGUI far heavier". Dear IMGUI will be faster than the average RMGUI implementation. It's doing way less work. In addition to being cleaner internally.
For simple use cases, Dear IMGUI behaves like an RMGUI internally. (flohofwoe mentions this in another thread.) So its only performance impact is some minor caching overhead, which is far smaller than extra GPU work.
I agree that immediate-mode is a pleasant mental model, though I've heard mixed things about performance. It does a ton of potentially redundant work (though maybe memoization could reduce that), but it removes so much complexity that there's a genuine trade-off. I'm not sure how things shake out in practice; I haven't used it for anything real.
Though the bigger reality for most front-end devs in 2021 is that most of us are building for the web, and the web doesn't really allow for immediate-mode
In practice, performance is excellent - Dear ImGui's OpenGL implementation trounces most other GUI libraries in terms of performance (we're talking 2 orders of magnitude faster than Qt, let alone Electron et al).
I was wondering why this was all so much easier back in VB than it is in the browser, and I think you're right: we've got to keep the state in the DOM and the "useful" state in the program in sync. Whereas in VB there was only one program so only one source of truth (though it was possible to get your GUI desynced from your database, you kinda had to work hard to do that).
I think this is what Elm solved (and then by plagiarising the same approach, React and Vue). Make the interface declarative rather than imperative and voila - state is automatically in sync.
Here is my personal opinion on it (not sure if it's the right one, though): Write the GUI independently of the model, let the GUI components update and communicate among each other as they like, and perform some validation in these GUI components. Translate between model and view only at one well-defined point at which there is also a final validation at the model side, and make sure model and view are only loosely coupled.
Good GUIs have way too many requirements to be controlled (via a controller) by the model. As a typical example, the fact that a button should only be activated if there is data in a textbox has usually nothing to do with the underlying model. The model should only ever contain valid and consistent data.
This is basically our approach with Blazor now. I am not sure Microsoft has even picked up on or documented our pattern yet, but we do something where:
1) Define a domain model + service(s) that fundamentally addresses the logical business functionality without any notion of a specific UI or its stateful nature. This service should be written in terms of a singleton and injected as such. This is something that should be able to be 100% unit testable, and could be exposed as a prototype in a console application or back a JSON API if necessary (i.e. pretend you are writing a SAAS).
2) Define UI state service(s) that take dependency on one or more domain services. These should be scoped around the logical UI activity that is being carried out, and no further. The goal is to minimize their extent because as we know more state to manage means more pain. These would be injected as Scoped dependencies (in our use of Server-Side Blazor), such that a unique instance is available per user session. Examples of these might be things like LoginService, UserStateService, ShoppingCartService, CheckoutService, etc. These services are not allowed to directly alter the domain model, and must pass through domain service methods on injected members. Keep in mind that DI is really powerful, so you can even have a hierarchy of these types if you want to better organize UI event propagation and handling.
3) Define the actual UI (razor components). These use the @inject statement to pull in the UI state services from 2 above. In each components' OnRender method, we subscribe to relevant update event(s) in the UI service(s) in order to know when to redraw the component (i.e. StateHasChanged). Typically, we find that components usually only inject one or 2 UI state services at a time. Needing to subscribe to many UI state events in a single component might be a sign you should create a new one to better manage that interaction.
This is our approach for isolating the domain services from the actual UI interactions and state around them. We find that with this approach, our UI is quite barren in terms of code. It really is pure HTML/CSS (with a tiny JS shim) and some minor interfacing to methods and properties on the UI state services. This is the closest realistic thing I've seen so far in terms of the fabled frontend/backend isolation principle. Most of the "nasty" happens in the middle tier above, but it is still well-organized and easy to test. By enforcing immutability on the domain services, we ensure discipline with how the UI state services must consume the model. Blazor then goes on to eliminate entire classes of ridiculous bullshit by not forcing us to produce and then consume an arbitrary wire protocol to get to our data.
Not necessarily append-only, but making sure that when we get a copy of something from a method - i.e. GetUser(), that the UI or its state services cannot cause mutations to propagate back down into the store without first going through another explicit method like UpdateUserName().
We don't play around with event sourcing right now, but it might be feasible at and above the UI state services if we wanted to do things like replay a user interaction with our app. The only caveat is that our domain services are very very messy in terms of side effects, so the practical utility is bounded to isolated UI testing.
First, thanks for this response. It's super interesting and get's my mind spun up in a lot of directions it wasn't going before.
Second, with this type of modeling on the separation of GUIs and system, what type of movement do you think there will be in Microsoft and Google going towards an even more minimal computer, almost entirely reliant on the cloud space for compute power. Google's machines are practically there, but from what you've mentioned above seems like a more fully "realized" approach than Google's.
This sounds exactly like what I settled on as React+Redux best practices. There's UI logic+data (that belongs in components) and business logic+data (that belongs in the Redux store) loosely coupled via actions and selectors.
I agree but inevitably you get team members who want to "reduce complexity" by removing similar datatypes. One man's clean adapter is another's DRY violation.
This is a great discussion on the challenges of designing UI for complex applications.
In the aggregate, what ends up being most effective for me, is rapid prototyping, and “paving the bare spots.”[0]
I find that I do a terrible job of predicting user mental models.
The rub with prototypes, is that they can’t be lash-ups, as they inevitably end up as ship code. This means that a lot of good code will get binned. It just needs to be accepted and anticipated. Sometimes, I can create a standalone project for code that needs to go, but I still feel has a future.
So there’s always a need for fundamental high quality.
What has been useful to me, is Apple’s TestFlight[1]. I write Apple software, and TestFlight is their beta distribution system.
I start a project at a very nascent stage, and use TestFlight to prototype it. I use an “always beta” quality approach, so the app is constantly at ship quality; although incomplete.
It forces me to maintain high quality, and allows all stakeholders to participate, even at very early stages. It’s extremely motivating. The level of enthusiasm is off the charts. In fact, the biggest challenge is keeping people in low orbit, so they don’t start thinking that you are a “WIZZARD” [sic].
It also makes shopping around for funding and support easy. You just loop people into the TestFlight group. Since the app is already at high quality, there’s no need for chaperones or sacrifices to The Demo Gods.
I like that it keeps development totally focused on the actual user experience. They look at the application entirely differently from me.
Great post and that first link is brilliant, thank you for sharing. I agree with you regarding your attitude to developing software - just get it out there first and then adjust the UX later.
I'm fairly sure I've said this before here and elsewhere, but bears repeating, especially for this post.
Statecharts is currently probably the most undervalued tool when it comes to programming GUIs with state. Statecharts are a continuation of state machines, but with less footguns and better abstractions to be able to build larger systems.
In the end, you either build a GUI that are using state machines implicitly, or explicitly. Tends to be less prone to bugs if you do so explicitly.
If you're interested, here is some starting points (copied from an older comment of mine):
And finally a very well made JS library by David Khourshid that gives you lots of power leveraging statecharts: https://github.com/davidkpiano
While we're at it, here are some links to previous submissions on HN regarding statecharts with lots of useful and interesting information/experiences:
Write a program that causes the url said (e.g. http://localhost:port/said) to produce a page with an input field and a submit button. When the submit button is pressed, that should produce a second page with a single link saying "click here." When that is clicked it should lead to a third page that says "you said: ..." where ... is whatever the user typed in the original input field. The third page must only show what the user actually typed. I.e. the value entered in the input field must not be passed in the url, or it would be possible to change the behavior of the final page by editing the url.
In no way was that dismissive, I thank you for providing another perspective, that's always welcome in my book!
I don't necessarily agree with all the implementation details of xstate, in particular to where the logic tend to be located in practice, and the reliance on the Actor model for many things in the wild. I rather try to guide people to Statecharts as a paradigm overall, and if you happen to use JS, I think xstate is probably the most mature library there. But as all libraries/frameworks, they can be over-relied upon.
If you're in the Clojure/Script world, which is where I mainly locate myself, then https://lucywang000.github.io/clj-statecharts/ is all you need and so far the library I've had the best luck with.
I just ... don't understand why anyone would do it this way. The code itself already says what to do. Adding this sort of data only subtracts from clarity with no additional flexibility.
You might argue that the data model makes it flexible. But I look at that and go, that's what a function is for. The only thing you need is `(println "transitioned to :yellow!")` inside of a function called transition-to-yellow, or if you're feeling adventurous, a function called transition-to which takes yellow as an argument.
From my perspective, it's better as a data structure because I can do more with it.
It's much easier for me to introspect, and I can easily build dynamic state-machines by changing a data structure, take a look at interceptors[0] as an example. There the stack is dynamically alterable based on what is within the request and each piece of middleware can look at the current context, analyse it and behave accordingly.
I write a lot of workflow based systems with dynamically changing functionality based on user input. This sort of thing is invaluable in that context.
I guess your comment highlight the real problem of trying to describe tools for handling complex scenarios with just simple examples, they often don't make sense because the first thought is always "Why not just have one function instead of all of that?".
If you're just printing some text to the console when doing one thing, then surely one function is enough. Once you start having some state and different views depending on values, normal state machines might be enough. But eventually, your application might grow so much in scope that you want to have state machines nested in other state machines, and you want to avoid the classic "state explosion" that you end up with when using state machines, then Statecharts are a very useful tool.
But before that, do the simplest thing that can work for your use case, complexity is evil and all that... Statecharts are simply a tool for handling complex and intertwined state, not for simple stoplights. Unless those stoplights are all connected to each other and also control/read other state machines, then Statecharts might be right for you.
Not sure if you took a look at the links I put before, but the https://statecharts.github.io/ website has a "Why should you use statecharts?" section with small descriptions and links to more elaborate information as well. Might give you a better view than what I can give.
Fair! I think we just have different perspectives. HN is enormously complex (it has far more complexity than most people realize or truly appreciate), yet it handles every case without any state machine: https://github.com/shawwn/arc/blob/arc3.1/news.arc
And it's nothing but a long list of functions that use closures.
I did take a look at your examples. I just ... well, we'll have to agree to disagree, and I'll try not to be so harsh in my criticisms of what I perceive to be ugly ideas. Just because I think an idea is bad, doesn't mean it's bad.
But there is one specific critique: the state machine approach will make programs longer, and consciously choosing to make programs longer seems fraught with danger. Every additional character you type is an additional character for bugs to hide in.
This moves the subjective "I don't like the style" to something concrete: What's the shortest program you can write with a state machine? My argument is that it's "significantly longer than the equivalent program without a state machine."
No problem! And as just a note, it starts everyday since it is a reoccurring self-paced course. But if I remember correctly, the instructors still answer questions.
I really enjoyed learning about the hierarchical state machines using statecharts.
This isn’t really solving issues for most teams, how do you handle maintenance and tech debt of these state charts? And why only javascript example? You’re missing mobile
Statecharts are the solution to technical debt, not the source, as you're formalizing the possible states the user can be in, and "locking" it to that. You can even apply analysis to your codebase to figure out if you're actually covering all possible states and transitions.
If you take a look at the syllabus of the course, it's not about JavaScript, it's about the formalism and understanding the core concepts, without locking you to a particular technology. Whatever they are teaching in the course, you can apply to JavaScript/Swift as much as you can apply it to Desktop/Mobile.
Disclaimer: haven't actually taken the course, but planning to and I've read the description of it.
I'm sorry, your "original" question doesn't seem to exist in your previous comments, so hard for me to answer it...
You get people to adopt technology or paradigms by explaining the benefits and drawbacks of adopting that set of technology/paradigm and then discussing it together with your team/s. Not sure why it would be different for Statecharts compared to any other paradigm?
What process is too hard for you now exactly, to describe states or something? You're already doing this implicitly when you write UI code with state. Statecharts only changes it from being implicit to being explicit. If you're having a hard time actually naming your states, you can use tools like https://sketch.systems/ to explore what you're building with a simple DSL, then implement it properly in the programming language of your choice.
Teams and technical debt go hand in hand. I don’t mean to sound snarky but this isn’t really practical for large teams to adopt at large companies.
1. writing code adds “debt”
2. Your solution is to now add state charts too which also adds “debt”
Where are these state charts tracked? Who maintains them? When product asks engineering to change code => you now also update state charts. Added technical debt.
If this is difficult for you to understand (the problem I’m describing is very common at large companies) I’m happy to expand more on it.
I'm not sure if you're a developer or not, I'm just gonna assume you're not in order to hopefully be able to describe things better.
Yes, writing code can add debt, but not all code is debt. "Debty" code is code that can be better, but was needed in order to take shortcuts temporarily, probably in order to trade moving faster now against moving faster in the future. If you're taking shortcuts you're gonna have to fix it in the future, or deal with having to move slower/more careful because of it.
And yes, the solution to code debt is to go back and fix it properly. When it comes to UI programming, I'd argue that doing implicit state machines with conditionals scattered all over the place (which is the de facto standard today everywhere I look), is code debt, which can be fixed by REPLACING it with explicit Statecharts. It can also be fixed in other ways obviously, but besides the point here.
The developers would obviously be responsible for the code they write and the Statecharts are all handled in the same source control system your developers already use (typically Git today), so nothing really changes here.
And yes, when you figure out that you have to change the code to do something different, you're gonna have to ask developer to change the code. The same goes for updating Statecharts (that also exist in the code). If you have to change the states/transitions, you're gonna have to update the code that handles the states/transitions. This is the same no matter if you use Statecharts or not.
In the end, Statecharts is not a programming language itself, it's just a way of doing programming. Basically like how functional programming is a way of programming, or object oriented programming is one way, Statecharts is a different way where you're upfront with what states exist in the application.
Again you’ve described a paradigm like functional programming but failed to address my original question.
I’ve been programming for over a decade now, and have seen this symptom over and over again. Someone sees shiny new paradigm => realizes is holy grail => fails to see how it fits into _actual_ working teams.
My problem is that you are going to have a hard time selling this academic approach to _PRACTICAL_ teams doing every day work with every day deadlines and bottom line business dollars.
Until you make this seamless and easy the extra work is just going to be _extra_ work which will be ignored.
I say this as a FAN of Statecharts and functional programming. I myself love this idea. I am looking and prodding people like you for SOLUTIONS to make teams adopt this.
So far you’ve failed to convince me on how to sell this to teams or make it easy to integrate.
> So far you’ve failed to convince me on how to sell this to teams or make it easy to integrate.
Yeah, because I'm not trying to convince you of anything. It's a tool in your toolbox, use it when you think it's advantageous, otherwise use your other tools. I couldn't care less of what you chose to do or how you "sell this to teams". I'm a developer who simply chooses the best tool for the job, sometimes that's Statecharts and sometimes it's something else. Also don't have any idea about what ideas are circling around in "Academia" as I'm far away from that ecosystem and only focus on shipping working products.
If you're looking for something "easy" in particular, then whatever you don't know is gonna be hard. Such is the life of a developer, where sometimes the simpler way is harder but worth it in the end. React was hard for people to grok in the beginning as well, but that doesn't mean it's bad, it just means people are not familiar with it. If you're just looking for easy solutions then yeah, feel free to stop improving and continue use the stuff you already know or is familiar to you.
So in the end, do what you will with the knowledge and experience I've shared with you, I have zero interest in selling you anything and I'm simply discussing stuff here on HN as I'm curious about things and want to have discussions with people who are also curious about things, but this discussion stopped being productive a long time ago.
Developers, probably. But generally, whoever your team/company assigns this task to.
> When product asks engineering to change code => you now also update state charts. Added technical debt.
So is documentation. I would hope a team handles this somehow.
But if you approach it as a problem you actually want to solve, here's an idea: have the canonical statechart in code, and have the visual statechart be generated from that code. Benefits include: your visual/documentation charts are always synchronized with actual code; if somebody starts playing fast and loose with statecharts in code, the picture generator will break - if you treat this as a build error, you've just removed one common source of project rot.
I mean, of course you lift the state. It's just like a database. You have a normalized model containing the state, and the GUI is a view on it. Events from the GUI trigger transactions that update the state and then every element is triggered to update itself based on the new state. That should never trigger a loop of events, because a view updating itself from the model should never trigger transactions on the database.
Many complicated views have their own internal models for things like where they are scrolled, what columns are shown, or what elements of a tree are expanded. But those compound views are written so that, from the outside, they appear exactly the same as any other view.
> I’d love to hear what the functional programming camp has to say about this problem
This is the key line that the original article is missing imo, which solves much of the headache that they claim exists with message passing/raising state:
> a view updating itself from the model should never trigger transactions on the database
i.e. a view rendering from state/messages, should never re-emit new state/messages.
I thought the article was too dismissive of concepts like MVC, immediate mode and functional programming, given that ideas coming from those directions might offer the answers the original author is searching for.
The separation of an underlying data model from any particular way of presenting its current state is a powerful idea that has proven its value many times. We’ve used it to build user interfaces, from early GUIs when we didn’t yet have the kinds of libraries and platform APIs we enjoy today, through games with unique rendering requirements and a need to keep frame rates up, to modern web and mobile app architectures. The same principle is useful for building non-user interfaces, too, in areas like embedded systems where you are “presenting” by controlling some hardware component or distributed systems where you are “presenting” to some other software component as part of the larger system.
Agree completely other than the functional bit (but I'm not sure if that was a recommendation or just Google keyword help). At least for Android, I've found that maintaining state when handling screen rotations etc becomes a real headache if the GUI binding code is functional (there is nowhere nice and clear to hang the state without storing it in an rxStream or similar). So far I favour reactive code absolutely, and functional (ish - I'm still using DI for example) code right up to the view layer, but then drop out for the last mile of data binding
IME, working with an immediate mode UI framework automatically gets rid of most such "architecture astronaut" problems.
But I found that it's almost impossible to describe to someone used to event-/callback-driven UIs why exactly that is. You really need to try it yourself on a non-trivial UI to "get it".
I used and implemented immediate mode GUIs, and I think it’s only good for simple stuff.
Accessibility is hard. The OS needs to know visual tree to be able to conclude things like “this rectangle is a clickable button with Hello World text”.
Complex layouts are hard. Desktop software have been doing fluid layouts decades before the term was coined for web apps, because different screen resolutions, and because some windows are user-resizable. Layout can be expensive (one reason is measuring pixel size of text), you want to reuse the data between frames.
Animations are hard. Users expect them because smartphones introduced them back in 2007: https://streamable.com/okvhl Note the kinetic scrolling, and soft “bumps” when trying to scroll to the end.
Apart from accessibility, isn't this mostly caused by the "immediate mode API" versus "immediate mode implementation" confusion I described in another sub-thread?
There's nothing in the idea that forbids immediate mode UI frameworks from keeping any amount of internal state between frames to keep track of changes over time (like animations or drag'n'drop actions), the difference to traditional UI frameworks is just that this persistent state tree is hidden from the outside world.
Layout problems can be solved by running several passes over the UI description before rendering happens.
For accessibility, the ball is mainly in the court of the operating system and browser vendors. There need to be accessibility APIs which let user interface frameworks connect to screen readers and other accessibility features (this is not a problem limited to immediate mode UIs, but custom UIs in general).
> There's nothing in the idea that forbids immediate mode UI frameworks from keeping any amount of internal state between frames to keep track of changes over time
When you call ImGui::Button("Hello World") it has no way of telling if it’s the same button as on the previous frame, or a new one. You’re simply not providing that data to the framework.
It’s often good enough for rendering, they don’t really care if it’s an old or new button as long as the GPU renders identical pixels. When you need state however (animations certainly do), the approach is no longer useful.
A framework might use heuristics like match text of the button or relative order of things, but none of them is reliable enough. Buttons may change text, they may be dynamically shown or hidden, things may be reordered in runtime.
> Layout problems can be solved by running several passes over the UI description before rendering happens.
You’re correct that it’s technically solvable. It’s just becomes computationally expensive for complex GUIs. You can’t reliably attach state to visual elements, gonna have to re-measure them every frame.
> There need to be accessibility APIs which let user interface frameworks connect to screen readers
Windows has it for decades now. For Win32 everything is automatic and implemented by the OS. Good custom-painted GUI frameworks like WPF or UWP handle WM_GETOBJECT message and return IAccessible COM interface. That COM interface implements reflection of the complete GUI, exposing an objects hierarchy. I’m not saying it’s impossible to do with immediate rendering, just very hard without explicit visual tree that persists through the lifetime of the window.
> When you call ImGui::Button("Hello World") it has no way of telling if it’s the same button as on the previous frame
ImGui does have stable widget ids, otherwise many features which require keeping state across frames wouldn't work. In case of your button example, the widget id is hashed from the label string "Hello World":
To consistently assign these IDs you gonna have a tree on your side of the API. You’ll then have two visual trees, one on your side on the API, another one on the framework’s side of the API. And then you gonna be debugging things.
Who and when should destroy backend visual nodes? If you won’t use an ID in some frame, does it indicate the backend should drop the related state? What if an animation is still playing on the element in question? What if the missing control re-appears later i.e. was only hidden temporarily? What should backend do if you reuse same ID for different things?
One can implement simple stuff using any approach, immediate GUI is often the simplest of them. It’s when use cases are complicated you need a full-blown retrained mode OO framework. Win32, HTML DOM, MS XAML, QT, UIKit are all implemented that way, that’s not a coincidence.
When building more complicated UIs, an architectural pattern I have often found useful is to have two distinct levels of state/data behind the rendering code, so my hierarchy looks something like this:
1. Application data
2. Presentation data
3. Presentation rendering
The first is the single source of truth for your application state, what we often call a “model” or “store”. It’s where you represent the data from your problem domain, which is usually also the data that needs to be persistent.
The second is where you collect any additional data needed for a particular way of presenting some or all of your application data. This can come from the current state of the view (list sort order, current page for a paginated table, position and zoom level over a map, etc.) or the underlying application data (building a sorted version of a list, laying out a diagram, etc.) or any combination of the two (finding the top 10 best sellers from the application data, according to the user’s current sort order set in the UI). This is often a relatively simple part of the system, but there is no reason it has to be: it could just as well include setting up a complicated scene for rendering, or co-ordinating a complicated animation.
The final part is the rendering code, which is a translation from the application and presentation data into whatever presentation is required. There isn’t any complicated logic here, and usually no state is being maintained at this level either. The data required for rendering all comes ready-prepared from the lower layers. Any interactions that should change the view or application state are immediately passed down to the lower layers to be handled.
The important idea is that everything you would do to keep things organised around the application data also applies to the presentation data. Each can expose its current state relatively directly for reading by any part of the system that needs to know. Each can require changes of state to be made through a defined interface, which might be some sort of command/action handler pattern to keep the design open and flexible. Each can be observable, so other parts of the system can react to changes in its state.
It just happens that now, instead of a single cycle where application data gets rendered and changes from the UI get passed back to the application data layer, we have two cycles. One goes from application data through presentation data to rendering, with changes going back to the application data layer. The other just goes from the presentation data to the rendering and back.
I have found this kind of architecture “plays nicely” with almost any other requirements. For example, data sync with a server if our GUI is the front end of a web app can easily be handled in a separate module elsewhere in the system. It can observe the application data to know when to upload changes. It can update the application data according to any downloaded information via the usual interface for changing the state.
I have also found this kind of architecture to be very flexible and to fit with almost any choice of libraries for things like state modelling, server comms, rendering the actual view, etc. Or, if your requirements are either too simple to justify bringing in those dependencies or too complicated for a ready-made library to do what you need, you have a systematic overall structure in place to implement whatever you need directly, using all the same software design and scaling techniques you normally would.
I think layout belongs to the views (you call them presentation rendering).
View models provide data to be visualized, but it’s rarely a good idea to compute pixel positions, or handle low-level input there. These things are coupled with views too much.
Similar to animations. Unless the app being developed is a video editor or similar where animation is the core functionality, animations don’t affect models nor view models, they merely prettify GUI state transitions or provide progress indication. No need to wire them all the way into view models. Some frameworks even run them on a separate thread called “compositor thread” so they play fine even if the main thread is blocked or starved for CPU time.
At first when I adopted this architectural pattern, it was a development of classic MVC so already used the terms “model” and “view”. I actually did adopt the term “view-model” for what I’m referring to here as the presentation data layer. After all, it was the connection between the view and model, so what else should it be called? :-)
However, while there certainly are similarities between the architecture I described above and MVVM — the VM can collect data from the Model and reformat it for the View, and the VM might manage some aspects of the view state — they also differ in some important ways.
Most significantly, MVVM has a linear V–VM–M relationship between the components. The V and M never communicate directly in either direction.
In the architecture I described, the rendering code might depend on both state layers, accessing both application state directly and derived or view-specific data from the presentation state. This avoids any need for boilerplate in the middle layer if nothing special needs to be done with the application data before it’s used for rendering.
Likewise, in response to events triggered from something in the rendered output, messages might be sent to the application and/or presentation state layers so they could update accordingly. The distinction is always clear: any event that can affect application data gets sent there, while anything that affects view state goes to the presentation data layer. In practice, it’s unusual for a single UI event to affect both, so usually only one message is needed. (There is an implicit assumption being made here that the starting point for handling events triggered by user interactions is defined as part of the rendering, which is not necessarily always the case, but let’s gloss over those details for now or this comment will get insanely long.)
In MVVM, there may also be direct two-way data-binding between the V and VM. The architecture I described never assumes that kind of 1:1 relationship between something in the rendered output and the underlying data.
I think layout belongs to the views (you call them presentation rendering). View models provide data to be visualized, but it’s rarely a good idea to compute pixel positions, or handle low-level input there. These things are coupled with views too much.
This is another place where I think the responsibilities in the architecture I described are allocated differently. I want any non-trivial calculations to be in the presentation data layer, and I want the presentation rendering layer to be as dumb as possible. This separates data processing from I/O, which is something I favour as a general principle for organising programs.
In this case, it has the specific advantage that you might have very different strategies for testing them. The presentation data layer is pure computation and can be easily unit tested. The I/O is all about rendering and, in some cases, simulating actions on the rendered output that trigger some sort of message to be sent, so for this you might want some sort of snapshot-based testing or simulated UI environment.
You might also want to incorporate totally different third party libraries or depend on different platform APIs to implement each step. For example, switching out one UI rendering library for another or updating to integrate with some new-style platform API probably shouldn’t affect any of the data (including presentation data like diagram layouts or animation logic) unless we’re specifically shifting responsibility for something like handling animations to a dependency rather than our own code or vice versa.
Some frameworks even run them on a separate thread called “compositor thread” so they play fine even if the main thread is blocked or starved for CPU time.
Sure. One of the nice things about the architecture I described is that it’s entirely neutral about these things. If your requirements so dictate, there is nothing to stop you implementing a more elaborate design at any level of the system. For example, that might include running time-consuming parts of the presentation data logic in separate threads, having changes in the presentation data triggered by timers or other system events, delegating some aspects to hardware acceleration where available, or using any sort of sophisticated caching or scheduling implementation for better performance.
Much of this would add unnecessary complexity in a simple CRUD application GUI. If you’re building a complex, real-time visualisation of an incoming data stream or you’re implementing a game engine that needs to draw part of a large game world with many moving parts, presumably you might have a different perspective on how much extra complexity in the design is acceptable if it gets you the performance you need.
> The V and M never communicate directly in either direction.
There’s nothing wrong in dropping a layer there.
That intermediate VM layer makes sense when the data being presented is not precisely what’s stored in the model. Good place to implement filters or pagination. Good place to validate user input. However, in other cases one doesn’t need any of that, in which case there’s nothing wrong with data binding directly to models.
> the rendering code might depend on both state layers, accessing both application state directly and derived or view-specific data from the presentation state.
In .NET with MVVM that’s easily doable, expose a property on VM that returns the model, and this will work.
> In practice, it’s unusual for a single UI event to affect both
In the apps I work on this happens all the time. User clicks “do something” button, the model needs to start doing that, in the meantime the view needs to disable that button and show “doing something, please wait…” with a progress bar.
For this reason, I normally handle such higher-level events in VMs and call models from there.
> The presentation data layer is pure computation and can be easily unit tested.
I think strongly typed languages handle 90% of what people usually unit test for, with better developer’s UX. If you change something that broke 33 places of other code, a compiler will tell you precisely which 33 places you gonna need to fix.
> switching out one UI rendering library for another or updating to integrate with some new-style platform API probably shouldn’t affect any of the data
I think that only works when switching to something extremely similar. I did when porting iPhone code to iPad, or WPF XAML to UWP XAML, but these were very similar platforms. UI libraries are complicated; replacing them with something else is huge amount of work.
> including presentation data like diagram layouts or animation logic
I don’t believe these pieces are portable in practice. Both coupled with UI libraries too much. For the diagram, half of the 2D rendering libraries are immediate mode other half are retained mode. Animations are totally different as well, CSS animations in HTML don’t have much in common with visual state manager in XAML.
No doubt there are many different architectures that can work well in this sort of situation. I’m just arguing that, contrary to your earlier comment that I first replied to, an immediate mode UI can be one of them even when the requirements are not so simple.
In the apps I work on this happens all the time. User clicks “do something” button, the model needs to start doing that, in the meantime the view needs to disable that button and show “doing something, please wait…” with a progress bar. For this reason, I normally handle such higher-level events in VMs and call models from there.
I used too strong a word before. As you point out, situations where you’d want to update both the application and presentation data aren’t really unusual. You gave one good example. Another might be controlling a modal UI, like closing a dialog box or moving to the next step of a wizard, when maybe you also want to commit data user has entered in a temporary form to the permanent application state. In these cases, handling a UI event would indeed need to get the relevant information to both of the other layers one way or another. This still easily fits within the architecture pattern I was describing, though.
I think strongly typed languages handle 90% of what people usually unit test for, with better developer’s UX.
I am a big fan of using expressive type systems to prevent defects, particularly as an alternative to huge unit test suites full of boilerplate that still don’t do as good a job.
However, no mainstream type system will verify that for a given list in your application data and a given sorting option chosen in the UI, a function in your presentation data layer has generated the correct top 10 items in order. (No unit test can ever verify that property in general either, of course, but at least we can test some representative example cases.)
I think that only works when switching to something extremely similar. I did when porting iPhone code to iPad, or WPF XAML to UWP XAML, but these were very similar platforms. UI libraries are complicated; replacing them with something else is huge amount of work.
I suppose that depends on what you consider extremely similar. Around 10–15 years ago, when what we call “single page applications” today were starting to gain serious traction, if you wanted an interactive visual representation of your data, you probably used one of the proprietary embedded technologies like Flash, Java or ActiveX. But if you were drawing, say, an org chart, you were still ultimately just putting boxes and lines and text in certain places within the allocated area on the page. Today we have tools like SVG, canvas and WebGL available handle the drawing part, but everything we’d want to display in that drawing might still be the same.
I don’t believe these pieces are portable in practice. Both coupled with UI libraries too much. For the diagram, half of the 2D rendering libraries are immediate mode other half are retained mode. Animations are totally different as well, CSS animations in HTML don’t have much in common with visual state manager in XAML.
I think perhaps we have very different types of animation in mind. From the above, I’m guessing you’re thinking of things like pulsing a button when it’s pressed or a little progress spinner while downloading some data? I’m thinking of things like smoothly moving all of the boxes on that org chart to their new positions, including animating the shapes of the paths between them, when the user does something that moves the chart around. Another example might be animating weather data overlaid over a map graphic through the next few hours when the user presses a “play” button. That is, I’m talking about animations where the data to be shown at each step needs to be determined according to complex requirements, not just playing a simple, predetermined effect that a UI library or platform API might handle for you almost for free anyway.
> immediate mode UI can be one of them even when the requirements are not so simple
Just because you can doesn't mean you should.
> a function in your presentation data layer has generated the correct top 10 items in order
There're too many things which can go wrong with visuals. Incorrect order has smaller probability than a bug in a style somewhere which makes data invisible. For the 2nd reason you want to test that thing somehow else anyway: unit tests don't catch rendering bugs. Some other automated tests can in theory, but in practice maintaining these tests wastes a lot of time i.e. expensive.
> Around 10–15 years ago, when what we call “single page applications” today were starting to gain serious traction
Traction is overrated. Around 20 years ago Outlook Web Access (a server-side component of Exchange server) already was a modern single page application, written in HTML and JavaScript. All the required functionality was already there in IE, and even documented nicely on MSDN.
> everything we’d want to display in that drawing might still be the same.
Totally different. SVG renders vector 2D graphics, WebGL is a 3D GPU API. Try to render something slightly more complicated than rectangle of solid color, and you'll see. One example is rectangle with rounded corners: trivially simple in SVG, relatively hard on GPU. A Mandelbrot set is opposite example.
> animations where the data to be shown at each step needs to be determined according to complex requirements
Doesn't matter if the data is static or not. Once you know the data, you do not want to run your code at 60 Hz (or 240Hz if you're really unlucky) computing functions of time and updating things. It's almost always a good idea to use the GUI framework for playing animations. They do better with rendering and power saving.
Sure, but that’s not an argument for not doing it either. As I’ve been trying to explain, I’ve had practical success building applications using the kind of software architecture I described.
I don’t like to get into personal backgrounds in these discussions, because as a rule I think arguments should stand on their own merit. However, just for some perspective, I wrote my first professional GUI around three decades ago. More to the immediate point, the longest I’ve maintained and extended a single GUI built using the architectural pattern I described was over a decade, and that one was probably more complicated in its requirements than a lot of web applications you’d see today. So although it’s just one anecdotal data point, I think it’s a good demonstration that this kind of architecture can work well over an extended period.
There're too many things which can go wrong with visuals. Incorrect order has smaller probability than a bug in a style somewhere which makes data invisible. For the 2nd reason you want to test that thing somehow else anyway: unit tests don't catch rendering bugs.
You seem to be making some implicit assumptions about the platform here with the reference to styling. In any case, this is why I mentioned that you might want a different testing strategy for the presentation rendering layer to the code that collects the presentation data.
Totally different. SVG renders vector 2D graphics, WebGL is a 3D GPU API. Try to render something slightly more complicated than rectangle of solid color, and you'll see.
Sorry, I’m not sure I understand the point you’re trying to make here. I was saying that web-based GUIs might have moved from rendering graphical content using one of the plugins a few years ago to rendering it using one of the native technologies today, without necessarily changing the content itself. Obviously which native technology you’d choose would depend on what kind of visuals you were creating and how you wanted any interactions with them to work.
Once you know the data, you do not want to run your code at 60 Hz (or 240Hz if you're really unlucky) computing functions of time and updating things. It's almost always a good idea to use the GUI framework for playing animations. They do better with rendering and power saving.
To the best of my knowledge, the GUI framework you are describing does not exist. You might be able to extract some basic movements that could be handed off to something running at a lower level with hardware acceleration, but ultimately you still need code to define the way the layout should animate according to whatever rules your system uses, for the same reason you need to define a static layout before or after the animation that way.
I’m not sure what you’re getting at with “doing better with rendering and power saving”. Can you clarify?
> I think it’s a good demonstration that this kind of architecture can work well over an extended period.
I think for me the figure is about 8 years of development of 1 GUI software, with MVVM and .NET.
> implicit assumptions about the platform here with the reference to styling
It's 2021, pretty much all of them support styling now.
> I was saying that web-based GUIs might have moved from rendering graphical content using one of the plugins a few years ago to rendering it using one of the native technologies today, without necessarily changing the content itself.
None of them were able to reuse much code while switching just the renderer: ActiveX written in C++, Flash in ActionScript.
> the GUI framework you are describing does not exist
WPF and UWP definitely exist. I think modern HTML+CSS can do that too, but I ain't a web nor electron developer and not sure.
> ultimately you still need code to define the way the layout should animate according to whatever rules your system uses
It's the matter how you define them. When you offload playing animations to a framework, you tell the framework which properties to animate and how: define function of time (typically supported ones include step functions, polylines, and Bezier), pass metadata about timing and repeats, etc. That's for the hard case when animations depend on the data, when they do not they're made offline by art people.
> Can you clarify?
When you do them manually, i.e. computing functions of time and changing visuals, following happens.
1. Your code runs at refresh rate of the monitor, typically 60Hz, but for high-end gaming monitors can be 240Hz. Even if you aren't animating anything, this alone prevents certain power saving modes from engaging.
2. If you manually change positions of things not just color, your updates gonna invalidate layout.
3. There're multiple ways of measuring time with different tradeoffs about precision and when it's running versus paused.
A framework is in a better position because it drives the main loop. It knows how many nanoseconds passed since the last frame. It knows when to render at 240Hz and when to sleep on GetMessage() because there's nothing to update. If it knows which properties are animated, it can integrate animations with layout for reasonable performance cost.
I think for me the figure is about 8 years of development of 1 GUI software, with MVVM and .NET.
Great. As I said before, there are many different architectures that can work well.
None of them were able to reuse much code while switching just the renderer: ActiveX written in C++, Flash in ActionScript.
But if you’re calling that renderer from JavaScript that has already done everything except the rendering, which would be the case with the kind of architecture I was describing if you were just using the plugin for the final rendering layer…
It's the matter how you define them. When you offload playing animations to a framework, you tell the framework which properties to animate and how: define function of time (typically supported ones include step functions, polylines, and Bezier), pass metadata about timing and repeats, etc. That's for the hard case when animations depend on the data, when they do not they're made offline by art people.
Right. But where do you get those data-driven paths from so you can use them as your key frames to drive your animation? That is what is generated in the presentation data layer in this example. No GUI framework can do this for you, because no GUI framework knows the rules of your system for how to lay out a particular diagram and how a transition between states should appear.
Actually moving lines or rotating text or whatever is then little more than an implementation detail, which can be done manually by the presentation layers or handed off to some platform-provided animation system, as the situation dictates.
Your code runs at refresh rate of the monitor, typically 60Hz, but for high-end gaming monitors can be 240Hz. Even if you aren't animating anything, this alone prevents certain power saving modes from engaging.
This seems like such an extreme case that it’s no longer very useful as an example. Who is using a 240Hz high-end gaming monitor, wanting our UI to produce animations that can keep up with that frequency of updates, yet concerned about some unspecified power saving mode not engaging if we do the maths manually when it would have engaged if the work had been delegated to some system service?
If you manually change positions of things not just color, your updates gonna invalidate layout.
We’re talking about immediate mode rendering. What “layout” is there to invalidate?
> if you’re calling that renderer from JavaScript that has already done everything except the rendering, which would be the case with the kind of architecture I was describing if you were just using the plugin for the final rendering layer
Writing such renderers is huge amount of work. I know because I once did something similar: https://github.com/Const-me/Vrmac When I can, I prefer someone else to do that. Unfortunately, they all do it differently. Not just in terms of pixels on output, but general things.
Like, compare SVG and PostScript – many features are common, yet you gonna need substantially different things on input to make these two formats with the same image.
It’s somewhat similar story about rendering libraries.
> No GUI framework can do this for you
Sometimes XAML can. I remember there was a checkbox in Blend for some target platforms called “fluid layout” or something, the framework automatically replaced some discrete changes with auto-generated animations. Not universally applicable, but sometimes it was good enough.
> Who is using a 240Hz high-end gaming monitor .. yet concerned about some unspecified power saving
On the first page of US Amazon bestsellers in the “Laptops” category there’re a few models with 144Hz displays. These people care, occasionally.
I can agree that was an extreme case, but to lesser extent that applies to laptop users in general. I think laptops have been outselling desktops for a decade now. People don’t like choppy animations; they cause otherwise good GUI to be perceived slow. They don’t like spinning/noisy fans and reduced battery life either.
Not so much, if you’re only asking them to do the last step of translating information you already have into whatever format/APIs are needed for a particular medium, which is essentially what I’m advocating with the architecture here.
Long-lived web apps have had to deal with this several times in recent years as the various plugin technologies have been deprecated and ultimately removed from browsers. It obviously takes a certain amount of time to reimplement that logic on what might effectively be a new platform, but it’s certainly achievable, and it’s easier than rewriting more of your application because you’ve been forced to change platforms.
Sometimes XAML can.
I think perhaps you’re misunderstanding my argument, because it is literally impossible for any GUI framework to do what I’ve been describing. It absolutely requires application code to specify what animation is to be constructed, based on the data available. A GUI framework can then handle the mechanics of actually running that animation, but it can’t know what the animation should be. That’s an application problem.
Take a simple example. Suppose we want to illustrate a sorting algorithm. We will do this by drawing 10 boxes in a row on the screen, labelled with different data values to be sorted. Without me telling you how I want to animate those boxes from one order to another after each step of the sorting algorithm, how will your XAML or whatever know what animations to draw? Are we fading out the items that will change positions and then fading them back into their new positions? Are we going to animate a moving item up out of the row, then slide the other items along one way as the moving item slides above them the other way, then animate the moving item back down to its final place? If we do that, are we going to do both horizontal slides at the same time, or one and then the other? What if we want a hybrid, where the moving item fades out, then we slide the others that need to shift, then the moving item fades back in at its new position?
There is no way any GUI framework can know what should happen in a situation like this. The required behaviour needs to be specified as part of your code. The GUI framework can handle the mechanics of fading things in and out or sliding shapes from A to B according to some timing curve, but only once you’ve told it what needs to happen.
I've tried it at the past and "get it" but that doesn't mean i "agree with it" - to me immediate mode GUIs are the graphical equivalent to...
Running = -1
WHILE Running
PRINT "1) Add record"
IF HasRecords THEN
PRINT "2) Delete record"
PRINT "3) Edit record"
END IF
PRINT "H) Help"
PRINT "X) Exit"
INPUT "Choice: ", I$
IF I$="1" THEN AddRecord
IF (I$="2") AND HasRecords THEN DeleteRecord
IF (I$="3") AND HasRecords THEN EditRecord
IF I$="H" THEN ShowHelp
IF I$="X" THEN Running = 0
WEND
...which, sure, it is ridiculously simple (like imguis) but it can very quickly become unwieldy the more you ask from your user interface. There is a reason why UIs moved beyond that.
My point isn't that you can't make them work for non-trivial cases, after all as long as you can draw something on screen you can do pretty much anything. The point is how well that is done.
For example from a quick look at the code it looks like this project needs to draw in a bunch of unrelated to each other libraries just to get some basic GUI functionality you'd find even in toolkits from the 80s provide (e.g. drawing text or using file dialogs).
And check these sources [0] and [1], this is doing at least as much bookkeeping as in a "retained" GUI toolkit - except it also has to do things that such a toolkit would do automatically, like explicitly drawing the widgets [2]. People at the past were complaining how tools like Visual Basic were mixing presentation and logic, yet the worst you could do in these tools is to have the logic in event handlers (...which if you think a bit about it at least that does make a bit of sense), yet here is actual application logic being part of the drawing code in [3] (this is inside a method View::DrawPlayback, scroll a bit upwards to find it).
Now sure, someone might say that this isn't the best example of immediate GUIs... but this is the one that was brought up as a good example. And TBH all it convinced me is that if (for some reason) i had to use ImGui (the library), to spend some time building a retained GUI wrapper around it :-P.
> , there's no question that immediate mode UIs also work well
I sincerely hope no one looks at the screenshot on that page and thinks "this is something that works well". This UI screams "I've been made that way because that was the easiest way in the tool I'm built with" (which is bad - good tools should not dictate the form of the resulting product)
>I sincerely hope no one looks at the screenshot on that page and thinks "this is something that works well". This UI screams "I've been made that way because that was the easiest way in the tool I'm built with"
That's where you'd be wrong. Profilers traditionally have looked like this regardless of the tool/GUI lib/GUI paradigm they'd been made with.
It's the domain need (to show lots of information, graphs, timelines, stack trees, etc, to compare) that asks for this...
How would a profiling tool UI need to look differently in your opinion?
And why do you think the UI looks that way because of restrictions of the UI framework?
I'm quite sure if the UI would have been written with (for instance) Qt or React, it would look exactly the same, because the UI requirements would be the same (minus the different default style). The question is whether this could have been achieved with less, or "cleaner" code (which I doubt).
And how is that different except that the same information is spread over more screen space and hidden inside tabs? White space and tabs also works in immediate mode UIs.
Probably a decent chunk of the perceptual issue you're seeing comes from their choice of font rendering library, which is noticeably "the IMGUI font rendering library", and looks amateurish. This is stb_truetype, which is a very primitive library, but is easy to install and incorporate. The other library is freetype+harfbuzz, which produces higher quality output, faster, and with lower memory. The cost of freetype is that installation is absurdly difficult, the maintainer is offensive, and the code is ancient, unreadable, and architecturally a disaster.
I don't think the chosen GUI library causes the other UI issues you perceive. The author probably hasn't put much time into learning visual design, and thus fell into whatever the UI library supported. But to produce a better one, the author would have had to do a lot more learning and thinking, which is independent of the library choice.
Immediate mode is great, but it can be hard to implement nice custom event handling on top of it. For example, say you have a collection of buttons. You want to be able to click on one button (the button highlights), drag onto another button (that button highlights and the first button unhighlights), and release on that second button to trigger its action. This is a real issue I ran into while building an immediate-mode UI... I found it hard to implement an interaction like this without a reified "layout" that I could query as the mouse moved around.
I was doing something like that with Godot [1], the approach I took was setting some global "dragged_item" when the drag started, and then each drop target would check if the dragged_item was compatible and if the mouse was within its bounds. This way each drop target would do the check instead of some drag manager having to check whatever was under the cursor.
It's easy to skip drawing frames when no state changes, and for frames where state does change, you can use techniques like rxi's cached software rendering to avoid redrawing the entire screen: https://rxi.github.io/cached_software_rendering.html
How does it handle scrolling? This is typically very fast and smooth (performed by bitblt and redrawing only the newly exposed part on every frame), but in immediate mode you'd have to redraw the entire screen on every frame.
I think this whole efficiency thing is a common misconception. Only the public API appears "immediate mode", the internal implementation doesn't need to be, and usually isn't (e.g. it keeps mutating state between frames instead of building everything from scratch again).
The user-side code basically describes what the UI should look like in the current frame, those "instructions" are recorded, and this recording is reasonably cheap.
The UI backend can then figure out how to "diff" the new instruction stream against the current internal state and render this with the least changes to the screen.
However some immediate mode UI systems came to the conclusion that it might actually be cheaper to just render most things from scratch instead of spending lots of processing resources to figure out what needs to be updated.
In conclusion: "Immediate Mode UI" doesn't say anything how the UI is actually rendered or generally how the internals are implemented, it only describes how the public API works.
Perhaps I should give a different example: a listbox filled with 1 million items, with a scrollbar.
If the public API requires you to give a new paint command on every frame (everytime the scrollbar is dragged), then regardless of whether the underlying rendering engine performs each of these paint commands, you still have to run through every item of the list (and so does the diff'ing code), making this a O(N) operation on every frame.
From what I've seen (in Dear ImGui), this is solved by not giving the 1 million items to the API, instead you can query what range of the list is currently visible, and only describe those to the API (in Dear ImGui this is called a ListClipper).
But I guess different UI frameworks have different solution for this. Creating and updating a 1 million item list wouldn't be a cheap operation in a traditional UI system either.
Ok. Yes my point is not that you can't find efficient ways around the limitations, but my fear is that you end up with a system that is less ergonomic in the end.
These aren't just workarounds. It's often the natural immediate mode api. A lot of immediate mode "scroll view" systems just have a child lambda passed with an index for which element to render, and it only calls the lambda for what's actually in view. (eg: the Gio API in Go)
Concrete examples of code that is not immediate mode api, vs. code that is immediate mode api, both implementing the same thing, can help discuss / reflect on that fear. IME immediate mode is great for mapping tree-ish data to tree-ish UI; when things have a lot of nontree linkages or reordering (eg. when you do drag and drop) it gets trickier. React's JSX / createElement API also feels somewhat immediate mode, tbh; the updates are just scheduled to fire on state changes.
> A lot of immediate mode "scroll view" systems just have a child lambda passed with an index for which element to render
This sounds a lot like paintEvent() in traditional OO-style GUI systems; i.e. event-driven.
So my understanding now is that with immediate-mode callbacks happen within the scope of a function-call rather than from some event-loop. I probably have to look into it deeper to get a good understanding. It is still unclear where state is stored (e.g. for the position of a scroll-view), and if state is passed through the function call tree on every frame.
Yeah the in depth look helps before forming conclusions like "you have to render everything every frame." Key thing is immediate mode can often mean just the sense of the API, not necessarily the draw scheduling.
re: widget-local state -- React is one of the most popular models for that. Basically if widgets are identified over time by the sequence of ids (includes array indices like React keys) on the path to them from the root, state is coherent across that identity, and mount / unmount events exist at the boundary of the identity. Callstack / push-pop based APIs like ImGUI maintain this sequence either impicitly or explicitly in the stack. Then there is some API to read / write to the state store (like React hooks or ImGUI generic state storage) with optional update triggering in async APIs like React's.
It's not difficult to implement a 'virtual' list which renders only the items which are visible. It's a little trickier if your list items are not all the same height, but not impossible.
Modern renderers typically draw scrollable content into a separate texture (or collection of smaller textures that tile the scrollable region) and use the GPU to scroll rather than bitblting on the CPU. You can use the same technique to render the scrollable part of the UI in an immediate mode system.
What if you have your ‘immediate mode’ produce not a direct rendering of the GUI, but a virtual object model describing what the structure of the GUI should be at that moment? Then you diff that object model against the actual object model of the GUI and apply the changes....
That’s how react works. It’s effectively an ‘immediate mode’ abstraction above a component based UI.
That avoids the component connecting and wiring problems, and creates the simple determinism that makes immediate mode systems easier to reason about.
I find that Qt signals and slots works pretty well for managing complexity in GUIs. In this case you would connect a signal that is emitted when inventory table changes state to a slot that changes the appearance in the user avatar. This would probably be done in the pane/dialog that contains them both. They 2 components would remain nicely decoupled.
This approach isn't without it's own challenges of course. For example it is sometimes hard to keep track of what is going on in complex applications with cascades of signals and slots. Some people also hate the fact that signals and slots use auto generated code, but I have never really found that to be a problem in practise.
If one user interaction triggers a "on user interaction" signal, which causes "value changed" signals to fire, is it possible that a widget which depends on multiple of these to get redrawn multiple times?
I'm optimistic about Qt 6's QProperty (I don't know how it compares to FRP or, as someone else mentioned, MobX), but Qt 6 currently does not have KDE libraries, or Linux themes to fit into desktop environments or distros.
Yes, potentially, if that is how you program it. But typically you call QWidget::update() which repaints the widget next time through the event loop, if anything changes. Calling QWidget::update() several times will usually only call one repaint.
One thing you have to watch out for it that programmatic changes can fire signals. If you don't want this you have to add QObject::blockSignals(true);...QObject::blockSignals(false); around your call.
It is, he's just pointing out that the Qt developers figured this out a long time ago. Web developers like to think they're pioneers when it comes to this stuff.
I never really thought of it in those terms. But I guess signals and slots are just way to publish/subscribe to a message bus. It is quite a nice abstraction of it IMHO (which is perhaps why it didn't occur to me!).
This is a really good take on the (still unsolved IMO) problem, though I suspect it only makes sense to people who have experienced all the issues personally, since so many of them are "it gets really awkward with big programs" type problems.
He did miss out a pretty significant flaw of message busses - the producers and consumers of messages are completely decoupled, which makes debugging really annoying because you don't have anything like a stack trace. You just know that your component received a message, and good luck trying to figure out where it was sent from and why.
That's also a big problem with magical reactivity systems like Vue. Your stack trace is pretty much "yeah something changed somewhere so we're updating this component. Don't try and figure it out."
> the producers and consumers of messages are completely decoupled, which makes debugging really annoying because you don't have anything like a stack trace
This is actually a really good point. I don't know why you were downvoted.
There’s always a stack trace. The question is figuring out deep your eventing layer is. Also dispatching an event on one thread and receiving it in a component on another thread essentially negates the trace in this sense too (sometimes depending on your environment).
A solution I’ve always had is to build your message bus with logging in mind initially
I think you misunderstood. Of course there's always a stack trace; you're still executing code. But with message buses and magic reactivity systems your stack trace always just goes to `mainEventLoop()` or `processEvents()` or whatever.
It doesn't go to the thing that actually caused the change as it would if you used direct function calls. I'm not saying it's a deal breaker, it's just a notable downside of those architectures.
As GP said, you have to compensate with some form of logging/tracing. At the very least, if you assign an ID to every event, then to debug an issue you can start logging sends and receives, and work out the relationship.
So nice to see this universal issue being discussed, it doesn't seem to get as much attention as it deserves.
IMO, a great solution here is along the lines of "Lift the state up" / "MV(X)". But... there is a vital detail which is usually missed when deciding how exactly the state in your M gets passed to your V for display: you must refresh your entire V when M changes in any way, not just the bit of V that you think changed. It's the only way to completely remove these difficult to test, hard to spot edge cases that the article discusses.
This is almost impossible to talk about without specific examples, so a while back I wrote such an example that I think distills the core problem, and demonstrates how refreshing the entire V not only solves the problem but typically takes less code to do it: https://dev.to/erdo/tutorial-spot-the-deliberate-bug-165k
> There is also another way of making GUIs called Immediate Mode that is commonly used for drawing user interfaces in games. In this mode the GUI components are no longer subscribing and waiting for events to come, but are instead a part of the main loop that runs at 60 fps and re-render themselves based on the current “global” state.
> Immediate Mode GUIs somehow never reached the critical mass and you are probably not going to find it outside of the games industry.
Isn't this what React is built on? I think this was part of the 'original' selling point by Pete Hunt: https://youtu.be/x7cQ3mrcKaY (2013). Around 20:00 in that video he compares React with the Doom3 engine.
Yes, Flutter does this as well and they’re both very popular. When done properly, UI is a pure function of state and you have some simple mechanism for notifying the UI that state has changed.
How does this work with e.g. a text editor? It doesn't seem practical to have your UI be a pure function of an input which is 100+ MBs of text and formatting.
Both Flutter and React will take the next UI frame and figure out what the diff is between the last frame and the next and only apply the difference. In React it’s called the shadow dom. Flutter will also only render what it needs to so you shouldn’t render the entire file at once. Just render what’s on screen and lazy load as you’re scrolling.
> Clicking on buttons will start triggering events which will modify the state in the model that will in turn start triggering event listeners causing your GUI to flash like a christmas tree. The problem of data bindings and change listeners is that they make it really easy to introduce a hidden circular event listeners that will trigger one another multiple times (event A changes state B and change of state B triggers event A).
Agree with both of these points. You can no longer treat assignment simply as the way you mutate data, you also have to anticipate its effects in the data-binding system.
I imagine the circular event problem could be addressed with static analysis, but I don't know of any framework that does this.
This is a flaw in how the widgets have been implemented. When they serve as both the view and the controller, a common misarchitecture is to “loop back” a value change in the view back into the controller. This loop should _only_ happen as a result of the user interacting with the UI. To fix this, the controller does not modify its value authoritatively. Rather, it makes a request of the model to change the value it represents, then gets a confirmation or correction from the model once the logical state of the application has stabilized.
Had a similar issue while using a JTree, where programmatic nodes modifications generated events as if user had clicked on the tree, such as I had to override and disable a few JTree treatments to avoid triggering a wave of undesired side effects.
That’s why in some framework I´m working on, the only events are those from input devices (keyboard, etc.) and windowing, views don’t generate any and are just here to paint the state and indicate to the controller where things are on the screen.
The idea is to separate the state of the application from the presentation, which in practice leads to not having circular dependencies in the first place. And you can update the display after all changes are finished, so you won't get the flashing xmas tree effect.
The problem is under-specified. When you move a slider, how should the other two adjust? There are many possible solutions and you haven't specified any preference here.
The easiest solution is to maintain a fixed order of preference, something like:
values = [a:0, b:0, c:0]
fn onChange(key, value) {
values[key] = value
for (k,v) in values {
if (k !== key) values[k] = adjust(k)
}
}
fn adjust(key) {
switch(key) {
case 'a': return max(0, values[c] - values[b])
case 'b': return max(0, values[c] - values[a])
case 'c': return values[a] + values[b]
}
}
The alternative is to maintain the latest selections made by the user and use that as the iteration order.
Whatever approach you go with, the "single source of truth" approach of react/vue/svelte + state (whether it is state in hooks, redux, mobx or whatever) holds. The "values" above is the source of truth, and the components just reflect that.
In other words: from a state point of view you don't have three sliders each with a value but a "three-valued component that happens to be shown as three sliders".
You can add a constraint solver on top of it. So when e.g. C changes, you solve for A and B. You then update the state of A, B, C and redraw. This kind of interaction can be handled by a separate library which is not necessarily part of a GUI system.
A constraint solver is overkill here, in particular as you cannot solve for A and B uniquely, you might want to add some further rule, like A and B should grow proportionally, when C is modified.
But otherwise, yes, just use a framework that separates State from UI and has bidirectional updates, like SwiftUI.
React itself doesn't have any way of "listening" to events, so the data is only flowing in one direction always. Redux et al introduces more things on top of that, where you can fire off events ("Actions" in Redux terms) that react/trigger other events, but doesn't really have anything to do with React itself.
Then that some people make React components change state on render/mount/update, is a different failure, but not really a failure of React as much as a failure of the one using React.
But in general no, React doesn't really prevent you from having a component updating and re-rendering because some state changed, and because of that, changes the state again and thus re-renders again, forever.
I have a simple rule for GUI design: build trees not graphs. Write components that accept a state snapshot and broadcast changes. If component A listens for state changes from B, then A is a parent node of B. If A sends state to B, then A is a parent of B. Components reconcile state before broadcasting changes toward the root of the tree.
Often there is a price paid in brevity, but I believe it is worth it. It may seem annoying to propagate a click explicitly through 5 parent components just to sum clicks into a count widget, but as soon as a short circuit is made, you've created a graph, and you lose the ability to isolate GUI sub-trees for testing/debugging.
This makes visual redesigns take foreeeeever though. Imagine moving a component from the main area of your app into a menu dropdown in the navbar, now you have to tear out all those props that you painstakingly passed down through 10 intermediary layers.
What kind of GUI app has a performance problem with the message bus?
We're in the age of 4K, 60 FPS rendering. If any GUI application has a message bus that's strained enough to impact performance, then either the application isn't made for humans (because if all that stuff is doing anything it'd result in a screen updating far faster than it could be read), or there's some horrible bug somewhere that produces a flood.
In reality none, but developers often think it's not a "real" message bus unless you have to install it on a separate cluster of machines (like AMQP), or it breaks your desktop randomly (dbus). The idea that a message bus could be part of the application and very lightweight is unexpected.
I've found the best way to handle this problem is a variation on "lift the state up", but instead of binding synchronous listeners to state changes in the model, have these state changes mark any involved views as "dirty". Then, after all events have been processed for the current run loop, go through all the dirty views and call a single update function on each.
For the example given in the article, the update function could look something like
def View.update():
if model.lightTurnedOn:
self.backgroundColor = red
else:
self.backgroundColor = ibmGray
This way, all view property changes happen in one place, where you can read the code and understand how the view will appear in each possible state. Circular listener loops are impossible, and view properties for animations can even be computed by calling update twice (once before and once after the state change).
That's basically what modern JS frontend frameworks do as well. I suppose as soon as something becomes too complex, keeping track of all interactions is just too complicated and rerendering the entire world efficiently looks like an easier problem to deal with.
I like the current trend of going back to renderless components as well. This way you separate the state changes from the way it looks like. Feels like each component is a miniature MVC framework with front and a back.
> I suppose as soon as something becomes too complex, keeping track of all interactions is just too complicated and rerendering the entire world efficiently looks like an easier problem to deal with.
In fact, it can actually be less work, since you're coalescing changes into a single update() call rather than sprinkling them across observer callbacks. Also, if your update function starts running too slowly, you can always make it more precise by keeping track of which states have changed internally to the view. For example, if setting the background color takes a long time for whatever reason, you can do something like this:
def View.update():
if self.lightWasTurnedOn != model.lightTurnedOn:
if model.lightTurnedOn:
self.backgroundColor = red
else:
self.backgroundColor = ibmGray
self.lightWasTurnedOn = model.lightTurnedOn
Now backgroundColor will only be set if lightTurnedOn actually changed since the last update.
FWIW this is a common approach - you can see it even in Win32 API's invalidation mode (which goes back to the 80s) where you mark parts of the UI as "invalid" and eventually (when you start pumping events again in the main loop) the window system combines all the invalid areas and sends you a message to paint ("validate") the window.
Several toolkits and frameworks provide for "after all other events have been processed" hooks/events for such logic, e.g. Delphi has the TApplication.OnIdle event and later versions as well as Lazarus/FreePascal have dedicated controls for this event and "idle timers" meant to be used for updating any invalidated parts of the UI after all other events have finished. Similarly wxWidgets has wxEVT_UPDATE_UI and i'm almost certain that Qt has something similar too - though i can't find it now.
You can also use the original web page model; the state is on the server, every button click generates an action to change state, and the updated ui is regenerated fron scratch based on the new state.
> I still don’t know what the proper solution to this problem would be. Keep your state manipulations as simple as possible and try not to share any data between different models. Every time I went forward with some fancy listener-binding mechanisms, I’ve ended up causing subtle circular listener recalculations that were extremely hard to debug.
I remember in my first real job I wrote a GUI for an RTOS (all the way up starting with the hardware, device driver, ...). Not knowing anything about how GUIs worked, or about events, I had a main loop which redrew the GUI on every action. This article tells me I wasn't completely wrong, these are called "Immediate Mode" GUIs!
> Immediate Mode GUIs somehow never reached the critical mass and you are probably not going to find it outside of the games industry.
And that's a good thing, because so far, AFAIK, no one has implemented accessibility (e.g. for screen readers) in an immediate-mode GUI. I hope to work on that problem sometime soon.
Can you qualify this? My experience has been the exact opposite. Angular rewrote their entire framework to be more one-way data-binding focused because the two-way in AngularJS was a nightmare. They still have a weird mechanism to allow it, and they still have lots of wiring all over the place.
Modern React, with hooks and functional components, solves the problem posed in the article by choosing option 2, lift the state up. The hypothetical problem (change the avatar background when the “working” light is on) is a non-issue. You wouldn’t add an event listener for the light’s state change, you’d simply pass the “isWorking” prop to both the light and the avatar, and they would each render based on the value of “isWorking”. There is no reason for one component to know about the other; they don’t actually do anything except sit there and look pretty, correctly.
The UI is always backed by a data model. The more explicitly you express that model—by keeping it all in one tree like Redux does, for instance—the simpler your UI becomes to reason about. React won’t (can’t) stop you from doing bad things like hitting random services when a component loads, but its design, especially recently, guides you away from that pitfall.
Regular binding in Angular is straight forward: you can pass params to child component and you can bind to events. Child does not know about parent by design. So it is a good practice baked in Angular.
If it comes to the question. I think you have answered yourself by writing React won't stop you from making bad design decisions.
Excellent writing, this is a great article to show a junior frontend engineer who would otherwise spend their next years having to crystalize these ideas on their own. I wish I had read this 8 years ago.
I rarely ever use a GUI. Certainly not for any recreatonal web use. Unless I'm using a GUI, I do not load the graphics layer. I stay in textmode. No X11, Wayland or whatever is compiled in. Ideally I have a designated computer on the local network that has a graphics layer, a full corporate GUI OS. When I need graphics to view stuff I can send it over the local network to the GUI computer. Otherwise I keep recreational use on text-only computers, away from the whiz-bang corporateOS-controlled computer.
> I’d love to hear what the functional programming camp has to say about this problem, but I guess they are too busy with inventing yet another $20 term for a 5 cent concept
At the end of the article it said something about immediate mode. I do think React or Vue are kind of working like this now? Both requires you define your state of the each component and define how it should look like based on the state values. When you update the state it will then update the view automatically as well based on what you've defined (JSX/template). It is not 30/60FPS though, it is re-rendered when it knows there is a state change (i.e. setState is triggered)
I’ve recently become interested in immediate mode UIs, and find that there are surprising similarities to React. In both cases, components are just functions that must explicitly render their child components and pass down any shared state. It’s conceptually a very clean way to handle UI state.
However, React introduces a lot of complexity to avoid unnecessary DOM updates, which makes me wonder about the viability of an immediate mode GUI in the browser using canvas.
The problem, as always, is how to make immediate mode stuff work with accessibility tools.
The value of essentially creating and mutating a tree structure like the DOM is that things like screen readers and UI automation tools can read it. With canvas they just see a big bitmap.
I've worked on an app with an IMGUI on canvas. It was amazingly fast and responsive. Nothing which was built after that to try and replace it was anywhere near as performant.
Given how IMGUI is a bunch of branches with code that redraws the same pixels all the time, 60x per second, I'm still surprised this is considered very (if not most) efficient UI. It would imply retained-mode UI frameworks are strongly undershooting their theoretically possible performance.
I totally disagree with the author. For me, the problem of state management in the GUI is totally solved: the GUI shouldn't manage state, plain and simple.
If you have a problem synchronizing your views with a light, that's because the light should exist outside of your views.
MVC was invented in the 70s. Why do we have to act that this is not a solved problem?
They just enumerate all child windows, these events are not frequent enough for this to be a performance bottleneck. The check is something along the lines of
Win* GetChildAt(Win* w, int x, int y)
{
size_t i;
if (x >= w->ScreenX1 && y >= w->ScreenY1 &&
x <= w->ScreenX2 && y <= w->ScreenY2) {
for (i=0; i < w->ChildCount; i++) {
Win* child = GetChildAt(w->Child[i], x, y);
if (child) return child;
}
return w;
}
return NULL;
}
Call this on the root window and you have the deepest child at the given coordinates.
(though there is usually a bit extra logic for handling, e.g., invisible and disabled windows)
Well no. There are applications with nice well thought out GUIs.
>"Congratulations, a large amount of your effort will go towards resolving weird message bus problems as opposed to writing the business logic of your app"
Sorry but I do not resolve "weird message problems". I use my own publish-subscribe mostly asynchronous message bus for my GUI apps (actually I use it also for non GUI parts as well). Components (visible or not and including running threads) can subscribe to events. It does not exhibit any performance / memory problems and in combination with the global app state object makes programming interactions a piece of cake.
>"Every codebase is clean and elegant when in the head of a single person."
Simply not true. I recently had to salvage business rules from a codebase written by single person over the course of many years. It was probably one of the messiest code I've ever seen.
>"How big is your codebase and how many people work on it"
Depends on a project. On some I work alone. Some had 2-3 persons. The biggest team I've ever had to lead was about 35 people (not all developers). Properly organizing and splitting work and using mostly experienced people the resulting code was very decent
and quite possible to grasp. Also well documented.
I've written front end for nearly all of my 13 years in this field. I've written some absolute dumpster fires in my time and have also written some very large and complex UI's successfully. I would like to think I have some insight to add here. I agree with a lot of the points brought up in this article. I'd like to add a few more points about why I think FE can be such a pain to get right:
1. Mixed concerns and a lack of application layering. In React code bases and others like them, its not uncommon for me to find business logic embedded directly inside the components themselves. This makes the component inherently coupled to the specific implementation its being used and can only be re-usable in other contexts by passing in modifier flags to the props etc. In my opinion, components should be mostly of the stateless flavor and delegate anything not directly related to their immediate UI concerns elsewhere. This increases the likelihood of re-usability of your components and makes testing business and component logic much much more straight forward.
2. This might just be my personal experience, but I've noticed a bit of a dismissive attitude around design patterns and traditional computer science principles among my front end brethren. YAGNI and all that. While I think its fair that the UI != to the back end, I think frequently the baby gets thrown out with the bath water. For instance, I frequently leverage dependency injection in my front end work as a way to make my code more flexible and testable. It's hard to sell the benefits of something like DI until your application grows in complexity to the point that you are dealing with things like nested mocks in your tests and/or need to make a non trivial change to some piece of functionality. I've been seeing the winds start to shift a bit on this which is encouraging.
3. Most of the time there is little to no overall _conceptual integrity_ to a given front end code base. Its uncommon for me to come into an existing code base and have anyone be able to tell me what the architecture is comprised of. Things like what logic lives where and why. I'm not saying people _don't_ do this, but in the more gnarly code bases I've encountered, this "accidental architecture" is more common.
4. Front end is still see as "easy" and the place you stick the less experienced engineers first. I sincerely hope this doesn't come off like I am gatekeeping or anything. I work with some absolutely brilliant people who are only a year or two into their career and much smarter than me. IMO its less about skill and more about having been burned enough times to know where unexpected complexity lie.
I love front end. Its challenging and interesting and maddening. My hope is that it will continue to mature and grow to the point that it gets the same respect as other disciplines within this field. These days I work full stack so its not all I do, but it will always be my favorite. :)
You're absolutely right, I see all the same things as you. I think there are a number of factors
1. Library authors optimize aggressively for the beginner. They specifically sell their tools as being all in one and you can do everything in the view layer. Maybe its because they legitimately want to set their users up for success or maybe they'd rather gain users even if they are setting them up for issues in the long run
2. The kind of people working on web frontend aren't necessary coming from a programming background. You have the web designer/developer types who were doing a great job when they were building out static templates that got integrated into some server rendering back end. Now that everything has become an app, the idea of handling all the associated complexity is daunting. A lot of self taught developers also come in through web UI and the last thing they want to think about is proper architecture.
I'd go so far as to say there is a current of anti-intellectualism in terms of frontend application design. There is a large section of developers who want a small number of tools to do everything for them and it seems they are unwilling to even consider creating their own architecture of which individual libraries are their own self contained piece.
> I'd go so far as to say there is a current of anti-intellectualism in terms of frontend application design
I try to walk a line here between relying on my experience and being open to the idea that there is possibly a better way to do things that is just unfamiliar to me. "Beginner mindset" and all that. React hooks are a great example. At first I hated them because I didn't really understand what problems they solved well. To my eyes they encouraged mixing concerns in the component layer in ways that made testing and re-usability way harder. It was when I started considering hooks as "headless components" that I started understanding where they fit into a front end architecture. I think they solve a specific class of problem that react had no answer for previously. I don't use them for everything but now that I consider them as a special kind of component, they fit into a larger set of tools that I have at my disposal.
I also think there are different kinds of front end developers and we all get lumped into the same bucket right now. There are those that work on flashy UI stuff (which I suck at) and those that do more "middle tier" type stuff where they are building out web apps that solve business problems (more my sweet spot). Those two skill sets are way way way different from one another to the point that I think they should be classified as different disciplines.
I would say I have the same strengths as you. I kind of see it like a "mid-stack engineer" or "middle tier" like you say. It's a lot of data wrangling. It can happen on the "back" of the client or the "front" of the server. I'm no good at design or meticulously tweaking animations. Neither am I interested in configuring Kubernetes or setting up a complex architecture of interacting backend services.
I get what you mean about hooks, I was initially apprehensive and like you I realized the problem they solved. I think the problem we have now, is they are the answer to everything, along with Context. Code that could have been a plain Javascript module is a hook that uses Context to pass refs between components.
I think a big issue is reactivity outside of a reactive view library isn't talked about much. You have libraries like RXJS and Effector, which I'm a big of, that let you handle reactivity separately from a view but they're still full black boxes themselves. I'm not sure many people think about reactivity as a quality separate from any library.
The libraries react-query and svelte-query are an interesting manifestation of this. I haven't diffed the code or anything but they share a core that is essentially the same. Observer based reactivity to handle async calls. Yet there are two different repos with duplicated code providing different frontends based on React and Svelte. I don't want to pick on how the code architecture or anything but it feels like the reason there isn't just a base "query" library that exposes its own Observer API with bindings to React and Svelte is because of the general refusal to accept splitting out responsibilities and there should be a "React way" of doing things.
I'm hoping the TC-39 Observer proposal along with Svelte's idea of a store contract might open a few eyes and show people that the frontend doesn't have to be the closed off black box that it currently is but a lot of people already seem set on the idea that Observers are some RXJS stream based thing that they want no part of.
Anyway sorry for the long reply. I've been thinking about this stuff a lot in trying to establish my course through all this.
TL;DR: there is no one silver bullet for the UI in general.
As a rule practical and manageable UIs use all these approaches at the same time.
Components, class based and/or functional (fe: UserSection, InventoryTable ), are in principle loosely coupled. They may not know about each other and use messages to communicate.
The Flight by Twitter ( https://flightjs.github.io/ ) was/is probably the first practical thing that pioneered that approach.
The Flight is not even a framework but rather a design principle.
In fact it is an anti-framework - it postulates that in practice each component may have its own optimal internal architecture. One component is better to use React-ivity, another - Angular-ish data binding, etc. On some, immediate mode drawing is the most natural.
Smaller the task (component) - greater the chance to find silver bullet for it.
Another option is injecting an optional dependency without connecting the box.
I see OP doesn't mention this option, but is slightly related to both option 1 (connect the box) and 3 (message bus / event emitter). This option is similar to how OS provides an API to user space application program. For example Windows provides an API to an application to flash its window without exposing all of its state like mentioned in option 1. https://docs.microsoft.com/en-us/windows/win32/api/winuser/n...
Here's the detail:
A self-powered object, let's call it workingIndicatorObject, can be introduced to work on the working indicator. It provides 1.) `getState() -> WORKING/NOT-WORKING` a function to get its state, and 2.) `register() -> None`, a function to register user activities. These functions are dependencies for UserSection component and InventoryTable component respectively. In term of lifetime and hierarchy, it precedes and outlives both components.
The workingIndicatorObject MUST have its own lifecycle to regulate its internal state. Its core lifecycle MUST NOT be managed under the same event loop as the GUI components, assuming the program is written on top of a framework that has it (React, Vue, etc). This is to assure that it doesn't directly affect the components managed by the framework (loose coupling). Although, a binding MAY be made between its state and the framework-managed event loop, for example, in React environment, wrapping it as a React hook. An EventEmitter can also be provided for a component to listen to its internal event loop.
Injecting it as a dependency can use features like React Context or Vue's Provide/Inject pattern. Its consumer must treat it as an optional dependency for it to be safe. For example, the UserSection component can omit drawing the "working" indicator if it doesn't receive the `getState` function as a dependency.
Internally, the workingIndicatorObject can use a set. The set is used to store a unique value (a Symbol in Javascript). `register` function creates a Symbol, stores it in the set, and deletes it after $TIMEOUT, while firing event both at addition and deletion. `getState` function is a function that returns `set.size()`. When bound to the component, an addition to the set fires an "update" command to the component, which redraw the component along its internal state. This is just one example of implementation and there are other simpler ways to have the same behaving workingIndicatorObject.
This approach allows UserSection and InventoryTable to only know `getState()` and `register()` function and nothing else, and having both of them optional. Using static type system such as TypeScript can help a lot in this since we can pin down the signatures of both function to `null | () => WORKING | NOT_WORKING` and `null | () => void`, where we can enforce both dependent components to check if it exists and to call it with the correct types, otherwise the compiler yells.
Your example is pretty frontend-specific, but I wanted to highlight what I consider its core point, and which is something I only figured out a year ago (incidentally, when I built my own DI system for a game):
As per dependency injection, you inject stuff the other party needs. As per "dependency inversion principle", the thing you inject stuff into owns the interface for what it needs. That means a concrete object you want to inject in many places should be made to conform to interfaces defined by the "recipients" of the injection, which may - and likely will be - different from each other. And in particular, an interface may be as simple as a function.
So, for instance, if a component needs a log sink to write to, and it only ever writes a single type of messages, you do not need to ship the whole logger object (and with it, the knowledge of what a logger is). All it needs is a function `(String) => void`, so all you need to inject is your logger's Info() method.
Thank you for extracting the core point. That is on point!
Another core point is actually the architecture of things, which lives/dies first/last, who depends on who, and how to connect them that each party is "happy" with what's on their plate.
> incidentally, when I built my own DI system for a game
Indeed systems in game are fun to play with! I've had my share of designing a game engine once and I learnt a lot from it too. I'm intrigued, what game were you working on?
> Indeed systems in game are fun to play with! I've had my share of designing a game engine once and I learnt a lot from it too. I'm intrigued, what game were you working on?
It's an unpublished roguelike that I'm working on, slowly, on hobby time. ASCII graphics, focused on distance combat, and with a hybrid ECS/event-based architecture, where the entire game state is stored in an in-memory SQLite database. More of a research project into the benefits of such architectural decisions than an actual game.
The DI system kind of grew out of my design goal to make the game logic testable in headless mode, without rendering or input (I have only so much time to write and maintain it, and I sometimes code over SSH). I started architecting things with dependency inversion in mind, and then figured I could have the whole game's object graph instantiated in a single place. I decided to make it declarative and composable (pretty much concatenative) by writing code that automatically instantiates the objects in correct order, and then I realized I've reinvented a DI container.
"the scrimers of their nation, / He swore, had had neither motion, guard, nor eye, / If you opposed them."
King James Bible:
"For I am persuaded, that neither death, nor life, nor angels, nor principalities, nor powers, nor things present, nor things to come, nor height, nor depth, nor any other creature, shall be able to separate us from the love of God, which is in Christ Jesus our Lord."
Rudyard Kipling:
"But there is neither East nor West, Border, nor Breed, nor Birth, / When two strong men stand face to face, tho’ they come from the ends of the earth!"
Charles Dickens:
"I had youth and hope. I believe, beauty. It matters very little now. Neither of the three served or saved me."
Jane Austen:
"Mary's ailments lessened by having a constant companion, and their daily intercourse with the other family, since there was neither superior affection, confidence, nor employment in the cottage, to be interrupted by it, was rather an advantage."
Thomas Hardy:
"Half an hour passed yet again; neither man, woman, nor child returned."
Samuel Johnson:
"Among these, Mr. Savage was admitted to play the part of Sir Thomas Overbury, by which he gained no great reputation, the theatre being a province for which nature seems not to have designed him; for neither his voice, look, nor gesture were such as were expected on the stage"
The authors above are not cherry-picked, except that I happened to remember the Kipling and St Paul quotations. Not one of the notable English writers I looked up failed to provide me with at least one example of "neither" applying to more than two things.
I thought I'd look up a few more. Once again, every single person I tried yielded positive examples; no cherry-picking here at all. (The ones that took most tries were Macaulay and Chesterton.) One caveat: I only bothered with people for whom I expected there to be a reasonable amount available online for searching; if "neither" has recently become acceptable only when there are exactly two things, that wouldn't show up in what I found. (But I don't think it has.)
John Milton:
"Into this wilde Abyss, / The Womb of nature and perhaps her Grave, / Of neither Sea, nor Shore, nor Air, nor Fire, / But all these in thir pregnant causes mixt / Confus’dly, ..."
Daniel Defoe:
"And now I thought myself pretty well freighted, and began to think how I should get to shore with them, having neither sail, oar, or rudder, and the least capful of wind would have overset all my navigation."
William Wordsworth:
"Neither vice nor guilt, / Debasement undergone by body or mind, / Nor all the misery forced upon my sight, / Misery not lightly passed, but sometimes scanned / Most feelingly, could overthrow my trust / In what we may become"
Thomas Macaulay:
"Neither blindness, nor gout, nor age, nor penury, nor domestic afflictions, nor political disappointments, nor abuse, nor proscription, nor neglect, had power to disturb his sedate and majestic patience."
Jonathan Swift:
"I find likewise that your printer has been so careless as to confound the times, and mistake the dates, of my several voyages and returns; neither assigning the true year, nor the true month, nor day of the month"
George Bernard Shaw:
"But there are men who can neither read, write, nor cipher, to whom the answer to such sums as I can do is instantly obvious without any conscious calculation at all; and the result is infallible."
G K Chesterton:
"He takes us into the schools of inhumanist learning, where there are neither books nor flowers, nor wine nor wisdom, but only deformities in glass bottles, and where the rule is taught from the exceptions."
Since the above are all notable English (or in two cases Irish) writers, here are a few notable Americans.
Herman Melville:
"But neither great Washington, nor Napoleon, nor Nelson, will answer a single hail from below, however madly invoked to befriend by their counsels the distracted decks upon which they gaze"
H L Mencken:
"I am thus neither teacher, nor prophet, nor reformer, but merely inquirer."
Mark Twain:
"It is not pleasant to see an American thrusting his nationality forward obtrusively in a foreign land, but Oh, it is pitiable to see him making of himself a thing that is neither male nor female, neither fish, flesh, nor fowl--a poor, miserable, hermaphrodite Frenchman!"
I didn't actually check exactly what OP had said. I find that OP now doesn't use the word "neither" at all, but I guess the sentence that now says "none of the presented options are without problems" used to say "neither". Which I agree looks very strange. (The corrected text is arguably still wrong: "none" = "not one", and you would say "not one of them is ...", not "not one of them are ...". But almost everyone, almost all the time, uses "are" rather than "is" with "none", so I'm reluctant to call it wrong; correctness in language is determined by how actual language-users actually speak and write.)
So, my apologies: I made what I now think was probably a wrong guess at what the article had done that you didn't like, and my examples aren't really to the point.
I had a look in the OED. With the neither/nor/nor/... construction, it regards any number of branches as normal (as one would hope given the examples above). In "neither of them" and similar usages, its main definition says specifically "of two". It does explicitly countenance using it with more options, but describes this as "Now rare".
So I still wouldn't go so far as to say that what OP originally wrote is outright wrong, but I do agree that it was odd and changing it was probably a good move.
Well, yes, but .. I think I neglected to consider how you do construct neither a nor b nor c (... ) lists. So I was over absolutist in my thinking. You made a good point. You made me think.
There already are too many (supposed) solutions that came about before properly understanding the problem, which is extremely valuable in and of itself.
The hardest part of dealing with state in a complex system is maintaining consistency in different places. Some instances of this this can be avoided by creating single-sources-of-truth, but in other cases you can't unify your state, or you're dealing with an external stateful system (like the DOM), and you have no choice but to find a way to keep separate pieces of state in sync.
I should probably write a blog post on this