It's not just about building special experiences. I want to be able to browse normal websites in VR -- VR and uncommon user agents introduce a new set of user inputs that Flutter will need to handle if it doesn't want its websites to just break if someone loads them up in a headset.
Currently as a web dev, if I want my website to be usable by a voice assistant, I don't have to do anything. If I want my website to be usable in VR, I don't have to think about it -- that's Chrome's problem, not mine, I just expose the exact same controls I've always used. If someone is building a user agent that interacts with my site in a novel way, as long as I've built my site well everything just kind of works, because of how the web is architectured.
But if you're rebuilding all of the controls from scratch, suddenly you can't think that way anymore. You have to think about the controls and gestures that every medium exposes, or else you have a website that just randomly breaks depending on what OS/platform a user accesses it from.
> and it's usually addressed by allowing native hooks
Not on the web though. I mean, for obvious reasons on the web we can't just expose native OS methods for websites to call. Flutter doesn't have a choice on that front, Google, Mozilla, and Apple realistically aren't going to break browser sandboxing just so the devs have an easier time.
It's true of course that as a cross-platform toolkit independently built on top of another platform, Flutter will have to consider new controls, gestures, and changes in many platforms. You could say the exact same things comparing Android/iOS/Windows to the cross-platform web: it's very hard to do and there are plenty of compromises, but it's still very valuable. There's been far more handwringing over the web's deficiencies as compared to native OS platforms; why reinvent the wheel with some crappy dynamic language called JS, why rebuild everything with a custom style language and layout engine? Yet Electron and web apps continue to dominate because the trade-offs can still be worth it.
The web is not really adopting hordes of new input modes like WebXR every year, there'll be plenty of time for Flutter to consider new stuff like that before it approaches relevancy in mainstream product development. Accessibility features like voice assistant support don't necessarily come free either. Rich web applications usually don't produce simple/accessible DOM structures even if you're developing with web-native tooling. Flutter or not, accessibility practically always requires some level of manual annotation in web apps, which is Flutter's primary use case.
> Not on the web though. I mean, for obvious reasons on the web we can't just expose native OS methods for websites to call.
I meant native-to-the-web. Flutter's MethodChannels & plugin system works on the web too. If there are browser APIs that you need to use, you can setup hooks to call them from within Flutter.
> You could say the exact same things comparing Android/iOS/Windows to the cross-platform web: it's very hard to do and there are plenty of compromises, but it's still very valuable.
The difference is that the web efforts aren't being headed by one team in one company, they're a giant effort with a huge amount of funding, shared resources, and attention from multiple companies and communities.
Flutter doesn't have that benefit. It's not that nobody could build a cross-platform replacement for the web, it's that the web was hard to build, and it's very unlikely the Flutter devs are going to be able to repeat that effort on their own.
> Yet Electron and web apps continue to dominate
Largely because Electron piggybacks on existing work by embedding the entire Chromium engine, they're not building an entirely new render engine from scratch. And Electron still struggles trying to keep pace with native systems, even though it's largely just hooking into them instead of rewriting them.
> Accessibility features like voice assistant support don't necessarily come free either. Rich web applications usually don't produce simple/accessible DOM structures even if you're developing with web-native tooling.
The web gives you a lot for free though. It's sort of like comparing OpenGL and Qt. Sure, both require me to be thoughtful as a developer, but one requires me to be a lot more thoughtful.
----
The bigger problem here isn't whether application devs need to think about accessibility, it's for the Flutter dev team. It's that the web doesn't expose the kind of accessibility APIs that a pure Canvas UI needs to recreate all of the accessibility and screenreader support that's built into browsers. Qt on the web has run into the same problem. What you end up doing (because it's the only interface you have to work with) is either trying to on-the-fly maintain part of the DOM and working really hard to figure out which parts need to be maintained, or just flat-out adding a second, hidden DOM tree that screenreaders can interact with. The only way that Flutter can tell a screenreader that a piece of text exists is by making a node and putting it into the DOM.
This is, from what I can see, what Flutter does in its forms. It has a DOM representation of all of the input elements (but interestingly enough not the links or other text, so they're not screenreader accessible at all), and then completely separately, it re-implements the entire browser rendering stack. And the performance is... well, you can see the performance.
Flutter's strategy for handling events to those elements appears to be (for lack of a better term) inverted. It's "Render first, worry about the DOM second", which is not the way that I would try to build something like this. I don't want to read too much into it, maybe I'm misunderstanding their architecture, but the tab errors that people are seeing look to me like they exist because Flutter is trying to poorly reimplement how browsers tab, rather than using an underlying DOM model that could for free signal to them what the currently selected element is. Similarly a lot of the input selection and mouse lag is because it looks like they're doing collision detection in WASM and performing a DOM mutation every time they need to change the mouse cursor into a different shape. Which, yeah, of course that's not performant.
Ideally, a framework like this would target the DOM directly in the first place and use that to drive input, accessibility, and styling wherever possible, adding canvases back in only when necessary to override specific parts of rendering. That's the path that frameworks like Blazer and OpenSilver are trying to go down, which is just a lot more realistic of a goal for an individual team to accomplish.
I'm trying to thread a fine line here, because I haven't read Flutter's source code, so I don't know for certain how they're doing everything or why. I don't want to confidently say they're doing something wrong if I don't have the full picture. But I can see the outcome, which is currently pretty bad, and I can see little bits and pieces of the architecture they're using to get that outcome, which all seem very counterintuitive, overly-complicated, and buggy. I'm happy to be proven wrong in the future, but in the present there's a nearly half-second delay to change my mouse cursor when I hover over an input and none of the links in-app are screenreader accessible, so that's not exactly inspiring optimism. Sure, Flutter could theoretically overcome everything and keep pace with the web, but is that what's currently happening? They're announcing v2 and the web version is basically unusable.
I can't say that matches my generally nice experience with the demos on desktop, although mobile scrolling was pretty bad.
As you mention yourself, Flutter Web already implements some things using the DOM and is clearly capable of doing so, it's just a matter of degree. I can believe that Flutter would benefit a lot from using the DOM more, but that could change any time by some Flutter contributor saying "hm the custom tab ordering impl doesn't work too well, let's render a DOM tree for tabs" or something similar for ARIA attributes, text inputs, etc. It's hardly a fatal or intractable issue with the architecture.
Many of these complaints are real issues that Flutter should tackle but they feel pretty overblown. There's a general tenor that jumps from "there are still many issues" to "what's even the point of Flutter Web if it can't get everything right". Flutter Web has to start somewhere and good desktop browser support (which is hardly unusable) is good enough for many real use cases. I don't think anyone is selling it as a mature second iteration which seems like a misunderstanding of the v2 semantic version which applies to the core framework, not Flutter Web that just left beta. If there's any org I trust to do the difficult work of cross-platform support, it's Google which already provides the lion's share of funding, standards design and implementation development to the web. IIRC Flutter's dev team also has Chrome engineers on it. I expect it to attract the attention of more orgs as it matures and hopefully becomes more popular, the core framework has already received open source contributions from Microsoft and Canonical.
> they're not building an entirely new render engine from scratch
The web platform builds an entirely new render engine from scratch. Native OS platforms give you a lot for free too and you see the exact same complaints about the web's independent behavior proven irrelevant again and again as the web/Electron continue to demonstrate incredible value despite reinventing many wheels.
Well, no, I'm claiming that it is a matter of approach, not just degree. There are two ways you can go about syncing the DOM to your rendering. Tabbing is a good example of that:
A) you can build a DOM tree, listen to a focusin event, and then when an input/link element gets focus, you can update your Canvas to match the state of the DOM. This means that you listen to one event and (with some exceptions) your rendering engine mostly just has to worry about rendering. Or
B) you can build a Canvas, reimplement tabbing, mouse selection, touch events, and everything else... and then when your new implementation decides that focus is changing, you can update the DOM to match your Canvas. In short, you can choose to have the Canvas drive the DOM, rather than having the DOM drive the Canvas.
What I'm arguing is that approach B, which Flutter seems to be using, is fundamentally much, much harder, buggier, and less performant than approach A -- and my evidence for that is that GUI frameworks surrounding Rust, Silverlight, and C# that are following similar approaches to A are almost all much more mature than Flutter is, despite having fewer corporate resources devoted to them. I'm arguing that path B is an high-level architectural decision that is not going to yield performant, high-quality results.
> The web platform builds an entirely new render engine from scratch.
Right, but again, the web platform at this point has spent over 20 years doing that, and for the majority of that time it would not have been realistic to build a project like Electron. A lot of this seems to come down to, "is the Flutter team big enough to handle this". I don't think they are, but maybe I'm wrong.
I mean, we can debate this endlessly, or next year Flutter v3 can start releasing demos that can handle smooth scrolling and don't lag in Firefox. My prediction is that Flutter is not going to turn into a performant, good quality target for the web any time within the next couple of years. But again, maybe I'm wrong, and luckily it will be very easy to verify if I am. I'll check back in January 2022 and see if anything has changed.
Currently as a web dev, if I want my website to be usable by a voice assistant, I don't have to do anything. If I want my website to be usable in VR, I don't have to think about it -- that's Chrome's problem, not mine, I just expose the exact same controls I've always used. If someone is building a user agent that interacts with my site in a novel way, as long as I've built my site well everything just kind of works, because of how the web is architectured.
But if you're rebuilding all of the controls from scratch, suddenly you can't think that way anymore. You have to think about the controls and gestures that every medium exposes, or else you have a website that just randomly breaks depending on what OS/platform a user accesses it from.
> and it's usually addressed by allowing native hooks
Not on the web though. I mean, for obvious reasons on the web we can't just expose native OS methods for websites to call. Flutter doesn't have a choice on that front, Google, Mozilla, and Apple realistically aren't going to break browser sandboxing just so the devs have an easier time.