This is cool. I just posted the link on a blind gamers' Discord. I don't do much D&D--really want to change that at some point--but they're frequently complaining about D&D Beyond being an accessibility dumpster fire and looking for something else.
I poked it a bit, and really appreciate that command output is automatically spoken under NVDA. I don't imagine that is accidental. :) Thank you.
Now you have me wondering what web-based command lines I might build. Maybe a crazy idea, but have you thought about adding external command support somehow? Accessible map representation is vastly unexplored, and it'd be fun to build that in the context of initiative.sh somehow. Maybe a web service, or a wasm module if you want to be one of the cool kids.
I could also see this being an embeddable widget in something like a Matrix room. Give everyone a shared context that all their commands can reference, maybe with different permission levels. The context might share all characters with the DM, each character with its own player, a representation of the map which they could move their character on in turn, etc. I've probably just scope creeped the hell out of your project, but this is really neat and I'm intrigued.
> I poked it a bit, and really appreciate that command output is automatically spoken under NVDA. I don't imagine that is accidental. :) Thank you.
I'm gratified to hear that. It occurred to me early on in development that this could either be fantastic or an absolute disaster from an accessibility standpoint, so I made a particular point of getting the aria tags correct. However, I'll confess that I haven't actually tested it with a screen reader yet, so perhaps it's better to say that it's not entirely accidental.
You're right that there's plenty of scope creep in what you're suggesting, and it's probably beyond what I'll be able to handle. However, you've given me a strong motivation for open-sourcing the frontend. It's an absolute disaster from a code standpoint, but it's very simple and could be used as a jumping-off point for just about anything that simulates a screen reader experience.
As of a recent release, egui pushes out some events that I'm able to consume and provide TTS feedback for. You can find a simple Bevy example with a bit of event coverage here:
In my game, I'm able to use this for buttons, checkboxes, and single-line text edit fields. Sliders are a bit wonky but I'm hoping to eventually get those working better as well. Much of this development is needs-driven for me, and as of now I only need these few widgets. I certainly wouldn't complain if coverage improved, though. :)
Before anyone jumps on me about screen readers not being the accessibility end game, I do know that. But as a totally blind developer myself, I can't easily work on solutions for low vision, high contrast, etc. And for my needs (simple keyboard navigable game UIs for games targeted at blind players) it works well enough, and should hopefully get better soon.
For true cross-platform accessibility with integrations into native APIs, we'll need something like https://accesskit.dev.
> my needs (simple keyboard navigable game UIs for games targeted at blind players)
Huh... I am surprised that, with that target, you are even using a UI library--and then dealing with its resulting level of accessibility--rather than building something more akin to an IVR tree out of direct usage of text-to-speech and keyboard APIs.
> Each node has an integer ID, a role (e.g. button or window), and a variety of optional attributes. The schema also defines actions that can be requested by assistive technologies, such as moving the keyboard focus, invoking a button, or selecting text.
This sounds very similar to what I'm using for my Semantic UI project (which has similar aims).
Accessibility systems require the ability to programmatically interact with the UI, too (install Accerciser if you're on an AT-SPI2-based system to have a play around); I'm not sure how your system supports typing. (Is it all done via Action::ReplaceSelectedText?)
Also, have you thought about latency? AT-SPI2 is really laggy (“bring down your system for several seconds at a time” levels of laggy), and from a cursory inspection AccessKit looks even heavier.
I'd like to know more about the Semantic UI project.
The way text input is implemented depends on the user's platform and input needs. When using a screen reader with a hardware keyboard, the screen reader will often use the accessibility API to programmatically move the keyboard focus, but once the focus is in a text input control, the input itself happens as usual, not through the platform's accessibility API. For users who require alternate input methods such as speech recognition, it depends on the platform. On Windows, for instance, text input isn't even done through the accessibility API; it's done through a separate API called Text Services Framework. But AccessKit will offer the ReplaceSelectedText action for platforms that can expose it.
I have certainly thought about latency; as a Windows screen reader developer, it has been a difficult problem for a long time. The relevant factor here is not the amount of information being pushed, but the number of round trips between the assistive technology (e.g. screen reader) and the application. If I'm not mistaken, this is what makes AT-SPI problematic in this area. This has also been a problem for the Windows UI Automation API, and a major focus of my time on the Windows accessibility team at Microsoft was to help solve that problem. As for AccessKit, I'll refer you to the part in the README about how applications will push tree updates to platform adapters. Since a large tree update can be pushed all at once, AccessKit doesn't make the problem of multiple round trips any worse.
> The relevant factor here is not the amount of information being pushed, but the number of round trips between the assistive technology (e.g. screen reader) and the application. If I'm not mistaken, this is what makes AT-SPI problematic in this area.
That explains a lot! AT-SPI2 has, as you say, a lot of round trips – and some applications (e.g. Firefox) seem to use a blocking D-Bus interface that means they drop X events while talking to the accessibility bus.
> I'd like to know more about the Semantic UI project.
I don't think it qualifies for a definite article just yet. :-) I got annoyed with the lack of good, lightweight, cross-platform GUIs in Rust, and I tried to make my own, but then faced the same issue with accessibility APIs… so now I'm trying to solve both problems at once: defining a schema and interaction protocol for the semantics of a user interface, as a first-class citizen – all the information needed to construct a GUI interface would be present in the “accessibility data”, but in principle any kind of UI could be generated just as easily. (Of course, a GUI auto-generated from the SUI data would be like a CSS-free webpage; I'm planning to make a proper GUI library too, later.)
There are three types of thing in the schema I've got so far:
• “Widget type” – basically a role. Each widget has exactly one widget type, which implies a certain set of features (e.g. section-with-heading has a heading)
• “Feature” – a group of attributes with a semantic meaning (e.g. the heading feature consists of a reference to the heading widget (which must have the feature providing its natural language representation)). I'm not sure how to deal with stuff like “can be scrolled”, because I still haven't finished bikeshedding things like “should there be implied zero-size features, or should widget types just have a lot of semantics, or should there be a load of explicit-but-redundant mandatory features on every button widget saying it can be pressed?”. (I'm leaning towards the latter, now, under the assumption that simplicity is better than trying to reduce bandwidth.)
• “Event”. Every change to the state of widgets is accompanied by an event. There are separate events for semantically different things even if the same thing happened; for instance, when LibreOffice Calc deletes table cell widgets that have gone off-screen, the widgets have been deleted but the actual cells are still there; that's a different thing to what happens when someone deletes a worksheet, so it should have a different event. This makes SUI retained-mode, but it should be usable with immediate-mode UIs in the same situations as AccessKit is.
I haven't worked out how to represent “alternate interface interacts with program” yet, but I'm leaning towards a second kind of event, with the set of valid user events (and hence what the alternate UI “looks” like) determined by the
Another question is how to represent cursors. Obviously there should be co-ordinate-positional (mouse-like) and cursors over the widget graph, but keyboard-driven GUIs don't behave like either of those things… so do I just let the alternate interface deal with cursors? But then how does the application know what's currently selected by the cursor? (Focus, hover, select… all with different semantics, not all of which I'm aware of.) Maybe SUI should just completely keep out of that, and pass through a cursor ID and various events without trying to fit them to a model?
You can tell I'm not very good at this; if I'd heard of AccessKit earlier than a week into this project, I wouldn't've started it! :-p
Since pretty much every OS supports Unix Domain Sockets, I intended to use that as the communication protocol. Backends for existing accessibility systems (e.g. AT-SPI2, IAccessible2) were planned as daemons, but to be honest I don't know enough about IPC to have planned this out properly, and I haven't really got attached to any one architecture. I don't even know that that would work properly; IAccessible is a COM interface, and afaik Windows cares strongly about which process provides those.
I thought amount of information was a factor in IPC latency (even though computers can download GBs of data over a network in seconds), so I've been distracting myself with trying to “lazy-load” lots of the data. If you're right about latency – which you probably are – then that's worse than useless, and I should just bubble that up.
A final question is: how to deal with reading order? I have no answers to this.
Same. I've been using Bevy to experiment with some accessible game mechanics for the past few months now. The speed from which I can go from "I really want map exploration to report this and that detail on an entity, based on this and that characteristic," to code that just works and does what I want, is astounding. The degree to which I can then optimize that code, to only run it when this or that component changes value for instance, is also amazing. And when some Bevy subsystem or other isn't up to my standards, I can plug in some other library, add a few components in my game, and more or less forget about the details of the lower-level integration. It's an amazingly productive engine. If it'd launched a year earlier, I'd probably have built my just released game in Bevy rather than Godot.
Funny you should say that. I'm a blind screen reader user, and assumed this was some new 7-inch laptop. Since I don't need a huge screen, I'm always on the lookout for small, performant laptops. Seems this one is anything but. :) Glad I read the comments first.
As I mention elsewhere, it's an emergency laptop for me. I am more or less constantly on pager duty but I only get paged like twice a year or so, thus the keyboard gets used perhaps two hours a year or less. It's good enough for that. Forcing me to slow down my typing in those situations is actually a boon.
I'm curious how well the new Raspberry Pi 400 could be adapted as an effective screen-reader–optimized portable computer. No need to waste weight and battery power on the display.
This is a surprisingly personal legal battle for me.
I'm totally blind. For a long while now, iOS has been the mobile accessibility leader, and unless you as a blind person have some specific reason to use Android, you're encouraged to use iOS.
Back in 2009 when Android accessibility was a joke, by which I mean as a developer I found major API issues within a few hours of development, I wrote one of the first and most popular Android screen readers. I think it was at least the first to be publicised even before TalkBack, where by "publicised" I mean TalkBack wasn't in Android's release notes, but I at least had an APK installable via tinyurl and posted to a mailing list. I quite literally had nothing else to do with my time at that point in my life, and Android 1.6 had an accessibility API, so on a lark I started writing a screen reader. The project has since died out, but for a long while, Spiel was a thin on Android.
iOS won't let me do anything like that. Further, iOS won't even let me code for it unless I use MacOS, and while MacOS has some decent accessibility features, it's been a royal pain in my ass to code on. I could elaborate, but to keep this comment brief, take it as a given from a subject area expert that developing on MacOS as a blind person is worse than any other platform I've used to date. And that's saying something, since Linux is my primary platform, and there are a number of blind developers who won't touch it with a 10-foot pole. I don't want to claim that coding on MacOS under VoiceOver isn't possible, but it's more difficult than it needs to be.
Yes, these are two huge corporations slugging it out, and it's hard to muster much sympathy for Epic. But I wouldn't be the developer I am today if it weren't for Android's openness letting me build a screen reader, and its further openness letting me build an accessible GPS navigation app which I've hacked on in some form for around a decade now, and which I've come to rely upon. It bugs me to no end that the more accessible mobile platform is so locked down such that a budding blind software engineer handed an iPhone can't code for it using JAWS or NVDA. So Apple is handing blind people internet-connected mobile computers with all sorts of sensors, and telling them the only way they can develop for these devices is by using a sub-par development environment that's going to fight them every step of the way and, likely, turn them off of software development more broadly. Then maybe we wonder why we don't see more blind software engineers?
So, go Epic. If they start being asses later then of course I'll oppose that, but if they have a big enough saber to start breaking up Apple's stranglehold on the most accessible mobile platform, they've got my support 100%. Hell yeah, the web wouldn't have been possible under Apple's rule. And while I respect those of you who want more secure devices and a curated experience, don't under-estimate the harm caused by locking blind developers out of one of the most accessible computing environments they're ever likely to get. No, jumping through certificate hoops to install something that will expire after a week isn't anything near what I'm asking for.
And to those of you who say "Just tell your blind friends to use a Pinephone," I wish I could, but Linux accessibility infrastructure has no support for touchscreens and touch events. Any of you millionaires reading this interested in funding that development? ;) Serious question, I'll work to put you in touch with the right folks.
Will it be possible to temporarily turn off example.com, let its absence propagate across the network a bit, then turn it back on under a new server? Synapse has been good to me, but I know Rust more than Python, and Conduit's use of Sled is one less moving part for me to maintain for my single-user server. Looking forward to migrating to a lighter server I can potentially hack on, and would rather not change the underlying DNS infrastructure much.
No, the idea is that to migrate you would have to run both servers side by side and then kill the old one. We haven’t yet figured out how you’d point the old matrix id to refer to the new server (as I assume your concern is keeping the same mxid as before), but presumably there will be a way to do so.
Wow, native English speaker and I didn't even get the pun until this comment. I thought it was a stupid play on "What app should we message each other on?" and thus kind of a silly name. But folks use it regardless.
I don't even have to open the link to know that it's that budweiser ad. That's where my mind goes too when I think about the name of the app.
I think whatsapp has a nice ring to it, even if people doesn't know how to write it. Surely being able to write it on a store search is important, but not as important as sounding good.
-----
And does people get the name of Waze? It took me a while, because in this case not being a native English speaker is a real problem.
Also a native English speaker, also had no idea it was supposed to be a pun until right now. I just thought it was another silly name from an Asian company.
Complete tangent, but, by any chance, when you read things, do you not 'hear' a voice in your head reading them?
Curious, because I'm pretty sure the first time I ran across Whatsapp it was in text, but I tend to 'hear' the things I read/write in my head, as though a voice read them to me. And because of that the pun stuck out. It might just be I'm more inclined to look for puns (playing games with them all the time with friends and family), but wondering if that might be part of it: if you just see the name and it's just a word, not something sounded out in your head, you of course would not notice the play on words.
Following on this train, as an fluent English speaker, millennial, Asian American that grew up in Texas public school all my life, and a lover of memes and puns...I also didn't realize WhatsApp was a pun...woah.
As a nonuser of WhatsApp, I don't care who owns it. Thinking the app comes from Asia is understandable; There's a large number of apps coming from Asia whose names sound nonsensical in the West (quite likely some of them have as much meaning as our own apps have, and are just foreign words or sounds). There's not many apps coming from Africa or South America these days (that I'm aware of), while Asia has quite a lot.
Careful about calling something a dumb justification. It's often the simplistic-seeming features that expose unforseen privacy issues, in large part because today's internet is such a complex place. In the end, it may turn out that privacy concerns are unjustified. But dismissing them out-of-hand isn't wise.
Do you have an automated way for turning it off when you're on home wifi? Trying a similar setup, and it isn't immediately clear other than via manual activation how to not use Wireguard in that situation.
The built-in "on-demand activation" is quite good. Can set it to specific SSIDs (white or blacklist) or cellular. I've it on for everything except my home SSID.
Edit: I'm talking about the iOS version, not sure what platform you're using.
No such luck on Android, you need to use tasker or similar. Really a nuisance but not a deal breaker. It would be dreamy to set up the client to not use specific SSIDs.
What iOS client are you using? I am using the Wireguard one (the the twisty snake/dragon) and don't see any of these options. Is this stuff that is done in the config files? Sorry, all new to me.
Presumably minimal latency from phone on home wifi to server on home wifi. I actually have no idea if this is true, or how I'd evaluate it. Does anyone understand how the internet and wireguard works well enough to know if this is true?
That's what I was wondering. I've actually done this using OpenVPN (connected to my home VPN server while on home wifi), and it worked fine. Biggest issue was battery impact, but wireguard might be better in that regard?
Haven't looked at Ghost in a while. The subscription system looks neat. I'd recently considered launching an indie game development site for a very niche style of gaming (audio-only) that could probably benefit from a Patreon-style model, but my margins would be so low that the Patreon cut would be hefty for little to no network effect benefit, and I know better than to try rolling my own.
Is there any mechanism for both subscription tiers and pay-what-you-want? So maybe I had a $5 tier and a $10 tier, and if someone wanted to pay $7 then they could and get the $5 benefits?
I'll research this myself when the 3.0 Docker image becomes available, but thought I'd ask first. Not even sure if Stripe supports that functionality, or if it does, how hard it'd be for me to add and contribute back.
At the moment we've just got a single plan with either a monthly or yearly price point, and adding support for more flexible subscription options is high on the todo list to work toward next!
Ah, this answers the question I had about membership models - I was wondering if it has the option to pay per-thing like Patreon does. ($x per Thing posted, with an optional cap of $y, processed at the end of the month.) Works a lot better for what I do than a flat per-month.
I poked it a bit, and really appreciate that command output is automatically spoken under NVDA. I don't imagine that is accidental. :) Thank you.
Now you have me wondering what web-based command lines I might build. Maybe a crazy idea, but have you thought about adding external command support somehow? Accessible map representation is vastly unexplored, and it'd be fun to build that in the context of initiative.sh somehow. Maybe a web service, or a wasm module if you want to be one of the cool kids.
I could also see this being an embeddable widget in something like a Matrix room. Give everyone a shared context that all their commands can reference, maybe with different permission levels. The context might share all characters with the DM, each character with its own player, a representation of the map which they could move their character on in turn, etc. I've probably just scope creeped the hell out of your project, but this is really neat and I'm intrigued.