Post author here! I wrote this post five years ago. Since then, my conviction in the value of customizable software has only grown, but I've also updated my thinking in a few ways:
1) AI
AI is rapidly getting better at coding. Current AI is often bad at high-level architecture but is capable of making small local tweaks. Seems like a good fit for the kind of code you need to write a browser extension!
I'm exploring this direction; wrote more about it in "Malleable software in the age of LLMs" [1]
2) Security
Having talked to people who worked on various extension platforms including the browser extensions API, I see more clearly than I did five years ago that security is often the key bottleneck to deploying extension platforms meant for mass adoption. Anytime you want everyday computer users to be installing invasive extensions to important software from untrusted third parties, it's gonna be challenging to protect them.
That said, I still think that conversations around extensions tend to focus too much on security at the expense of all else. Customizability is important enough that it may be worth prioritizing it over security in some cases.
I also think there are many reasonable paths forward here. One is to exchange extensions with trusted parties -- e.g, coworkers or friends -- rather than installing from random people on the internet. Another might be to only build your own extensions; perhaps that'll become more viable with AI-assisted programming, although that introduces its own new security issues. And finally, I've met a few people who have smart ideas for architecting software in a way that helps resolve the core tensions; see [2] for an example.
3) Backend access as a key limitation
I've increasingly realized that the fact that browser extensions can only access client code in a fairly server-centric web means that many deep customizations are out of reach. Perhaps you can't read the data you want, or there's not a write API to do the thing you need.
While I'm optimistic about what extensions can do within the boundary of the client, this is an inherent limitation of the platform.
At Ink & Switch (the research lab I now work for), we're working towards local-first [3] software: collaborative software where the data and the code lives on your device. Among other benefits like privacy, we think this is the right foundation for more powerful extensions, since your data and the app code aren't locked away on a server.
The security problem of open platforms is the key.
Anything that is open enough to let someone who knows what they're doing customize the system to their liking, will also be abused by bad actors persuading people who don't know what they are doing to customize the system in ways that harm them.
The fact I can write my own custom keyboards on Android is great! But the fact someone can convince your grandparents to install a keyboard that includes an embedded key logger is not!
Browser extensions have always been a malware-rich ecosystem. Joking about removing all the toolbars from your parents' Internet Explorer whenever you went home for thanksgiving dates back to about 1999.
Custom keyboards are a great example of an app that - by default - shouldn't have write access to shared resources (that is, no network access, no writing to files which other apps can read).
Adding either of those entitlements to a keyboard app should require extremely scary dialogs. Needs to be possible - perhaps you want your password manager with sync to be part of the keyboard app - but it's clearly a huge risk.
Until you want to be able to download language dictionaries or updated language model.
Or if your keyboard is actually a remote keyboard or shared keyboard taking input from some other devices.
> Until you want to be able to download language dictionaries or updated language model.
You don't need the keyboard application to be able to communicate externally for that. You could have a separate, optional, downloader/installer. That's better for security all around.
Mobile OS vendors have already thought of that and came up with the exact same solution of requiring entitlements to access the network from a keyboard app:
The question is do you actually trust regular users to understand what’s going on when they’re asked for permission to grant an app the ability to do something sketchy?
Bear in mind that on iOS, you can't just prompt for permission; those "regular users" need to be able to navigate to the settings app, find the relevant (deeply nested) section, and enable it there.
That narrows the gap significantly - to users who can't understand the issues, but can (even with the app providing an explanation) find reasonably well-hidden settings.
I've heard from a couple developers over the years that it's entirely impossible to implement a setting that will not be changed by people who don't know what it does.
It doesn't matter if it's behind a footnote, an easter egg, a password input, a magic email code, a call with the main project developer, all of the above, etc. No matter how many steps you try to add, there are still an incredible number of idiots who will mindlessly tap through literally any number of dialogs, warnings, and disclaimers to get to what they want.
Their brain will entirely filter out the path they took. They will probably not even remember a single one of those intermediate steps. The only thing they care about is that they're fixing some problem.
This could be one of the reasons Apple and Google don't want you jailbreaking/rooting your devices. Someone will inevitably make a guide, and millions of idiots will follow it. It will legitimately make the device less secure for them because they won't have any idea what they are doing and likely won't even remember doing it. The only thing they care about is that they're fixing some problem.
This is one reason why some people get so panicked and upset when anything on their computer changes unexpectedly, even if the change is actually harmless. They never actually understood anything. They had managed to accidentally get it how they want it through a combination of stuff that they don't remember. When anything changes, they have to go through that process again.
Look, these people are great at following guides and learning routines. Repetitive, mindless tasks like data entry are perfect for them, because they have no other talent to worry about wasting. But because these people exist, you have to be really careful about what settings you add, no matter how well you think it is hidden, because they will be changed by people who don't know what they're doing.
So far, the devs that have told me this have done so because I asked for some setting to turn off some safeguards, and they said that it's a near-universal request from power users, but they still can't do it, because the rest of their userbase is too clueless to be trusted with that setting. They'd receive bug reports from people who have no clue what went wrong, when the reality is that they disabled the safeguards in order to make something work, and then promptly forgot what happened once it worked the way they wanted. This has supposedly happened so many times in the past that they just don't take the risk anymore.
Anyway, all this is to say that while hiding a setting, as opposed to automatically prompting for it, can definitely rule out a decent chunk of idiots, you will never be able to rule out the resourceful idiots that can mindlessly follow instructions.
I'm biased because I'm neurodivergent, which means I don't have as much experience with neurotypical thought processes.
While I do use search engines and the resultant resources all the time, I don't follow steps completely cluelessly/mindlessly and later forget that I did it. I don't know what the equivalent would be for non-tech - I at least try to understand what a guide is doing so I can reproduce it independently later. I try to develop basic intuition for everything that I do. It is hard for me to imagine someone who lacks that ability. I don't mean to be offensive to anyone in particular, I just use "idiots" for the sake of argument to explain how any setting will eventually be found and changed.
Is it normal to forget the steps you took to accomplish a task? To, say, specifically turn off a setting for crash protection, then completely pull a blank if the program gets into a crash loop later?
It’s not necessarily that you will forget that you changed a setting.
What’s more likely is that if you change a setting with an incomplete mental model of what that setting affects, you might later discover that it opened you up to some risk that you did not appreciate when you made the change.
This affects technical users just as much as nontechnical users, it just kicks in at a different level.
A user who clicks the ‘install anyway’ button on an OS warning dialog telling them they are about to run untrusted software might be doing so without an appreciation of quite how many safety features they just disabled, so when asked later on ‘when did you turn off your firewall?’ they honestly don’t know that was something they ever did.
But likewise, a developer who enables a setting to solve problem A, without realizing that that setting will also screw them when they run into problem B, is… basically the cause of 99% of debugging.
‘It can’t be DNS because that would always be cached, unless there’s some setting that… son of a bitch, who knew that when you enable debug logging it disables DNS caching?’ - some developer somewhere at least once a day
> Yes, it’s very common. Immediately after doing it, in fact.
Do you not even make mental notes of permanent changes you've made to the system...?
I mean, I don't think you'd, say, turn off some crash protection and then later complain about crashes. You'd remember that you previously turned it off, wouldn't you?
> I’m biased right now because you assume stuff about me that you maybe shouldn’t.
I only talked about "typical thought processes" because you said "we all" which I assume meant the general population. Didn't assume anything about you.
Even though the base problem was given to me by another, everything I wrote about "what makes a resourceful idiot / how they are a problem" is based on my personal perception of the ones that I've seen. Which is most likely going to be a neurodivergent's impression of certain neurotypicals. AKA biased.
And the "I don't think" was leading a question, not making an assumption about you.
> Everyone’s experiences and thought processes might be starkly different from each other.
...which is I'm so hesitant to believe that everyone is a resourceful idiot.
And why I made a disclaimer about the fact that my own thought processes might be starkly different from not just who I'm describing, but other brains in general.
At this point I don't really know if you understand what 'neurodivergent' means. People who suffer for neurodivergency does not have different mental mappings than those who are neurotypical. Also, the way they construct their own world does not differ from neurotypical.
The mind process you have described is pretty standard, even using some different things to recover information instead of saving it. There is no neurodivergent path of extracting information and there is no neurodivergent understanding of reality or neurodivergent thought process.
This comment is puzzling to me on several levels, but I'll just go to the centre of the topic. Do you feel that something needs to be addressed about the way LoganDark disclaimed that they were biased on account of their neurodivergency?
> At this point I don't really know if you understand what 'neurodivergent' means. People who suffer for neurodivergency does not have different mental mappings than those who are neurotypical. Also, the way they construct their own world does not differ from neurotypical
to
> Do you feel that something needs to be addressed about the way LoganDark disclaimed that they were biased on account of their neurodivergency
> People who suffer for neurodivergency does not have different mental mappings than those who are neurotypical. Also, the way they construct their own world does not differ from neurotypical.
You would be surprised.
> The mind process you have described is pretty standard, even using some different things to recover information instead of saving it.
Well, I'm glad that it seems accurate at least. I was trying to describe a "standard" process, after all.
I should let you know, though, that my brain doesn't work that way. Reason why I say I'm biased is because I don't see what I described as a particularly interesting way to live life, so my description of it might be overly cynical / insulting.
> There is no neurodivergent path of extracting information and there is no neurodivergent understanding of reality or neurodivergent thought process.
I don't know about a neurodivergent path of extracting information either, but you should know for a fact that certain neurotypes, such as autistic ones, do have a different thought process than normal.
When I think about something, my brain will also pull up every possible related thing and assemble an entire picture for me automatically. This is usually called something like "increased associative ability". I'm just very good at considering very large quantities of facts simultaneously.
It's not the same as being reminded of something I know. It's recalling every thing I know simultaneously that could possibly have any effect or be related in any way. Anything that could possibly have relevance.
I get that "for free" as a part of my neurotype. A neurotypical person would likely have to do that consciously or go through some sort of mental process in order to reproduce the same result. I don't have to do that. It happens automatically and instantly.
But because it happens automatically, I can end up looking really awkward because I tend to not be conscious of my processing delays. For example, someone asks me a question, I go "what?" and then give them an answer anyway before they can repeat it. For a second I thought I didn't hear the question, but it was just processing in the background.
Everything processes in the background for me. Thoughts just evolve on their own, draw from relevant memories on their own. All I really have to do is watch.
Try telling me that everyone's thought process works that way.
No, some set of people will forget. Even if they intended or desired to remember. Mental notes fade for some set of people. And at different time lengths.
hmm, I suppose that's true. I have a lot of friends who also have dissociative disorders, and some of them just dissociate all the time and forget everything, regardless of whether they would've forgotten normally
Executing untrusted code would be a lot safer if browsers and mobile OSes would make it easy to provide fake resources to the app/extension.
Yes, you may read my phone contents, and as far as you know, it's the contents, the whole contents and nothing but the contents - it just happens to be a folder to me. An empty folder. It's a new phone you see.
Yes here's my contact list. Sorry it's mostly empty, there's just the costly premium number in there. I hope your mothership doesn't try to call it.
Yes, here's my microphone. Oh thank you, yes, I do a good impression of Rick Astley.
Pictures on my phone? Oh yes, right this way. It's all pictures of turnips. Do you like them?
Similarly every browser should have the capability to report to sites that the user has notifications enabled when they actually don’t to end those annoying in-site “pre-prompts” which bait you into saying no to the pre-prompts so they can try to ask you again later, rather than just deal with the fact that the user denied permission with the browser-level prompt and isn’t interested.
I don’t think this is a bad idea per se (after all a fundamental principle of the open web is that the user should control the browser). However, although your suggestion is fun, it is mere civil disobedience for geeks.
The million dollar question is: how do you deliver those capabilities (a) without having grandmas phone full of spyware and (b) without giving your favorite Silicon Valley thought leader a 40% cut and total control of the ecosystem?
I don’t have the answer. Just trying to formulate the problem.
> The million dollar question is: how do you deliver those capabilities (a) without having grandmas phone full of spyware and (b) without giving your favorite Silicon Valley thought leader a 40% cut and total control of the ecosystem?
That seems orthogonal? Grandma's phone has the same spyware either way, but this makes it a toss up whether it can spy on anything real
iOS does offer options for "read selected photos" and "add-only photos".
Contact list subset and pseudo-sensors (camera, microphone, accelerometer, barometer) are much needed.
Preset location is also needed, but some apps enforce DRM or other policy by location.
App-level network policy (whitelist, blacklist) is needed. For enterprise MDM, iOS allows per-app VPNs, which could enforce app-specific network filtering. With Apple Configurator policy files, Safari can have on-demand VPNs for specific websites.
> iOS does offer options for "read selected photos" and "add-only photos".
The annoying thing here is how apps insist on either requiring full album access so they can implement their own photo picker or don’t provide a button to re-trigger reselection of “selected photos”.
I wish they’d just use the standard OS selector dialog and call it a day. I don’t care if the standard selector doesn’t meet some stupid product requirement, it’s good enough.
The issue the parent is trying to solve is you don't really have fine grained enough control, or apps nag you and won't load until you give them everything they want. My mom has a cheap camera security app that allows me to see the live streams from remote. Every single time I open the app it asks me again if I want to allow it access to my local network. The answer is a resounding "no". If I could just say "fake yes, here is my fake network", then I wouldn't be continually coerced into giving permissions to something I really don't want to share. I can think of many similar examples, another really common one is giving apps access to my contacts. Absolutely not, stop asking me, here is "Uncle Bob" with phone number 1-222-222-2222. Leave me alone
I wish it were easier to deny internet access to Apps. It isn't a perfect solution but it prevents the simplest data theft. Unfortunately side channel attacks are still too easy: Either a cooperating app, or send once of high value data via a link click opening the browser.
From what I can tell, internet access is the default just to allow apps to have advertising. Too cynical?
Android originally could deny internet access to Apps which I found useful.
Certainly I don't want an extension or plugin to have pull access to the internet. That may limit functionality. But often only push is needed (e.g. blocking list could be pushed). No third-party keyboard should have internet access.
Denying access to apps: if you're on android, you can root it and use AFWall+, which just sets up a basic linux firewall - but apps are installed as individual users, so you can just allow the apps that actually need internet - messengers and browsers, and things you want to sync across networks.
XPrivacyLua for Android does just that. It requires LSPosed, which enables deep modifications of the OS and other apps. Needless to say, that has its own security implications.
Denying "local network" permissions is hilariously worthless. On both Android and iOS all it does is prevent software from sending out multicast packets (for things like device discovery, Chromecast, etc. that don't use DNS-SD), it can still go ahead and just start trying to iterate through the entire RFC 1918 address space and try to connect to everything on your network.
I spent a bunch of time trying to figure out how I would implement such a feature on a standard Linux system to sandbox apps on my PinePhone, but there's no sane way you can implement a standard "you can have internet access but not touch my local network" policy.
Well, maybe the best reaction would be to uninstall the app and give it zero stars.
Of course, if you've bought hardware controlled by it, that's unfeasible. Keep it in mind for next time.
I don't suppose there are review sites that mention how predatory and nagging a mobile app is?
I've basically given up on mobile apps around when the ipad 3 was launched and never looked back. The reasoning being that i got an ipad 1 when it was new, and you could still find pay once games then. But they all got replaced by free to play gambling applications mislabeled as 'games'. Then the news about utility applications tricking you into $50/month subscriptions came about...
I'm so excited about the malleable software / local-first / local-AI crossover, I feel like we are at the dawn of a new era of software. If we play our cards right, we can bring back control of our data from the large corporations, have ownership, and more control of how we work.
I'm particularly interested in how general purpose CRDT toolkits like Automerge and Yjs could become the backing filetype for local-first software with interoperable sync/collaboration backends. The user can then have direct access to the underlaying data via standard tooling. Files can be linked, embedded within each other, forked and merged.
We could have a new hypermedia platform built on this, where all documents are possible to be shared, forked, edited in realtime...
Basically, love what you are all doing at Ink and Switch, excited to see what you publish next.
taking back control from evil corporations is a funding/finance problem, not a technology problem. Everyone dreams of democratized ownership until they have to pay the huge developer salaries. and the go to market costs are even higher than that, all channels are saturated and you have to be louder than the noise.
It’s absolutely a technology problem. The hacker mentality is still the one who innovates and a single person is more than enough to make a significant contribution towards a very different future. That person is probably already working on it.
And here I will interject and argue a third point that it's primarily an organizational problem, and I am already working on it.
Not ready to spill the beans yet though on my projects, first have low back surgery tomorrow to get an artificial disc put in between L5-S1 - and will see how much my overall pain goes down, and how much my productivity can go up - before knowing when I can make any public announcements.
Major limitation of browser extensions is that if you want to just write them for yourself, there's no user friendly, scalable way to install them. There's no way to tell the browser that you trust all extensions in some directory to be loaded automatically and be used without signing and without maybe even having to be packed into XPI file. There's no "put a bunch of code+manifest into a directory and have browser use that" feature. This kind of simple deployment drove me to write a ton of userscripts when greasemonky just loaded plain files from gm_scripts/ subdir of browser profile directory. It was fun and easy to extend websites back then. Mozilla killed all that.
Deployment is just terrible. There's no way I'm sending my extensions somewhere over the internet to get signed after every change so I can use code I wrote on my own computer. WTF distopia is that? Nevermind the last time I checked the tooling for signing is some stupid ass 100MiB+ NPM/node app I have to now trust too. It's bigger than a freaking Linux kernel build itself.
1) AI
AI is rapidly getting better at coding. Current AI is often bad at high-level architecture but is capable of making small local tweaks. Seems like a good fit for the kind of code you need to write a browser extension!
I'm exploring this direction; wrote more about it in "Malleable software in the age of LLMs" [1]
2) Security
Having talked to people who worked on various extension platforms including the browser extensions API, I see more clearly than I did five years ago that security is often the key bottleneck to deploying extension platforms meant for mass adoption. Anytime you want everyday computer users to be installing invasive extensions to important software from untrusted third parties, it's gonna be challenging to protect them.
That said, I still think that conversations around extensions tend to focus too much on security at the expense of all else. Customizability is important enough that it may be worth prioritizing it over security in some cases.
I also think there are many reasonable paths forward here. One is to exchange extensions with trusted parties -- e.g, coworkers or friends -- rather than installing from random people on the internet. Another might be to only build your own extensions; perhaps that'll become more viable with AI-assisted programming, although that introduces its own new security issues. And finally, I've met a few people who have smart ideas for architecting software in a way that helps resolve the core tensions; see [2] for an example.
3) Backend access as a key limitation
I've increasingly realized that the fact that browser extensions can only access client code in a fairly server-centric web means that many deep customizations are out of reach. Perhaps you can't read the data you want, or there's not a write API to do the thing you need.
While I'm optimistic about what extensions can do within the boundary of the client, this is an inherent limitation of the platform.
At Ink & Switch (the research lab I now work for), we're working towards local-first [3] software: collaborative software where the data and the code lives on your device. Among other benefits like privacy, we think this is the right foundation for more powerful extensions, since your data and the app code aren't locked away on a server.
[1] https://www.geoffreylitt.com/2023/03/25/llm-end-user-program...
[2] https://www.wildbuilt.world/p/inverting-three-key-relationsh...
[3] https://www.inkandswitch.com/local-first/