For the last weeks until a few days ago I've been unable to access twitter unless doing a hard refresh on every click. Because they installed a faulty service worker of some kind that would break most of the requests. (Both on desktop and android)
And on Android, every time I follow a link to Twitter in an app that opens in a web view, it gives me a faulty page that I have to refresh a few times before I'm able to view it. It loads the page fine, but some rest call or whatever to fetch the tweet crashes.
Edit: Not heard many others complain about this before, so mainly thought it was something about my setup. But the huge amount of upvotes this got suggests some Twitter engineers better look into this..
> And on Android, every time I follow a link to Twitter in an app that opens in a web view, it gives me a faulty page that I have to refresh a few times before I'm able to view it. It loads the page fine, but some rest call or whatever to fetch the tweet crashes.
I get this issue too, I figured it was part of some trick to make me want to use the app.
Maybe it works when navigated to directly, but the only time I ever go to Twitter is if someone links it from Reddit. And without fail that looks like this: https://imgur.com/fQJ27MF
The button to “Try Again” doesn’t work either, you have to manually reload the page.
This is in Apollo on iOS, but it sounds like the same broken behavior happens in web views on Android.
I thought this was intended: any page load that does not finish within x seconds is aborted. It will normally succeed for me on the reload with the cached assets. I understood this to be a (bad) optimization for rejecting slow clients.
Thank you for mentioning this. I've been getting frustrated with my Firefox install for a while because everything is so slow and it eats so much memory. After deleting the 200 service workers [1] I didn't even realize were installed, it is taking wildly less memory and seems generally snappier. I wouldn't call it "night and day", but it's faster.
I had been under the impression that service workers required approval from the user to be installed. I had service workers from websites that I haven't visited in years and don't even exist anymore, sitting there chewing on my RAM every time I started Firefox.
Also Twitter has also been broken for me this way for months.
Service workers have some nice features, but I submit that they need to be something to be whitelisted by users, not something websites can just toss in there whenever they feel like it. The odds of one of them screwing up and eating far more resources than they have any right to approach 1 as the number of them increases. I don't know exactly how they managed to collectively eat 2GB of RAM and I don't care; there is no way they were bringing that much value to me, especially as I permit no website to simply push me notifications.
[1]: A suspiciously round number; it was exactly 200. Is that a limit? I don't see it in about:config.
Thank you for the hint with the service workers, quickly checked my settings and figured out that thankfully Cookie Autodelete [0] can take care of those too. Down from ~200+ service workers to a few with my existing configuration.
I'm not a fan of service workers myself. Another team at a previous job added a service worker to a site that served many separate backend apps. It caused endless loading, caching, and crashing problems for my team, for no clear benefit.
If you want a cache, use HTTP caching, not custom JS code.
Not just RAM, I noticed on Android 11 that with the broken Twitter service worker installed, Firefox was gobbling up battery as well.
I ignored Twitter being broken for a bit since I didn't mind not using it. Then I realized my Firefox battery usage had essentially doubled and was draining my battery faster than normal. Once I cleared all Twitter site data including the service worker, back to normal.
So it wasn't just my browser or my internet connection. I used CTRL + F5 in FF and it always worked, but on a normal refresh it would either display an error or the page didn't load at all.
Thought that my IP had landed on some list where Twitter degraded the connection and I wasn't even logged in.
Yeah, I got loads of these - this tweet is not available to you etc.
Recently, it appears that I can't load twitter URLs with query parameters - the Guardian use some really weird ones of these, but when i delete them the URL loads. Super weird.
In Safari it will just drop you to an error page 70% of the time you navigate to it through a link, I assume because it doesn’t get its tasty tracking data. You have to reload the page to get anything to show up.
This has been my experience on mobile twitter for years, and in the last year on the desktop one as well. I thought it was a passive aggressive push for users to log in or install the official apps.
Me too. First follow from external website to previously unvisited tweet, there's an error. If I open it again it works, and keeps working from then on. Also happening 100% of the time on new tweets.
Kinda funny that twitter can't figure out such basics such as opening a page correctly when following a link. This has been happening for months.
Loading a tweet really isn't that hard; it's just Twitter's engineering team is incompetent. Even tiny little websites that custom-roll their own forums have less trouble loading their content than Twitter.
I don't use Twitter except when I'm linked to it and for occasional searches, and it's been a huge step backwards with the new UI that was forced on everyone a few months ago. It even has a loading screen because it's much slower, and somehow also manages to show less content than the previous design.
If I remember correctly there was a brief period a while ago when they did use a JS-only "web app", but then switched back to simple HTML with JS enhancements (and I seem to remember they announced that change with much pride).
Of course, there is always mobile.twitter.com that is still static-only and quite usable without JS, but IMHO Twitter is the perfect example of how the "modern web" is doing less with more.
So there's a persistent cache of javascript that can't be cleared except one at a time? My cache is full of fly-by-night websites. Surely this is a shocking privacy hole?
Web browsers need to be more transparent about what state they are storing, and more accommodating of attempts to clear it.
Also a performance/battery suck. Apple refuses to implement it in Safari, and it's one of the big reasons web developers deride Safari for being "the new IE"
That's far from obvious. Good cache control and offline usage are great for battery usage.
> Apple refuses to implement it in Safari
Wrong, Safari supports them. What Apple doesn't do well is enabling webapps that you pin to your homescreen ("PWAs") to work (they allow it, but from what I hear it's buggy and has weird restrictions), because they'd rather force you to make a native app.
As I understand the docs, they can be killed any time, e.g. when their site is left, then they aren't different from page scripts. Well, it doesn't look like they solve any problem either.
The most crucial thing they solve is giving websites access to their cached pages, and they can thus show cached content while also making a request for new data in the background, still provide cached functionality while the device isn't connected to the internet, ...
In the wider concept of "PWA" (Progressive Web Apps), they also are used to enable webapps to act a bit like local apps if the user opts in - a user can add them to the homescreen, and they then are launched just like normal apps (but still sandboxed in the browser). An example that does this quite well is the Web-IRC client "The Lounge" - you can open and use it in the browser, but also pin it as a standalone app, including support for push notifications etc if you allow that.
Well, except there's nothing stopping a site from handing out a unique service worker with a hard-coded i.d. to every visitor. It's persistent state with behaviour, like a cookie that can do things.
I agree with the transparency, I was not aware about service workers until this post. In the meantime, there are Firefox/Chrome extensions that manages the service workers. Those extension can block service workers from installing without the user consent which that was nice.
It doesn’t answer how Twitter can be so bad at this, though. Looks like a like of us have experienced this for a long time, I’m mystified why they wouldn’t fix the bug.
Yup, that fixed it for a few days at least last time. The issue about the whole page being inaccessible. The issue with the page loading but not the tweet is still ever present..
Wow, I guess this is common. This has happened to me for months using Chrome on Ubuntu with UBlock & Privacy Badger. I always assumed it was the extensions but I guess not.
Twitter is literally so buggy. Most of the time I have to refresh a page several times. Maybe it's because I don't have an account and refuse to use their apps?
I've been experiencing something similar, every time I click on a twitter link from outside of Twitter, Firefox errors with "There has been a protocol violation" error and a hard refresh fixes it.
I'm not sure whether it's because I've setup FF containers to open all twitter.com links in a separate container.
Service workers are hard to get right and if they've for example updated the service worker, it might be damn near impossible to unregister the old one and installing the newer one before the old worker is set to be cleared from the cache.
No freakin way... this whole month I thought twitter was broken... I figured it was some censorship related filter. Every visit (for me at least) requires a reload to work properly.
I’ve noticed something weird moving between hacker news and Twitter safari on my iPhone 7. If I first visit Twitter then in the same tab go to hacker news. The favicon is the Twitter one but the tab says hacker news. This doesn’t work with other sites like google that I tested. It’s quite odd. It’s like the favicon for Twitter overrides the hacker news one for the first page load
I have the same issue with Firefox on MacOS. 3rd party cookies disabled, privacy protection on, uBlock Origin, and PiHole. I assume it's a poorly handled call to a tracking or ad domain that fails, blowing it up.
I have to refresh the page once or twice when attempting to load a tweet in Safari or Chrome on iOS, otherwise I usually get a cryptic "this tweet is not available to you" message or something similar. I also just assumed it was a dark pattern to get me to use the app.
> It strips out the www. prefix to make a ”display version“ of the URL. I have no problem with this, as the prefix is entirely meaningless to humans. It does serve important technical functions, however.
I know most people don’t know the difference, and it would generally be a bad idea to have your www not redirect to the bare domain (or vice versa), but personally I prefer when we don’t hide these things. Just a bit of pedantic correctness, I guess.
> I can’t look at older versions of Twitter, as its pages don’t work well in the Internet Archive’s Wayback Machine.
All of this gets to me. A www subdomain is _not_ necessarily equivalent and interchangeable with the apex domain; treating them as equivalent is presumptuous in the extreme.
I think for most common domains it’s far to late to make a distinction; we already lost http:// and now www is pretty much gone everywhere. I wonder if we’ll get rid of .com anytime soon.
While we’re on the topic, I wonder why some websites chose to redirect to www and some keep the apex domain as their homepage. I use the latter because it’s cleaner and, well, my domain is primarily used for my website, but I wonder if there is a technical reason behind choosing one or the other.
Besides the security questions people have already talked about, there's a sizeable number of non-technical people for whom "www" signals "type this into a browser address bar", whereas the plain apex domain doesn't give that context because it's also in your email addresses, other subdomains etc.
I don't think many people get seriously confused by it, but accessibility is for everyone and it just makes it that much easier for people to know what to do.
One reason is that you "can't" CNAME your apex domain. So redirecting foo.com to www.foo.com might give you a little more deployment flexibility. The servers hosting the redirect don't need to be super beefy like your main site so just a A/AAAA might not lose you much.
Of course, were WWW browsers like a lot of other softwares are, there would be a SRV lookup of the domain, plus the protocol and transport, and the deployment flexibility would be in that mechanism. (-:
There should be no scare quotes around can’t. CNAMEs on the apex don’t exist because CNAMEs are not allowed to coexist with other record types, and you can’t have an apex without NS records or SOA records or whatever else it may be.
You can have a wacky DNS server that allows you to do weird things like ‘alias’ records, but those are sidestepping the DNS RFC: They’re answering with A records, but in a dynamic fashion.
If you use www.example.com and then add forum.example.com the probably external forum.example.com can't reach your cookies. If you put everything on example.com and then later add forum.example.com you have to think about security/privacy.
Whoa! I had no idea this had changed. It used to be that cookies for .example.com and example.com were different -- one was accessible by foo.example.com and the other wasn't. Seems RFC 6265 changed the behavior[0].
I assume the RFC explains the reasoning, but prima facie, this seems like a bad change to me.
Thinking one can take care of every aspect of security or privacy when implementing a public website, especially one that publishes UGC, is similar to believing in ability to deliver bug-free software: very likely presumptuous. However, a good way of achieving reasonable security is by reducing the scope of things you have to think about in the first place, preferably by offloading them to trusted implementations someone else (e.g., browser vendors) took care of where possible. Scoping cookies to subdomains, for example, comes in very handy.
I think the point the GP is trying to make is that if one has thought about security and privacy then one is more likely to use www.example.com instead of example.com for one's website for this very reason.
The suggestion is that using basename.tld instead of www.basenamne.tld adds to the security matters you need to think about, if not now then later if/when you add features on a subdomain that you (and/or your users) want to keep separate in terms of cookie sharing.
In that sense using www.basename.tld is thinking about (or at least autonomicly mitigating, by way of scope limiting) those potential security/privacy issues.
> [..] and now www is pretty much gone everywhere.
I don't think this is true. There was a time about 15 years ago when everyone fancy went without www but the pendulum has swung back completely in the meantime. Most sites nowadays redirect to the www version. Twitter is the only page I am aware of which was no-www from the very beginning and stuck to it consistently until today.
Ah, that was poor wording on my part. What I meant was "users type in the apex domain and either stay there or get redirected to www", instead of "users still type www".
Netscape Navigator used to automatically prefix "www" and suffix "com", and this carried through to Mozilla as "domain guessing". What you are describing is the state of the art regressing to that. (-:
it stems from the old ages when there was more than http to the internet and `www` it was simply the web server machine, and `ftp` would go to the ftp server ...
now everything is HTTPS (or tunneled over HTTPS) so it makes little sense to specify mundane things such as the machine you want to connect to and the protocol to use... there is only one machine (the apex domain) and only one protocol
Even if everything is HTTPS as you claim (spoiler, it is not, there's still plenty of other protocols in use today) subdomains are still relevant information used in SNI to serve virtualhosts.
There is no reason that imposes www.example.net to serve the same content as example.net. Whether or not it does is left to decide by the website operator.
You're right that it isn't everything, but the non-https stuff out there is super buggy. I had a Canon 5D Mark 4 and it had the option to (s)ftp up raws so I wouldn't have to take out the card. So I went to the trouble of setting up my own secure ftp server. Guess what. It didn't work. I tried everything from raw IP, to a naked domain, to a ftp domain, whatever it was there was absolutely no way of getting Canon to make the connection securely.
That's why everything is moving to HTTPS endpoints. It's simple. You just give a url and it works.
Specifically SFTP is a SSH subsystem, when your client connects to a SSH server it gets to specify what subsystem it intends to talk to, and so a remote SSH server can choose to offer the "traditional" shell service, an SFTP server, or any arbitrary thing. In this way SSH subsystems are rather like ALPN in TLS (except years earlier) which is how HTTP/2 works among other things.
It’s not necessarily equivalent and interchangeable, but for public websites, one should redirect to the other, and you’re a troublemaker if you do otherwise, because it’s a well-established and understood convention, and different people will start by guessing one or the other.
Unless you have a very good reason you just shouldn't change what a user has entered; just trust the user's input where there's no security or integrity reason to do otherwise. This example of Twitter messing with user input and breaking things is a good example of why you don't do that.
Remember that the problem here is purely a performance optimisation having become a pessimisation; it’s not actually breaking anything. I tend to agree with you, but only as far as functionality is concerned: that you shouldn’t act otherwise than the user specified. For display, I’m quite happy with dropping https://, and not too distressed about dropping www.. But for my comment, I wasn’t talking about changing what the user entered, but rather being willing to accept either of the two most common forms that users may enter.
Entering www.example.com and getting example.com is a breaking change in the sense that you wouldn't expect the user's machine to make a request to a different domain.
edit: prepending software.* to a domain prevents the subdomain from being removed, https://twitter.com/lukerehmann/status/1321310972468973568
I've also tried messaging this same format of unique link and the preconnect slides right into the DMs
Right, I wonder if using IPv6 link would be enough. It would have a lot of space for differentiating the user, and they may not do much pre-processing on it...
Due to browser design, one might be tempted to hover links to see their content (small popup in the bottom left corner), before clicking. Kind of a habit for me.
Yes, a feature that originally was helping security (allowing you to "look before you step"), has been hijacked to actually harm security and/or privacy. Makes one laugh, bitterly.
Is there much of a downside to just disabling all hover events in the browser? I've also considered disabling the clipboard API after hearing about clipboard hijacking attacks recently
> Twitter redirects links through its t.co link-shortening service. It was once a useful addition to its service as it helped people stay underneath the strict character limits. The link shortener reduced all links to 23 characters. Twitter gains some more data-insights about its users in the form of click-stream data and insight into popular links.
The t.co link also helps them block URLs that they deem problematic on their platform - in the event of spam, attack, or abuse, the redirect can instead be a black hole.
It wasn't arbitrary. Twitter started out with an SMS interface, and kept it for several years. SMS are 160 characters max [1], and some characters are needed for the protocol, so the payload was limited to 140 chars.
[1] Note: Modern SMS apps support a protocol that sends larger messages as a stream of multiple interlinked SMS. This is transparent to the user, but at the time of Twitter's SMS interface this was not common.
Well I learned something that's not in the article.
I thought browsers requested a domain lookup (gethostbyname()) and basically got back a "zone file" which would have the cnames in. So, I was confused, when people complained about Twitter forcing a domain lookup "on the wrong domain" as I was assuming this would at least cache the domain lookup: It's the right domain, of course, but the lookup for an address on a subdomain includes the subdomain and then gets the cname directing wherever.
It always confused me that dig/nslookup didn't seem to provide all the info. They can, using nslookup 'ls example.com' or 'dig example.com -t AXFR' but the server in general refuses to serve the zone file (seemingly for security by obscurity reasons).
So, for example, if the browser looks up example.com it doesn't get that there is a cname from www->example.com . It only gets that relationship from looking up "www.example.com".
So, TIL, and now results provided by dig/nslookup on the command line make more sense!
Almost everyone uses a “recursive DNS resolver“ provided by their ISP, or one of the big ones from Google/Cloudflare/Cisco. The recursive resolver does all the hard work of resolving the root, top-leevl-domain, the apex domain, the subdomain, any CNAMEs, and finally find the right IP addresses to respond to the DNS client. Recursive resolvers benefit a lot from caching responses at each stage of the chain, the same way your browser/OS/router DNS client benefit from caching the final responses from the recursive resolvers. If you run a full DNS resolver, you have do to all of these steps locally.
How does the preconnect work with the t.co redirect in between? t.co will return a 301 right? Then we see the real domain, then the browser can preconnect to the server, not earlier, or can it?
Twitter run t.co so they know where the redirect goes without actually asking t.co. So they can preconnect to the target domain as well. And that's where they goofed up (I mean, apart from preconnecting to all these domains in the first place).
that is just matter of opinion. The quality of developer has gone down in the years. That's all I'm saying. To provoke a knee-jerk reaction is expected.
For the last weeks until a few days ago I've been unable to access twitter unless doing a hard refresh on every click. Because they installed a faulty service worker of some kind that would break most of the requests. (Both on desktop and android)
And on Android, every time I follow a link to Twitter in an app that opens in a web view, it gives me a faulty page that I have to refresh a few times before I'm able to view it. It loads the page fine, but some rest call or whatever to fetch the tweet crashes.
Edit: Not heard many others complain about this before, so mainly thought it was something about my setup. But the huge amount of upvotes this got suggests some Twitter engineers better look into this..