There are two situations that I hate:
1. I have a different IP than usual, so all the sites change to that country - including language, news and search results(Google), etc, even though my browser sets a list of Accept-Language options. It's still me, I know you track me, so why?
2. Changing a multi-language website to a different country, also changes the language to that country, even though by browser sends an Accept-Language header and they have the translations for the website from their other sites/countries. I don't suddenly start speaking a different language if I look at a your site from a different country.
Not only languages change, but many other things like date formatting, mathematical formatting (using commas rather than dots breaks few things around) and currencies too! And it's a pain to fix as settings often collide and with many applications abusing microfrontends you can even end up in conflicts on the very same pages.
Google's the worst offender on all of its services and not just search, youtube, etc (even for work, their forced localization breaks Google Sheets in multiple ways), even my goddamn portoflio prices get converted in the local currency or using google flights shows me prices in the local currency forcing me to open more tabs to convert non stop.
And changing it to my own, on so many different services is such a pain.
But I know why this happens as a developer who advocated multiple times with my managers to not forcefully try to compute the language/region/market of users without giving them a simple button flag to change locale: product people don't listen and would rather have thousands of "ifs" that eventually will fail rather than listen to reason.
Hell, as a developer I have all my software and preferred language to english, even though I'm italian. And sometimes I use VPNs!
Even Amazon at times gives me some text in a language, and some other in others (completely unrelated like German) on the very same page.
> Not only languages change, but many other things like date formatting, mathematical formatting (using commas rather than dots breaks few things around) and currencies too!
The most egregious instance of this is Google search with the news articles at the top. When I’m in Thailand, even if Google correctly figures out that I want English, it displays the news articles with the year in the Thai calendar! How many English speakers know that the current year in the Thai calendar is B.E. 2567‽ This is the correct configuration for approximately zero people in the world!
This is just a guess, but I suspect it wouldn't be in The Google's best interest for other engineering departments to be too involved with the ones that deal with data mining, otherwise they'll end up knowing too much. Even if someone working on a particular web product at The Google thought it would be a good idea to use the existing tracking to improve the user experience, I imagine there would be so much organizational friction in place that they're likely to give up.
Having worked in the ecomm industry for quite a while, one thing that always drove me nuts was stores that did not understand that your current location should not drive your store choice and certainly shouldn't drive your language choice. Sure, if you need to guess, then you can use assumed geographic location as a starting point if you like. If you don't know what language someone wants, by all means, use the Accept-Language header to make a best effort. After that, let your visitor select a location and language that they want.
I went through a lot of effort on one project that had something like 90+ country stores to make it work as expected. We used IP-based location to select a store if you hadn't already selected one. We used Accept-Language header if you hadn't selected a language. If we couldn't determine a location via IP address, we had a world map landing page you'd get redirected to so you could select a store where you'd be directed back to the product page for the correct store. Language display was best effort. All of the 'chrome' of the site could be displayed in most every language. Products had limited subsets of language. I don't recall how we resolved that disparity.
> I don't suddenly start speaking a different language if I look at a your site from a different country.
As someone that traveled a lot, I always hated this. Sorry, maybe I'm not fluent enough in German to read your site, but I really do want to order from your German store. Or maybe I really want to order from the US store even though I'm in Germany.
One other aspect of all of this that was *extra* fun was payments. Anti-fraud systems frequently get super suspicious if your location and the store location don't match.
1.b. When somehow they identify you’ve change IPs the first time, and now that you are home, there’s still some sticky session that doesn’t get cleared without purging all cookies for the domain.
> Interestingly, Google does respect the Accept-Language header. While testing in Yelp domain, Google Sign-In widget popup appeared in Japanese.
I’m surprised by this. In my experience, Google are the one big organisation that I can count on to get this completely and utterly wrong. I’ve lost count of the number of times I’ve been stuck with a Google interface, popup, or some other thing that is in a language I don’t speak, just because I was visiting another country. I’ve specifically encountered it repeatedly with the very Google Sign-In widget they say works for them.
Sure, if it happens to be a Google website and you know the magic incantation of &hl=en then you can sometimes work around it, but that doesn’t help when it’s a Google login from another website or something of that nature.
Indeed, google never behaves for me, always hands me language localized to my ip, even though my Accept-Language contains "en-US,en;q=0.9" (the local language is my native one, but i find using services or software that are originally in English translated to my (non-PIE) language awful, even anger-inducing).
For search i have had &hl=en added to the search parameters since forever.
What works surprisingly often is setting language to en-UK instead. Apparently many sites consider en-US to be some default that they can safely ignore, but en-UK to be a deliberate choice. Nowadays I use that not just for the browser, but for my phone and computer too.
I can say that it doesn't work on Google Docs, which uses the IP address's language^, even though my language is set to en-UK. Unless you're logged in, but I often have to use incognito because of another misfeature which is that Google will dox you to other editors if you access a public Google doc while logged in. Googlers reading this comment: please fix this!
^ I wonder how this works in places like Switzerland and Belgium that have more than one language. The notion that IP addresses can be mapped to languages needs to die.
One drawback I remember was that sometimes experimental features in some app are available strictly for the US (i.e. "vanilla") version, presumably simply because lack of localisation blocks it for "other" versions.
(Sadly cannot recall concrete program, but I think it was some web browser.)
I had my language set to en-NZ when I lived in Indonesia; definitely had Google serve pages in Indonesian to me (and double frustratingly, with US-style dates).
Even Accept-Language headers aren't really a particularly good solution. I might prefer some sites in Danish, but others I'd like to have en English. Unless browsers allow you to set language on a per site basis, we still aren't really capturing the nuances of language preference.
Accept-Language could be a pretty good indicator of a persons language skills, but it's just to far into the browsers settings to be something that a normal user would adjust. I can see why checking the domain is quicker and perhaps more inline with a persons intent, even if it is ignoring travelling.
We where trying to book a table at a restaurant, it's part of a fairly large chain. We where in Sweden, but the chain also operates in Denmark, so we know that they have a Danish version of their website and ordering app, you just can't use the Danish language when you are in Sweden. You can get English for some reason, but not the Danish version. That to me seem like a missed opportunity.
> I might prefer some sites in Danish, but others I'd like to have en English. Unless browsers allow you to set language on a per site basis, we still aren't really capturing the nuances of language preference.
This problem is solved already by competent sites. On your first visit to a site, it uses Accept-Language to guess what language you want to see. If you use a language switcher menu to pick a different language, a cookie is used to save your choice. On your next visit to the site, the cookie is sent and that takes precedence over Accept-Language to send you the page in your preferred language. Easy.
If/when you decide to create a user account on the site (if the site even supports that) then your language preference can be stored with your user data so you would see it automatically after you log in even if you're on a different browser/device.
As with many other things, there are already best practices on how to do all this stuff. It's just a matter of getting web devs and their managers to be aware of them and/or care enough about providing a good experience for their users (unfortunately this has turned out to be a huge ask) to find and implement them.
> […] picking your preferred language is an unskippable step of setting up a new phone.
This does not address, or more precisely it even prevents the notion of:
> I might prefer some sites in Danish, but others I'd like to have en English.
, i.e. what you set in your OS or UA (browser) is always one preferred and optionally some fallback languages. I second that granular per-site control would be preferable for me either: some texts I prefer in my mother tongue (news, non-tech encyclopedic articles), the rest, namely technical documentation I prefer in English. (Also, I have very bad experiences with localised GUIs, where 1. familiarity with English "original" vastly helps when searching and 2. translations tend to range from bad to hilariously comical.)
But I must admit that most websites serving of content in multiple languages, support client-side pref pretty well, so the situation is not so terrible, in my experience.
Yet, the problematic of content negotiation and user <-> agent <-> author (provider) information passing is interesting and I'd say even under-engineered, and currently terribly suffer from security (anti-tracking) constraints.
I do not ever use any OS or other software in my native language, however I do want most non-technical websites to display in it when possible. How do I communicate that?
My hunch is that Accept-Language became the best way to determine a user's language about the time it became common to bring a laptop while traveling. People use arguments today against Accept-Language that have been wrong for decades.
No they don't? I have never got that at least. It asks for my UX language, not for my preferred languages for websites. My preferred UX language is always English since most translations to Swedish are horrible while Swedish made websites are best read in Swedish and not a potentially bad English translation.
accept-language is a list. Do I want English or Spanish first? What if your content is better in the second? I'm much better at English than Spanish, but I'm better at Spanish than the AI translation to English. Plus I may be watching that video in German to learn German even though I'm so bad at German I wouldn't put it in...
Accept-language is accurate, but just isn't easy change it which is required to handle all the possible situations and get the right answer. Still everything else is worse.
Accept-Language headers aren't used in a particularly good way by browsers and servers. If used better, they would be a particularly good solution.
The idea you mentioned of browsers allowing the header to be set on a per-site basis would be particularly good.
Another very good solution would be for servers to assign weight to the languages they support, and to combine their weights with the Accept-Language weights to choose the best language in common.
But even as they're implemented and used now, Accept-Language headers are already a way better approach than geo-ip.
> Interestingly, Google does respect the Accept-Language header.
Google knows I use/want English. It knows this because I'm logged-in and have set my preferences, but also because of that Accept-Language header my browser sends in every single request. Yet for years, perhaps because I am 20km from Germany, it has assumed I would always prefer German when doing social sign-in.
Google may occasionally glance at Accept-Language, but it does not respect it. It even demotes my own logged-in preferences below a guess based on my location.
When there are no other indicators, fine, guess my language from my location, but otherwise location != language. But I know for fact that my browser is sending out Accept-Language headers in a convenient format that will tell you all you need to know.
I always design i18n with a preferred language on a customer profile and I use Accept-Language to pre-select it on registration. Browser settings must be respected, of course, but there are also non-browser communications that can use this preference. It is very small amount of work if done at the start of the project rather than as feature upgrade.
Special mention to Amazon who insist that you want to browse their entire dropshipper directory, on all devices, in the language of whatever you last opened in a non-dubbed version on your TV.
(if the original version is even available, they even find ways to offer some BBC shows, of all things, dubbed-only)
I get all of their websites in french (correctly) but some of their emails used to always come in german. I don't speak german, I've never set the language to german, I don't think I had even visited amazon.de at that point.
I've worked on a couple multi-language/location projects... Usually using the Accept-Language header as the default response to closest match or the default language (en-US fallback). In which case, I would allow for a cookie and/or url override that would set a cookie and redirect.
https://en-gb.foo.com >> set cookie and redirect to bare domain
...?lang=en-GB >> set cookie and redirect without the lang portion
<cookie>
<Accept-Language Header>
I'm also a proponent of having a mini-flag for the current language as a dropdown/flyout menu in the upper-right of the header where each local/language has the appropriate flag/representation in addition to the language name IN that language.
You can use just language (en) instead of language+locale (en-US) as desired.
edit: also, make sure you set your cache directive to include the cookie/header in question. You can also work the other way and always use the locale for subdomain.
There are many many cases when you can’t determine user’s language by location only (immigrants, VPNs, business trips, multinational corporations, countries where people speak multiple languages, etc). If you do any effort in localization, you should recognize Accept-Language. If unsure, you should ask the user for their preferred language directly. Inferring the language by location is not acceptable.
Here in Switzerland, I am so tired of websites offering German localization by default. We don’t speak German in Geneva.
1. Check if there is a language cookie set previously
2. If no cookie, check if the Accept-Language is present and contains one of the supported locales
3. If no Accept-Language matches, use the fallback application language
Not that hard to implement, except perhaps parsing the Accept-Language preferences with their quality value can be cumbersome.
> Not that hard to implement, except perhaps parsing the Accept-Language preferences with their quality value can be cumbersome.
Many web frameworks provide a tool for parsing these for you. If yours doesn't (or you're not using one) you may be able to find a package that does. Some will also do the matching for you, so you can pass it a list of languages you can support alongside the Accept-Language string and it will pick the best option that's found in both.
`Accept-Language` contain both language and locale. Depending on your app you might have supported languages or language/locale combinations. When there's no exact match after step 2, you'll want to do a partial-match on only the language.
Same as what we do. Only one additional thing: when people log in to their accounts, it switches to the preferred language for that account (and any changes in language selection whilst logged in are also saved into the account preferences).
You’re talking about the Accept-Language header. They are asking about the cookie.
To answer the question, yes, normally you would have some kind of manual switcher that sets the cookie. Please don’t use flags for this, because flags indicate countries, not languages. Don’t make a Russian-speaking Ukrainian seek out the Russian flag, and don’t make an English-speaking Irish person seek out the Union Flag.
> To answer the question, yes, normally you would have some kind of manual switcher that sets the cookie. Please don’t use flags for this, because flags indicate countries, not languages. Don’t make a Russian-speaking Ukrainian seek out the Russian flag, and don’t make an English-speaking Irish person seek out the Union Flag.
Yes, why make things easy. :-)
Controversial opinion: Flags are great as a visual aid, shorthand, look neat, and give a better UX for 90+% of users, even if it hurts some linguists' and nationalists' feelings.
Its not just a matter of hurt feelings, it has multiple problems. If it was just hurt feelings I might agree with you, but it is ambiguous, inconsistent and fails to cover many cases. That is bad UI.
What flags to you suggest people use for Cantonese, Tamil, Arabic and Xhosa?
Why use the Union flag for English? If the language is en-US surely an American flag would be more accurate? You can then use the British flag for en-GB. However, if you offer Welsh you would use the Welsh flag (and presumably the Scottish one for Scottish Gaelic, although I cannot recall ever seeing that) so it would surely be more consistent to use the English flag for British? Even for or generic English?
I dunno, which flag would serve as a comprehensible shorthand for Arabic? How many Arabic readers would recognise that flag? How many Spanish speakers would recognise the Spanish flag?
If you use flags I would still recommend putting the language name in the language next to it, for example the French flag next to 'Français'.
But then where's the flag for Liberia and Belize and Singapore? What if I'm in Indonesia and you don't offer an Indonesian option but I know enough English to get by - what country's flag should I click?
Just don't use flags, man. It's ridiculous that this is even still a discussion.
I gave an example of Russian-speaking Ukrainians. Why are you characterising this as “hurting some linguists' and nationalists' feelings”? It’s not nationalism or pedantry to be offended at being forced to identify with the flag of the country that is currently murdering your friends and family.
In my experience Accept-Language header is pretty unreliable. It tends to just be one of a few values. Between people never updating it, proxies stripping it, people concerned it will make them easier to fingerprint, etc.
Also, form an implementation standpoint, translation is a lot of work to get right and some companies may not do it for all languages so they may support the header but just not support the language choice you have. Yelp being a weird example where they apparently have the Japanese translation on another site.
Combine all that together and it's not usually worth the effort to do it unless you have specific legal requirements or high demand from users.
(Sorry for the delay, I haven't checked HN is a while and just noticed this)
So some proxies (forward and reverse) will strip this header for different reasons.
The reverse proxies sometimes strip it to improve caching and/or because they were configured to have a allow-list of headers and this (being a less common one) was excluded.
The forward proxies will sometimes strip it for privacy reasons (it can be used in fingerprinting) or for the same reason (they have a list of allowed headers and this isn't one).
This drives me nuts when I visit foreign countries with a language I don't speak and then the internet stops working. My browser literally tells the server I speak but 90% of the content I get back is suddenly in the wrong language.
Unfortunately not. I’d really like to have the semantics of “the website’s native language if I understand it (if it’s one of A/B/C), otherwise English”.
But nowadays it’s a clusterfuck anyway. You often get WebView-based apps were half of the UI uses the OS language, half the browser preference, and the third half the service’s native/local language.
Yes, but if the web site supports both of them, then it will pick whichever one is first in the header. For example, "en;q=1.0,de;q=1.0" technically states you want equal weight to be put on English and German, but the site has to pick one to show you, so it will most likely fall back to the behavior that's used when the quality score isn't provided (such as "en,de"), which is to pick the first one it can service from left to right.
There’s ambiguity in the spec in selection itself. f(a-l-input, supported-locales) yields different outputs contingent on what selector youre using, and its not incorrect
This is a problem for me as a language learner on mobile. I set a keyboard with (for example) Russian, and suddenly the world looks Russian, even though I am far from comfortable in it yet. I wish there were a way for the browser to ignore some system languages. This is worse for apps because I also get served with localised versions, where I need to uninstall, remove system language and re-install.
Yeah, but Accept-Language is not always good idea.
E.g. I prefer to view some websites in English and some in my native.
Generally I prefer all the websites that are not from my country domain to be in English.
Setting up Accept-Language per website is annoying.
The most annoying part is when my local website sees "en" there, and decides to show me English version.
It is easier to just add e.g. `.jp` to domain, than to find out where browser sets up Accept-Language.
