Hi all, Founder and CEO of DuckDuckGo here. I’m literally just waking up and reading the comments here.
I’m new to this issue and happy to commit us to move to doing this locally in the browser and will have us move on that ASAP.
That said, I want to be clear that we did not and have not collected any personal information here. As other staff have referenced, our services are encrypted and throw away PII like IP addresses by design. However, I take the point that it is nevertheless safer to do it locally and so we will do that.
If anything, it's much better than 'no big deal'. It's "We made this design decision, thought you would like it -- we've learnt, changed, and will avoid it later".
Can you imagine Google doing something similar? Heck, they're just about to throw the Android rooting community under a hardware-attestation DRM-filled bus.
Could have been quicker but it's still far better than some smarmy non-apology and continuing as usual, which is sadly what we've come to accept. I don't think they've done too poorly.
They started a fire through mild negligence, denied the fire existed, and only put out the fire when the entire neighborhood started yelling.
It was a forgivable-but-negligent decision to write/approve that code in the first place. It was a sign of a bad process that a reported security vulnerability was not escalated to people security-conscious enough to immediately identify this as a major problem.
I don't agree with the outrage. Anyone who has followed DDG knows they're legit. They just need to do a bit better. They probably will.
Their main feature is privacy. They should be at least as sensitive to privacy vulnerabilities as their most aware users.
DDG should announce that they now pay out privacy-related vulnerabilities like this and send the reporter $5k. It would be good honest PR and well worth the expense.
Correct me if I’m wrong, but by default DDG uses redirects to prevent leaking your search queries through the referrer, so they already can technically see every URL you visit. Except their whole product and system is designed around protecting privacy and not storing that data. If the favicon endpoint respects the same rules (which it obviously does), it is no different.
Wow. That was really not very complex at all... Did they leave off some capabilities that the server side version performed, it was the complexity overstated?
Creating a throwaway account to disparage a particular point of view is also questionable, no?
At what point did we stop taking people's word and commitment as valid? Sure, I too want to see proof that they are doing the right thing here (because I don't understand the design decisions that led to the creation of that service in the first place), but because these changes are not immediate, this statement does at least answer some of the questions I personally had (Like "will this be fixed"? "when"?).
> At what point did we stop taking people's word and commitment as valid?
At the point that they where caught violating privacy while claiming to respect it, and standing to make a profit off of violating it.
If I trusted people as blindly as people seem to trust DuckDuckGo, I would have trusted Google to not be evil and never have switched to DDG in the first place. This breach of trust destroys the whole point of using it, so I switched my default search to Searx between reading this submission and writing this (had been procrastinating the switch for a while and this was exactly the motivation I needed).
I will of course be keeping an eye on whether or not they follow through, but the admission of this being a problem and committing to fix it is -far- more than we see from most companies these days. Is it everything I ever could have wanted out of the company? no. However it is a positive direction I would like to see more of so displaying my appreciation is hopefully validation for the company to continue in that direction.
It's a little confusing to see 2 accounts, with very different usernames (epi0Bauqu and yegg), that appear to be posting as Gabriel Weinberg . Are these both legitimate? And if they are, what's the reason for two of them?
Yeah that's my bad. epi0Bauqu is my original account, but since no one knew who I was under that account I, some years later, made the yegg account and I try to post from there. The issue here is I logged in to post the comment, and then switched to my laptop where I was already logged in this other account.
It was only a minor issue that got resolved quickly thanks to your question, but DDG's response to this whole issue of losing users' trust seems to be accurately characterised by the phrase "a succession of embarrassing own goals". :-/
We never used this data, other than showing favicons.
In fact, we didn't (and don't in general) collect any user-level data in the first place, per our strict privacy policy: https://duckduckgo.com/privacy
In this case, the way it works is you hit our favicon service and it returns the favicon, not using any PII in the process, and our web servers are configured not to log any PII. In other words, our system is technically designed and we are legally bound by our privacy policy to not use this data for anything other than showing favicons.
You might want to clarify you never retained any PII, unless you can 100% confirm that that favicon request didn't include any embedded user- or installation-id in cookies, headers or the like?
However, this problem demonstrates gross incompotence for a browser team supposedly concerned with privacy. Will you please do a post-mortem on how this code made it through your code review process in the first place, as well as how it managed to stay in place for a full year after it was pointed out that it represented a privacy problem?
"Sends every URL you visit to the vendor's servers" is the single worst thing DuckDuckGo could have done for privacy in this web browser, and that needs to be accounted for. There was a major failure in the code review process, ticket review process, and in how you treat your community. A standard marketroid "by design" response with washy promises that "we'll take very good care of this highly sensitive personal data, just trust us" is not something I want to see in the future from this team.
I’ve worked with many companies who have demonstrated “gross incompetence” when it comes to privacy and information security. This is absolutely not an example of gross incompetence.
I agree that for a company built around privacy even the appearance of impropriety needs to be avoided. DDG holds themselves to a higher standard and their users hold them to a higher standard.
This was a design flaw and a process flaw. DDG prioritized speed and efficiency over privacy (or in this case, perceived privacy) and I suspect there isn’t a soul on HN who hasn’t made that trade off at some point. They assessed the cost/benefit and risk/reward and it turned out their assessment was wrong. Now they’re fixing it. It happens. But to call this gross incompetence is really blowing it completely out of proportion.
I'm not blowing it out of proportion. This one specific "design flaw", if we're being generous, has been raised many times with many different browser vendors and add-on vendors as a very bad thing that you cannot do. There is plentiful wisdom on this issue.
The first rule of privacy is never handle the private data in the first place. An accidental leak is one thing, but deliberately designing a feature whose side effect is exfiltrating heaps of private data, then doubling down on it for a year after it's pointed out to you, then doubling down again when it's raised on HN - this is gross incompetence.
You can’t think of anything they could have done that would be worse than sending URLs to their lookup server? It’s the single worst thing?
My browser syncs URL history between my devices, and that’s a feature that I value about it. Your comments on this topic seem to suggest that all users are making the same decisions about what is acceptable usage of their data, and that’s pretty obviously not true.
Maybe you’re taking this a bit too far? They explicitly state they will not store your data anywhere, and the main safeguard you have for that is your trust in them, not this one specific line of code somebody happened to notice which can’t even break that promise on its own.
Privacy is never built on trust. It's built on mathematical and logical facts. The only effective way to keep data private is to never handle it in the first place.
Looks like they developed the favicon service for their search engine so they could show favicons in search results.
It actually increases privacy there, DDG already know what domains your search returnedy and the alternative would be fetching favicons directly from websites, leaking information to them before even clicking.
Then they re-used the favicon service for their browser without rechecking the privacy issue and realising that browsers have different privacy needs.
Sure humans make errors, but this service is for a privacy-concered-company like you or me walking over a red traffic light without looking to the left or right and beeing crashed by the biggest Truck one can imagine. Anyone who belives in an error is beyond help.
The issue was highlighted from open-source code from a company who proactively works to reduce privacy.
It seems rather reasonable to assume that in the goal of maximising privacy in search engines, it would be wise to add user-friendly and convenient features so that the alternative is appealing and does not sacrifice too much convenience.
> That said, I want to be clear that we did not and have not collected any personal information here.
So, you do collect information just not the kind you would classify as 'personal information'. I wonder if my personal domain with my full name qualifies?
There is no way this feature would be created in a company built on privacy considerations.
Sure it could. A junior programmer aware of the values but not the repercussions could easily overlook this without any malice or disregard for privacy.
> So, you do collect information just not the kind you would classify as 'personal information'. I wonder if my personal domain with my full name qualifies?
Considering they have operations in the EU, I would imagine that would fall under what the GDPR considers personal information (https://gdpr.eu/eu-gdpr-personal-data/), or risk being in breach of it.
To be more clear, your staff, and you, have said PII ‘like IP addresses’, and have said ‘thrown away’ some places and ‘not collected’ others.
Contrary to this framing, it’s not possible to not incidentally become aware of every single browser users’ usage timing and user IP addresses if the browsers are phoning home this way — a colloquial understanding of ‘collect’, not the James Clapper NSA dodge definition of ‘collect’. Most normals think of collect as become known not as permanently store. You knowing it means others can know it if you break trust or are required to comply with authorities.
And regardless of end-to-end encryption, that this user is phoning home to your fave icon endpoint, when, and from what IP, is revealed to every ISP in the chain. You’re leaking browser usage telemetry to every single party to that traffic — the source IP address PII you mention is in unencrypted metadata.
The fact this browser connects to that endpoint reveals demographics (choice of privacy browser) and behaviors (when and how much web surfing) to e.g. ISP or nation state firewall operators who are certainly not bound by your ‘just trust us’ privacy policy.
Privacy policies are a patch for insufficient privacy engineering.
To be a strong privacy browser you could consider what it would take to be “NSL proof” such that if handed a national security letter with gag order, you cannot comply. That is not the case with this faveicon telemetry endpoint.
Just to be fair, as a matter of fact, you surfing that site is revealing you, surfing that site to your ISP and state actors, in the first place. A change, where to get the icon from (origin vs ddg), will not change this fact.
It is all about ddg not getting to know, which sites you are surfing, when not searching for it on ddg. Which should, indeed, be a no-brainer.
"You surfing that site" not adding a single surveillance point for every user of a particular browser, it's not compromising your other privacy measures (e.g, VPN that prevents your ISP from knowing) by sending your site surfing data out to that surveillance point, etc.
Getting the icon from each site means surveillance would have to be at origin or every site, while telemetry going to DDG gives a single surveillance point.
I've already posted this somewhere else, but I'll copy it here again as well:
It's not immediately obvious whether it is more privacy preserving if the client automatically makes a request to each site in the search results while scrolling through the results, especially since you're already trusting DDG when performing the search.
Maybe this should be an opt-in rather than an opt-out feature?
All in all its really not as big of an issue as people here make it out to be.
This was not about search results, where their favicon webservice is in fact privacy increasing, but about the privacy browser and the favicons it displays, where it is privacy decreasing as it involved sending information about visited sites to a central authority while you are not on the DDG search engine. For example the TabRenderer will fetch the favicons from DDG instead of from the site you are actually visiting: https://github.com/duckduckgo/Android/blob/db728523240e37727...
Thanks for pointing out that the service was already in use on their search results pages. To me, this goes a long way toward explaining how this could have happened:
Scenario #1 - "We need to show favicons in our browser tabs. Lets develop an API that requires every domain be sent to us!"
Scenario #2 - "We need to show favicons in our browser tabs. Hey look, we've already got a service that provides this. We know it collects no PII and our users trust it already."
Obviously the second scenario is flawed thinking, because (of course) it's better to not send that info at all. However, I can easily see how their developer(s) may have arrived at the conclusion that this is still compliant with their privacy ethos.
The fact that the favicon service already existed (and was trusted by users) before this was implemented, makes it much easier to understand how this could have been a legitimate mistake and thus, they deserve the benefit of the doubt.
Ah I guess I should have read TFA l, because search results have a similar feature (that is opt-out). For a browser I agree it makes more sense to get the icon directly from the site being visited without any privacy risks.
Yes, after consulting with staff, I understand we thought it was more privacy protecting because we know our services are already encrypted and throw away PII, and so to get the favicon you could either (a) make another request to our known anonymous service or (b) make a request (or possibly multiple) to a non-anonymous service. On the other hand it is another request to a distinct domain that traverses another path on the Internet, albeit an encrypted one.
Exactly. Everyone should decide for themselves how big of an issue it is for them and not one person who just shrugs it off. That person made their statement sound like a universal fact.
I think the most charitable (and probably the default) reading of that clause has an implied "I believe" prepended. It certainly does not have an implied "It is a universal fact that" prepended.
By way of analogy, when you earlier said "This is ludicrous. Can we switch to the metric system already?" no one thinks that you're speaking for everyone, but rather are advancing your own belief in the "This is ludicrous." sentence.
DuckDuckGo staff here. As mentioned in the linked page, the purpose of the request is to retrieve a website's favicon so that it can be displayed in certain places within the app or on the results page. We use an internal favicon service because it can be complicated to locate a favicon for a website. They can be stored in a variety of locations and in a variety of formats. The service understands these edge cases and simplifies retrieval within our apps and our search engine.
Like our search results, the favicon service adheres to our strict privacy policy[1] in that the requests are anonymous and we do not collect or share any personal information.
I wonder how many lines of code from big open source applications are generic enough to be reused in other projects.
Firefox and Google Chrome probably have the equivalent of many small high quality libraries embedded in them, implementing 'business' logic or protocols, that could be reused in more places.
I guess a large scale study on github could be done, with a graph analysis to show potential "cut off" points in codebase.
One issue is that if a bunch of code of a similar theme can be broken off, many times it will be (depending on the culture of the software team). Is looking at the source code of a big project with a hundred dependencies that were all libraries spun off to support the project different to looking at a project where people tried to keep everything monolithic (where you’d expect better re-use potential per line-of-code?)
Yeah... the gesture is nice, but good luck extracting any code from a massive project. Might as well say “Here’s some free oil; all you have to do is dig for it.” Unlike oil, this might not be worth the excavation.
It’s a bit telling that they linked to the GitHub repositories rather than specific lines of code they were talking about.
Android components is a foundational technology for our browser products on Android but also for many other applications. We’ve designed things in such a way that you can pull in just those things that you need. If that is not the case, file an issue and we will take a look at how to improve things.
There is also a content script I think and storage for these icons and their meta data. It is probably less generic on iOS than on Android but should be fairly simply to take some big chunks for reuse.
Yes, apart from Chrome's implementation there is probably no better tested and more mature implementation. Still it seems to be a non-trivial problem because Firefox sometimes shows me a favicon from another site I used.
If you don't visit the site explicitly, just have the site somewhere in a list/bookmark/whatever, that site now has your IP address and basic header info when it needs to go retrieve it. By going through DDG, the site has a bot hit.
To me it looks to be trying to uphold your anonimity until you commit (click) through to the site/link. But certainly other ways they can approach this if it really bothers people.. I'd prefer DDG doing the lookup.. or having no fav icons.. over my computer going and downloading all my bookmark or other source icons
I think you're being downvoted because you chose to piggyback your comment on a seemingly unrelated one at the top, are being vitriolic, and didn't back up your claims with respect to their intent and refusal to change this.
Well it's related because Mozilla actually cares about your privacy vs DuckDuckGo which obviously could care less from their reaction to this issue. Their refusal to change this is all the proof you need to know. I dont even use DDG I use Google I just think its funny they have a "Privacy browser" that sends all the sites you visit back to their servers
It's not immediately obvious whether it is more privacy preserving if the client automatically makes a request to each site in the search results while scrolling through the results, especially since you're already trusting DDG when performing the search.
Maybe this should be an opt-in rather than an opt-out feature?
Edit: as pointed out by warpspin in another comment, this is about the DDG Browser, not search results.
Scrolling through results is not a good place to ping websites for their icon. Not from a technical perspective but also not from a privacy perspective.
I appreciate you answering, probably knowing you'd face some negative feedback. Saying "we should trust you, it's for a good reason" is what google and everyone else says. You'll be better off if you just end this. The loss of the fav icon is less important than keeping your credibility.
thirded. an inconsequential token is not a suitable reason to engage in this kind of data collection, especially for a search tool that prides itself on 'not tracking' users..
Fourthed, and honestly it's strange that they are willing their promise to privacy on saving 2-3 queries on something as trivial as favicons? Why does no other browser need to do this?
They've already started down that path, judging by all the DDG billboards I see driving my 18 wheeler around the country, and all the DDG ads I hear on NPR-related podcasts.
Not that I'm against making money. But there's a tipping point associated with some height value in a pile of cash, and once you cross that point then the pile controls you. DDG probably hasn't crossed that point yet, but self-justification is one of the steps on that path.
Chrome, according to my understanding, hardcodes a few favicon URLs for builtin search engines, and caches everything else on site visit. There's SQLite3 database named Favicons in your profile directory (e.g. on macOS it's ~/Library/Application Support/Google/Chrome/<Profile>/Favicons). Here's the schema:
CREATE TABLE meta(key LONGVARCHAR NOT NULL UNIQUE PRIMARY KEY, value LONGVARCHAR);
CREATE TABLE icon_mapping(id INTEGER PRIMARY KEY,page_url LONGVARCHAR NOT NULL,icon_id INTEGER);
CREATE TABLE favicons(id INTEGER PRIMARY KEY,url LONGVARCHAR NOT NULL,icon_type INTEGER DEFAULT 1);
CREATE TABLE favicon_bitmaps(id INTEGER PRIMARY KEY,icon_id INTEGER NOT NULL,last_updated INTEGER DEFAULT 0,image_data BLOB,width INTEGER DEFAULT 0,height INTEGER DEFAULT 0,last_requested INTEGER DEFAULT 0);
CREATE INDEX icon_mapping_page_url_idx ON icon_mapping(page_url);
CREATE INDEX icon_mapping_icon_id_idx ON icon_mapping(icon_id);
CREATE INDEX favicons_url ON favicons(url);
CREATE INDEX favicon_bitmaps_icon_id ON favicon_bitmaps(icon_id);
I would hope that it can also be made to cache 404 and 410 responses to favicon requests too (or at least 410 even if not 404), so that it won't keep trying to access it.
Also, in SQLite, note that LONGVARCHAR is the same as TEXT, and that you don't need to specify both UNIQUE and PRIMARY KEY (it is redundant), and that if it is not a INTEGER PRIMARY KEY and not WITHOUT ROWID, then it isn't the real primary key but just an index (same as UNIQUE); add WITHOUT ROWID if you want to make it a real primary key, but note that the way the data is stored differs then, and WITHOUT ROWID is inefficient with tables storing large blobs.
In germany we have the words "Datensparsamkeit" (data parsimony) and "Datenvermeidung" (data prevention) [1]. Which wikipedia merely translates as "Privacy by design" [2].
DDG is unneccessaryly producing (aggregating), transmitting (and collecting?) very sensitive user data here, which is just the opposite of data protection. I can't even understand why they try to justify their actions. It's like omitting the seat-belt in a car, then telling customers that this was required to make the in-car entertainment system more usable.
In fact I think what they do here is illegal by GDPR. It does not matter that they say they do not collect the information, it is enough it is unnecessarily sent to their servers to make the whole function illegal.
The transmission of ip address alone, which is necessary for the TCP request to happen, deanonymizes the request enough to not be considered anonymous within the GDPR framework.
GDPR Article 5 (1) c:
"Personal data shall be ...
adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’);" - this is the "Datensparsamkeit" you mentioned.
Exceptions from Article 7 do not apply: The user has to give wilfully give informed consent, which he cannot do as the privacy policy of the browser omits the information that all visited domains are transmitted to DDG servers.
GDPR Recital 30
"Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags."
Oh, and the fact I'm downvoted for a purely informational comment additionally does not shine a good light on DDG.
Yes it does not look very GDPR conformant. They may try to argue that the transmission of visited domain names serves a purpose (browser performance) and that the user agreed to that transmission by accepting the TOS etc. However, I think they may be in trouble as consent for data processing under GDPR needs to be given "freely" [1]:
"When assessing whether consent is freely given, utmost account shall be taken of whether, [..] the performance of a contract[..] is conditional on consent to the processing of personal data that is not necessary for the performance of that contract."
As DDG's favicon-hack is not strictly neccessary for operating the DDG-browser, DDG would need to give users the option to opt-out of the favicon-retrieval, otherwise they may have "forced" the users to consent to the data processing, thereby voiding that consent as far as the GDPR is concerned.
You are completely right about the not "freely given" aspect. But well, it already breaks down at the "informed" aspect.
Their appstore pages link to the generic privacy policy of the company instead of for the browser specifically. This alone will usually already NOT be accepted by the supervisory authorities at least in Germany - the Landesdatenschutzaufsichten usually require a product specific privacy policy being linked for it being valid (personal experience), or at least making clear in the general policy what applies to the product, what not. And as mentioned, the fact visited domains are sent to the DDG servers (also outside the European Union) was not mentioned at all in that policy when I looked some minutes ago.
I think you are being downvoted because your interpretation of GDPR is extremely broad and people disagree with that. I also don't think it's in line with the way it is being applied in practice. Nothing to do with DDG.
I guess I might be one of the few people here who actually have been regulated under the GDPR by a supervising authority.
When that happens, you have to do insanely stupid seeming stuff like explaining in your privacy policy even, why your application, which is clearly for accessing a webservice, actually needs the Android permission to access the internet at all, true story. So I am pretty sure, an app sending visited domains to a central server outside the EU without even a mention of that fact in the privacy policy will cause problems with them, should they ever check the app. Of course, other countries might handle this differently, as Ireland shows.
So whoever thinks my interpretation is overly broad should first have the decency to step forward and actually explain why, instead of hammering a button and second, talk to me again after he had a meeting with the responsible authority and listen to THEIR interpretation of the GDPR ;-)
People should not mistake my interpretation with endorsement of the overly broad text of the GDPR itself.
If it is an unnecessary request to another service, yes.
IP-adddresses are considered personally identifying information. TCP requests transmit IP addresses.
Under the strict interpretation of the GDPR, a lot of things which are common outside the EU might be illegal, like e.g. embedding Google Fonts. To be on the safe side, people usually at least list these external dependencies in their privacy policies to construct some kind of "consent", but till we have more actual court rulings, this is a huge problem area.
For the problem at hand, it is pretty clearly illegal, as it's not only an ip address transmitted, it is a combination of ip address plus visited unrelated domain. This allows the creation of profiles. It does not matter for the GDPR, if the profile is ACTUALLY created, the pure possibility of creating it any time is enough to be a problem.
I don't think this is an accurate way to analyze GDPR compliance. As the staffer points out this favicon service follows their own privacy policy, if by this policy they keep (or analyze, sell, distribute, etc.) no data on your use of the service then there is nothing of interest for the GDPR.
They might have to prove that their privacy policy is indeed GDPR conformant and that their service works as advertised, but in practice this is likely more about public trust that legality.
Art. 4 GDPR (1) clearly makes the (ip-address, visited domain) tuple personal data
Art. 4 GDPR (2) defines "processing" data, and the pure "collecting" of data, even if immediately thrown away, is usually already considered "processing", therefore the GDPR applies.
If you are doubting this, just for a moment imagine, instead of the visited domain they would have sent all form data, including for example credit card data, you entered somewhere on a third party webpage to their central server and did not mention the fact in their privacy policy.
Do you really think then there is "nothing of interest for the GDPR" just because they do not actually permanently record that information? It would clearly be a violation. But to the GDPR, the importance of that data is equal. In fact, the domainnames might actually be more important to the law, as article 9 establishes event stricter rules for "sensitive" data about e.g. health or sex life of a person, and the domainnames might just leak that information.
> Wait what? a TCP request already breaks the GDPR rules?
If the TCP request carries personal data like the name of a visited website plus the user's IP address, then it "breaks the GDPR rules" in so far as you now have to fullfil your GDPR transparency/consent etc. duties /before/ sending that request.
Maybe not all website names look like sensitive data to you, but some website visits you surely want to be treated as sensitive, personal data (like names of hospitals, doctors, political parties, religion etc.).
You really can't use "we promise we won't misuse the information" as an argument, that's what everyone says whether it's true or not, and the whole point of using a privacy-centric browser is that as a user you can't trust those kinds of promises.
They’re primarily a search engine, so yes, you definitely have to trust that they won’t misuse information about what search terms you are querying for.
I suppose that's valid. This is the first time I had heard that DDG has a browser too, and I was just assuming that anyone who would use the browser probably also uses the search engine, which they obviously have to trust when they send search queries to it.
By the way how DDG spend money on the ads like at bus stop, I already starting to NOT trust them. Spend the money on development of development of products.
We had already had created this anonymous favicon service for our private search engine. In addition, doing it this way avoids another request (and potentially multiple) to the end site.
The service is private as we do not collect any personal information (e.g. IP addresses) on any requests for this or any service and the requests are all end-to-end encrypted.
> In addition, doing it this way avoids another request (and potentially multiple) to the end site.
Potentially saving a few requests here and there is certainly not worth phoning home with that kind of data regardless of what records you keep how much you do to anonymize it. This is especially true for a company that has built its brand on promises of privacy!
Besides, favicon requests are small potatoes compared to the kind of tracking, ads, metrics, and other often-unnecessary page resources that bog down most of the modern web. And a well-designed website can mitigate the issue pretty easily.
It seems like DDG just doesn't want to do more work to make the browser do the sensible thing and instead piggy back on their existing favicon service, and then going to make the argument that it is safe with us. The domain of browser is totally different from reaching out to the website and rendering favicons server side. Do you not see this as a problem? I think it is acceptable to say "We agree this is a problem, and we are gonna look into fixing this" instead of pushing back on an obvious problem.
Surely you could fetch the favicon asynchronously and update the address bar or tabs or wherever you display it only after the other assets are loaded? It’s not like the DOM has elements that depend on it. This is a really, really weird hill to die on.
I understand we’re suppose to take your word but it’s making me skeptical now about the actual intentions. If DDG truly cared about user privacy then you shouldn’t transmit user information for something as trivial as favicons.
I'm sorry, but this is bullshit. If the received HTML contains a <link rel="shortcut icon" ...> element, you already only need one request, so this is not an improvement. If that is present, you should NEVER use your API.
If it's not present, then you have options, and yes, using your weird API is an option (which I still don't like, but ok).
But sending private information to your servers even when sites follow the standard show either that you're probably not trustworthy, or that your product team is so painfully incompetent that I'd be afraid to use their browser at all.
> The service is private as we do not collect any personal information
This sentence is 100% meaningless. I understand you have good intentions, but these things must rely on proof, never on trust. Either you get this information or you don't; whether you say you "collect" it is inconsequential.
> We had already had created this anonymous favicon service for our private search engine.
I don't think here's a need for adjectives here. Why stress that it's anonymous (when that's hard to verify) or that the search engine is private, when that too is starting to come into question? Repeating these things won't will them into the reader's perception.
> In addition, doing it this way avoids another request (and potentially multiple) to the end site.
This isn't true, unless I'm missing something here? When I access a website, the HTML response I get from that website includes all the information my browser needs to, on its own, get and display the favicon. Can you clarify why you think/say this avoids one or more requests? What mechanism is this service a substitute for?
Saving a request is not a good reason to phone home, especially given what the DDG brand is built on and the main reason I would use it over Google. Please reconsider.
Please ask your higher-ups to educate you about essentials of privacy in the modern age. It's not about what you may be doing wrong, it's about what you technically could do wrong.
I’ve been an avid DDG user for years and it worries me that DDG staff don’t see why this is an issue. We shouldn’t have to trust your privacy policy if you minimize exposure.
That's not fair. There is a very clear reason: it simplifies development for them and avoids duplicating functionality in two different codebases. You can argue about whether or not that's a good trade-off given the privacy implications (and I would agree that it's not) but you can't claim that there is "literally no reason".
Does Gabriel know about this? If not could you please clue him in and get some guidance because you are absolutely getting roasted here and are wrecking DDG's carefully built up reputation. I can easily see how this might seem to be a good idea to you and other DDG engineers but it goes 180 degrees against DDG's stated mission. In other words: you may be well outside your paygrade on this.
Not as worse as publicly denouncing an honest engineer while referencing his paygrade. I hope there is no affiliation you have with DDG to be honest, because this is much, much worse.
What do you mean? This response is fine. It’s honest feedback, and it isn’t a personal swipe to say it’s above someone’s pay grade to be responding to a PR crisis as an honest engineer.
I was once an honest engineer too, publicly. Being honest in private is enough for me now. It’s a lesson worth learning.
That said, it’s not like a single HN comment will make or break a company, so if they’re really just a rank-and-file engineer, I hope the company won’t come down on them too hard. A simple “don’t do that” would suffice.
Yes, but chastising an employee is not in the interest of good PR, even if we sadly see that quite often in the days of social media. That is especially true if the employee just honestly laid out the facts, I wouldn't even call that a mistake.
I use DDG and the possibility of getting a statement directly from an engineer conveys much more trust than a carefully crafted PR statement ever could. I would think again about using it if the company does indeed come down on employees that live the values the company writes on its flags to have honest and transparent business practices.
That said, I am careful too when I state things about my company, even if I believe there is nothing to hide. Still, people that think it isn't the place for others with knowledge to comment are often not too impressive and would have difficulties in convincing me that privacy and transparency are real goals instead of just looking decent enough.
Furthermore the naming of management of DDG creates a stark contrast to the suggestion for more professional distance. I don't like PR very much as you might have guessed, but like a good design it needs some congruence.
If people find out that you just shut up for your company, it might give people the wrong impression about their business.
It’s your duty as an employee to shut up for your company. I didn’t learn this until later in my career. Fortunately it didn’t have lasting impact.
By commenting on an ongoing PR crisis without consulting management, you are both undermining their ability to respond in an effective way — imagine how strange it would look to see a “Hey, X from <company> here” after an existing one was already posted — and you’re acting on your own rather than in a team. You’re a part of a team; how could you think it’s a good idea to act alone?
Of course, I am talking to my former self with this comment, since that’s exactly what I did at S2 when working on HoN. It was a mistake, and I gave the community the wrong impression about the company’s priorities.
You have to understand, when you’re given money to do a job, you’re not given authority to become that job. Just because your job is getting beat up on social media doesn’t mean you should just jump in and go “Hey, that’s not true!” It doesn’t matter whether it’s true. Here, let me pretend to be DDG:
“Hi, Shawn from DDG here. You’re right; this was an oversight on our part. Obviously we dropped the ball on this. To clarify, we were unintentionally gathering the data as a side effect of our favicon service. <some technical details here>. We’ll be acting immediately to reverse this, and we’ll be enacting policy changes to ensure that user privacy — our core mission — is maintained going forward.”
But that’s not what they said. And if you’re gonna tell the community the opposite of what they want to hear, you’d better be in charge of the company’s Telling The Community Things division.
Thank you for explaining this all far more eloquently than I ever could. DDG is precious and worth preserving, it wouldn't take much for the press to have a field day with a couple of careless comments. And no, I don't have a stake in DDG other than as a user.
I don't think it's fair to say this comment is 'wrecking DDG's carefully built up reputation'. If what he describes is damaging their reputation to you, that's on the business, not on the commenter. If anything, the commenter seems to have been honest and straightforward with what's going on.
Sounds like you'd prefer him to have run a message past management/public relations first?
If you speak in a public form in response to something that seems damaging to the reputation of the company in a way that is probably even more damaging to the reputation of that company you are either an official spokesperson or way out of your depth. Pointing that out is a service, not a sleight. And yes, if this is not in an official capacity he should either keep quiet or run it by the PR dept. People have lost their jobs for far less. Anyway, I've pinged @yegg, hopefully he can shine some light on this.
No, it is the response of someone who was once upon a time CEO of a startup that fortunately did not operate in the present day news cycle where reputations can be destroyed in a couple of hours. DDG is precious, saying things that are very much against what I know to be the founding principles of the company is something that should only be done by those that have been given the proper authority. The response here is candid but ultimately not the one that DDG would and should give. Gabriels' contribution upthread pretty much confirms that. My comment was simply to ensure things would not get worse than they already were until someone in a position of authority at DDG could step in.
All you have to do is to read this comment thread to see the kind of damage that a single statement by someone affiliated with the company can do.
For me it does the opposite. Hearing engineers speaking off-message inspires trust, while pr-evasion and weasel speak makes me think they have something to hide.
That's nice. But in this particular case the engineer says things that I know for a fact are not in line with DDG's mission statement so even if he speaks off-message he's actually destroying that trust. There is nothing about @yegg's response that strikes me as weasel speak, it is the content that is different and exactly where it matters: privacy first.
I’m very disappointed in how you guys responded to this. As a privacy focused company I would not have expected an answer that sounds like it came from a data collecting company like Google.
I just switched to DDG browser a week ago and will now be looking for a new browser now. I hope you know this is not an appropriate response to the situation. Especially because all you guys do is preach about how much you protect your users’ privacy. Now you’re here asking us to trust you not to abuse our data and just linking us your privacy policy. I’m sad to say that my faith in the DuckDuckGo company and team is now lost.
Are all these edge cases part of the html/w3c/whichever standard? If not, let the edge cases fail. I'm not going to lose sleep over an icon not showing for a site I'm visiting once.
I'll state this in no uncertain terms: this is not acceptable, and you need to stop doing it. It makes sense on your search engine, but adding it to your web browser is very much over the line.
I have read your explanations in good faith and they don't cut it. This behavior cannot continue. Good privacy promises are not based on trust - they're based on not ever handling private data in the first place. If you don't quickly admit your mistake and roll this back, it will jepoardize your entire brand - and rightfully so. If you believe this behavior is okay, then it demonstrates incompetence; if you don't believe this behavior is okay but do it anyway, it demonstrates malice.
I'm surprised at how you're handling this. DDG is supposed to be friendly to privacy-aware users. You're dismissing people's valid points and asking them to trust you, just like any other privacy-non-friendly service would do.
Edit: I'm speculating here. But specifically because of the way you've replied here and on Github, my actual level of trust in DDG team went down.
What a bizarre potential privacy flaw to introduce for a tiny little icon nobody cares about. I understand it, usability and UX is important - but you guys are DDG! Come on! Your customers all have tinfoil hats!
At this point we're all well aware why the app phones home. Continuing to spout that like it's some ward against the fact that this is a very real vulnerability is an insult. Trust me, your target audience doesn't give a crap if favicons work; they care that DDG acknowledges the risk of a glaringly obvious vulnerability. Who do you even think you're arguing with on HN and GitHub? My children can't multiply yet but they'd be able to understand why this is bad practice.
The repeated handwaving that no one in your company is ever going to do something bad or stupid when the browser phones home for what amounts to a cute sticker is extremely suspicious.
Maybe I'm too old for this, but wasn't a favicon supposed to be located at "fancy.url/favicon.ico", or alternatively as a "<link rel="shortcut icon" \>"?
Doubling down is fairly ridiculous when one has to imagine the original reason for doing this was to save on time and engineering for the app by leveraging what DDG had already built, but a mindful response would be that you are aware of the downsides of this approach and you'll be working to change it.
Besides, how do you handle Intranet, VPN sites, and auth-only sites where DDG's god-tier favicon parser in the cloud couldn't fetch the URL anyway?
This should be the top response. Just under it should be a DDG staffer telling us they will be turning this config option off by default starting with the the next release.
This is essentially the same, but pi-hole gives you a nice UI, stats etc, and of course you can point all your machine at a pi-hole instead of editing the hosts file on each.
Sadly, not anymore - used to be the case quite a while ago, but not today anymore as the ads come from the same domains the videos are streamed from.
Those simpler popover-ads which can be closed clicking an X in the upper right corner still are blocked tho...
Spilling my secret tho (and YouTube execs hate me for it!): i block YouTube in my mind and only rarely go to it if i really need to watch a video (which, for me, is rarer than i ever thought it'd be).
The reviews are in for this response and they are bad. It's concerning that given the react it got, there's no edit addressing the concerns. The HN audience has to be the power user, bread and butter of a product like this and when you see a company ignore the concerns of a key constituency like this, their future almost never looks bright.
It’s amazing how tone deaf technologists can be when it comes to privacy, even when they have nothing to gain by exploiting the user’s data. DDG’s response reminds me of Mark Shuttleworth’s argument that they “have root”, so we can trust them with our life.
Dear DDG, you are getting complaints on GitHub and Hacker News. This is not the general public, it’s people who understand the issue. You should definitely reconsider whether you’re doing something wrong.
The road to hell is paved with good intentions. At the end of the day, your privacy policy is just a bunch of words with nothing to actually prevent you from abusing the data collected. Instead of us relying on DuckDuckGo to act ethically, just don’t collect the data in the first place.
... is completely irrelevant. Even if they were trying to save babies from a fire (which they really aren't) it wouldn't excuse the fact that they're doing something orthogonal to their stated policy and sole reason for existing.
Everyone makes mistakes, that's not the point. The point is to correct them when they're found, instead of digging one's heels in the ground and pretending it's nothing.
This is a bit concerning that for a company with a marketing so focus on privacy, you guys thought it was a good idea to have such a service.
Nobody in the company at any point thought that it could be a problem?
Your strict privacy policy mean nothing by the way, because you are a USA company and you must respect your local laws such as the patriot act, which are not very privacy friendly.
Does it have an option to disable the internal favicon service and/or an option to disable favicons entirely? (I disable the favicons on my own computer, since I don't use them.)
What is this possibly meant to imply? Nonprofits run international ad campaigns, governments run international ad campaigns, the military runs international ad campaigns, most organizations could probably find a reason if they have international domain. What is your insinuation specifically?
There's an interesting disease showing up here in the responses.
I accept DDG's statement that this is about a favicon and that they "do not collect or share any personal information", and despite that, I also agree with others that DDG should be on the safe side and just stop doing this small thing. It's just the safer and more moral thing to do (So DDG, as many are suggesting, plz stop doing it. Today is good).
But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do. There's no measure of proportion in the responses, someone is making a mistake then there's a wolfish, pack-like desire to get stuck in and hurt someone.
Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong. DDG isn't the big evil on the web but from reading some of these you'd think it was the 2nd google.
This isn't about DDG, just the proportionality of responses in public errors and what society you'd like to have.
Oh, please. There's a million threads on HN complaining about Google and Facebook. Doesn't even matter if they did something, any post will still have those comments. You can't consider the proportionality of the response by just one thread.
It's really quite amazing that when a company that's hitched it's brand entirely to privacy first commits a big privacy faux pas, hides it for a year, and then doubles down on it not being a problem, you have somehow managed to turn the top voted thread to a discussion on the failings of other companies instead. Bravo.
Huh? It's an open source project. Maybe you consider them closing the issue on GH to be "hiding it" but I don't. And the plenty of people talking about it in that thread still apparently don't, either.
IMHO their response made sense. You're already trusting DDG with your search history which for most people might as well be a list of 99% of the domains they visit. If you trust their privacy policy for that, I don't see the big deal here.
I do agree that it should be changed, but only because as is it seems imperfect and I seek perfection in most things. Again, it's an OSS project. AFAIK, any one of these people comprising the screaming masses are free to fix it themselves. Personally, I don't think it's worth my time.
The point is DDG has insisted that they put privacy first, & don't collect user data, then (supposedly) in the interest of a small performance increase they introduce code that — wait for it — COLLECTS USER DATA. All they would need to do is use {URL}/favicon.ico which most if not sites have. From what I understand according to posts on the github page this could be done on-device so no need for sending anything to a server. Then when the issue was opened it was closed without the issue being fixed.
It was not IMHO meant to deflect from DDG. It was a more general statement about peoples responses. I dislike trendy phrases but the one in play here is "outrage culture". Its not good and it's getting more common.
This breaks the HN guidelines against accusations of astroturfing or shilling without evidence. If there's one thing I've learned from looking at the data for countless hours, it's that internet users are far, far too quick to jump to this interpretation of what they perceive on the internet. The overwhelming majority of such accusations are purely bogus, and they poison the community, so we don't allow them here.
Actual evidence is a different story, of course, but then you should be emailing it to hn@ycombinator.com so we can take it seriously and investigate.
What would "actual evidence" look like? Proof that the comments in question originate from a known social media marketer? Do you think they all share a specific IP?
I think what angered people was actually that a company saying to hold privacy high was simply refusing to change something after a mistake was pointed out and instead kept on defending it with a technical argument, which makes no sense at all.
The reaction would have been actually a lot different if someone from the company admitted the mistake and promised it will be changed.
You are faulting someone for defending thier own argument. You suggest that people who do not cow and apologize to the mob deserve the anger and retribution the mob has to offer.
People have a right to think differently and express themselves without threats, bullying, or shaming.
The mob does not deserve apologies. The comment above is spot on - we've lost all sense of proportionality.
It is an indication of the modern online mob sickness that they always demand others beg for forgiveness.
What emotional void are mob participants trying to fill with the apologies of others?
you obviously should be allowed to make a mistake and be forgiven for it. that does not mean that i personally would ever forgive any `company` that markets itself as pro-privacy after its been caught gathering data on its users.
i could forgive the people working at the company and would definitely expect future employers not to hold that against them, however.
but if a `company` does something while claiming to stand morally opposed to exactly that.... proves that it doesn't actually care about the topic. it just wants the publicity for marketability, discrediting them entirely for all future communication.
in this particular case, i wouldn't go that far however. they weren't gathering any data on their users if i understood it correctly. it was just a badly implemented feature, which will get changed
I was specifically responding about your outrage how internet 'mobs' demand forgiveness from the people they've supposedly wronged.
The whole comment was just me talking from the perspective of a possible self identified victim and how that person (me) would respond in such a case.
Forgiveness would've to happen for me to trust that party again, which would be a requirement for me becoming a user/paying for the service again. If that possible other party doesn't care about wherever the people they've supposedly wronged use/pay for their services, then there is obviously no need for any interaction after that point.
And as I said before: none of this applied to ddg, as they didn't spy on their user. They implemented a leaky feature, which they've committed to changing.
Once people have gone down the avenue of earnestly reporting private information leakage, the correct answer is to investigate. DDG decided not to do this and instead dismissed the problem completely out of hand and ignored it for almost a year without taking any action.
No one asked for an apology, they asked DDG to admit fault and then to fix the problem. ie "we shouldn't have done that" not "we're sorry we did that."
You keep saying "mob" but these people didn't collect to harass, if you actually read all of the comments the vast majority are people who are (rightly!) concerned about their data privacy advocate built software leaking every visited URL.
You’re absolutely ignoring the possibility of an argument where there is no fault and this is being blown out of proportion. You just assume your opinion is correct and they owe you to fix it. That is in itself the problem OP was exposing.
A performance optimization around a trivial thing like a favicon is no reason to destroy everyone's security.
Let's assume DDG is a great and honest company and will collect all the info, but never use it for anything bad. Guess what, they can get still hacked and all the info leaks out.
This is a horrendous breach of trust that they WERE collecting it however, and I'm glad they got caught. It will not be tolerated. They'll have to change this or face a revolt.
If you claim to fight for privacy and rise to popularity by shaming your competitors' evil anti-privacy (or shady) practices and do something exactly what the rest do, you deserve every bit of criticism. Why should you receive a free pass and the competitors shouldn't? Simply because of your marketing stance??
And for the record, collecting your browser history just to display a stupid favicon is the most ridiculous excuse I've heard in a long while. And I am not going to blindly believe them because they said that's what they use it for.
As a privacy-first company, for them to make any decision that is for performance but adds additional privacy risk shows they make the wrong decisions.
They also left this for over a year after being told about it and part of the only reason it was caught was that their browser app is open-source. What about all their closed-source services, like their favicon service their browser apparently relies on.
We shouldn't blindly trust any company and a privacy-first company should be willing to assume we don't trust them and therefore it is up to them to prove everything they do.
I get a few of the responses on their Github page are now non-helpful, but the big point here is that by DuckDuckGo taking this series of actions (implementing the browser like this, ignoring a valid bug report about it, and only reopening the issue after massive push from users) it shows that DuckDuckGo isn't as privacy-focused as I thought, nor many others.
I've been using DuckDuckGo as my sole search provider for several years now, on all my devices, but I'm concerned about what else they may have done in the sake of "performance" over privacy.
"collecting your browser history just to display a stupid favicon is the most ridiculous excuse I've heard in a long while"
Though I agree the the implementation could be better, they should just check the head and the root for the icon and if not found that's that.
But the possibility of something be used for malicious ends does not entail confirmation of it being used that way.
Just because we have knifes in our kitchen does not make us automatically guilty of stabbing people.
I find it easier to believe that this was actually, if misguided, an attempt to solve a problem rather than a nefarious plots to track users across the web.
> But... the reaction here is "they made a mistake, let's pile on like kids in a playground"
It's not one mistake, but several. Other then the initial mistake there is also the sloopy reaction and the fact they just closed the issue without bothering to fix it. And this was 1 year(!) ago. Nothing changed in the meanwhile. Now someone pushed it to public and after just some hours they reopen the issue and promise to fix it.
This is the reason why people react loud, because it works. And often it's even the only way that works.
Part of the point is that likely this has zero privacy implications (except potentially disclosing to someone monitoring your traffic that you are using their browser).
The mistake is rather an area of improvement where they can change something that respect privacy by policy to something that respect privacy by design.
>Part of the point is that likely this has zero privacy implications (except potentially disclosing to someone monitoring your traffic that you are using their browser).
It has zero implications if you trust DDG and good privacy is not based on blind trust. Keep in mind that you also need to trust the government under which DDG acts to not require them to disclose this data, trust the government to not put black boxes in the DDG data center, trust DDG's security apparatus against external state actors, trust any rogue DDG employee to not use this data and so on.
I agree, I still think it is relevant to treat actual privacy violations differently from engineering choices that might make privacy violations easier in the future.
My opinion is that DDG should have never made this choice in the first place, but as far as I am concerned this is at the level of an implementation detail that can be improved, not as if DDG was intentionally using user data in some non-private way.
The concern as I see it is that this issue and the initial DDG response to it shows a lack of understanding of technical privacy controls. What they are and why they matter so much. DDG's backend is not audited and so one wonders if that same lack of understanding applies to the backend as well. DDG cites privacy policies but, in my experience, the best policies are backed with strict technical controls as humans never follow policies perfectly on their own.
Anyways, what’s this got to do with Google? “Privacy browser violates basic privacy to do something useless” is actually ridiculous. And the response is even more ridiculous! How does literally every single other browser do it?
I’m disappointed because I put my reputation on the line to recommend DDG to users based on...privacy. But here we see they actually do not hold true to their stated values. And they don’t even seem to care.
It’s an explanation of why complaints are being directed at DuckDuckGo (privacy is their raison d’être) even when Google is worse (privacy is not their product – or, in wording that’s more ambiguous in isolation, Google don’t sell privacy).
Or why wear a motorcycle helmet since there's still a chance you'll sustain a serious head injury and die if you crash? Like you, I find the extremity of the GP's point of view lacking in wisdom.
I don't know how the rules are here about getting personal, but aside from making other users be uncomfortable you're missing the point completely.
I don't personally feel a need for a high level of privacy, but I can understand other people who do.
And if they are using DuckDuckGo for that purpose, this issue would seem like a big thing.
If you served a dish for a vegan with 5 % meat in it, you wouldn't justify it by comparing to the amount of meat in non-vegan dishes.
You're okay with them having data on what you're searching for, but them potentially having data on the domains you're visiting from those search pages is where you draw the line?
I mean, I agree with the principle of storing as little data as possible but this isn't a huge deal in my eyes.
I disagree, or maybe we read different responses, but the ones I read where more critical of how (a) ddg (employee) handled this.
Didn't see anyone claim that this was on a google-level of bad, more like pointing out that google started out as a small company wanting to "do no evil", but slowly turned into what it is today.
Is it really that weird that people are worried that this might be the first of many small steps down the slippery slope?
I didn't have to go through the posted link for long to find people reacting as though this was more than a slippery slope issue:
> But instead to react professionally and contritely you made it worse to stamp on the shards to make sure no useful piece of trust will survive.
> I've just de-installed the Duckduckgo app and also won't be using their search engine anymore. Trust is lost. Their CEO can put his statement where the sun doesn't shine.
> But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do. There's no measure of proportion in the responses, someone is making a mistake then there's a wolfish, pack-like desire to get stuck in and hurt someone.
Sadly people simply derive satisfaction from piling on like this. The pop psychology explanation is that it's because people are dissatisfied with their own lives and are lashing out at anything that allows them to vent that underlying frustration. It sounds plausible, but it also sounds like it might be an over-generalisation.
I think it's certainly fair to say - as depressing as it is - that it may be somewhat in our nature to behave in this way, that most or all of us may possess this characteristic to a greater or lesser extent, and that the current political, cultural, economic, and media climate is only serving to amplify that tendency.
The current social media climate favors disagreement. This is directly in correlation with 'engagement' around which platforms are designed. Call it banking on outrage/hacktivism/quarrelling or simply engaging in peaceful debates. Companies and their algorithms have come to a funny conclusion that this is what increases said engagement (I'm not questioning their business model). User retention is crucial in this era, so is the intention of getting them to check back in ever so often.
Thus, dark patterns emerge. Point is, once you get used to 'that' online-aura, you unwillingly carry it with you wherever you go. Human brain does not have buttons to switch between modes. It's comically easy to get into this mindset and hard to shun it.
My guess is that DDG is being held to a higher standard because it's the liferaft competitor in a world of Google data harvesting, targeted manipulation and selling customer data.
Most complaining or being heavily critical about DDG are probably already upset to the point of abandonment with other services and they don't want the same trend to happen to this competitor (DDG). This sort of reaction is, IMHO, due to poor diversity of viable competitors.
In our societal structure, competitive options are the only things that keep power in check. I'm personally not entirely convinced you can have a reasonable amount of diverse competition in our economic system and there are some inherent equilibria that we tend to converge on over and over again in a market space (without corresponding massive social equalibria shifts).
If you do have any sort of faith left in our economic system, then you certainly want competitors like DDG to be different and be successful. Even if you don't have much faith, outside of say stringent regulation, supporting these sort of competitors is really the only practical option we have in the current state of affairs.
Many argue that "other things are worse". This argument is invalid in my opinion. We're talking about this special case and if people feel like they need to openly show their opinion about it this is okay. Even if the response is big in numbers.
That just shows how much also the small things matter.
If you only care about "the biggest" or "the worst" you'll never get anywhere...
> But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do.
What about genocide too? Please people, stop with these "but the other bigger unrelated issue should get more visibility".
> Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong.
Can you point me where DDG admitted they were wrong doing this in their first response? They didn't... they just explained why they did it but completely ignore the greater issue because they consider themselves "good". Just like that politician you may talk about, or Google, or whatever. They are part of that bigger issue you mentions.
This is about DDG.
Luckily that pile of kids in a playground made them realize that mistake, they would have ignored otherwise (like they did on their first respond).
> But... the reaction here is "they made a mistake, let's pile on like kids in a playground" ignoring the genuinely huger issue of the amount of info and mining that google et al. do.
What about genocide too? Please people, stop with these "but the other bigger unrelated issue should get more visibility".
> Which is why politicians rarely admit mistakes, because it's taken as a sign of weakness, not strength, to admit you were wrong.
Can you point me where DDG admitted they were wrong doing this? They didn't... they just explained why they did it but completely ignore the greater issue because they consider themselves "good". Just like that politician you may talk about, or Google, or whatever.
Ubiquity did the same thing with their routers. They couldn’t understand why users had such a problem with their phone home feature that was on by default when the purpose of it was to ultimately “improve” the user experience. I didn’t buy their router as a result. I also removed kaspersky from my computer because I didn’t like their phone home feature. Turns out they were selling my data despite holding my trust as a security company. DDG, don’t turn this into a PR nightmare. We don’t trust anyone anymore. Privacy policies are worthless. Nobody cares about favicons anyway.
This is a bad look for a company that is trying to build its brand on privacy and trust. Even though I don't use the DDG browser I hope they own up to this, rectify it quickly, and learn from it.
In my view, anyone who trusts ddg is a bit silly - founder has a bad track record on user privacy. Founded Names Database[1], a social media website designed to collect user information as aggressively as possible, before selling all the information to classmates.com.
Also worth mentioning they're closed-source, US-based and for-profit. Why exactly do people trust them? Simply because they write a few articles/ads saying "privacy is important"?
If you're willing to sacrifice search quality for privacy, as in switching from Google to DuckDuckGo, then you might as well take a step further and switch from Google to Searx/Ask.Moe.
Because they have a good privacy policy. They would face legal consequences if they were lying. Nation state actors can presumably override privacy polices but it's better than nothing.
The reason I use them is not because I trust them but because they do not put me in my own search bubble like Google does. I hate that part of using Google. Also as a bonus I do not get any AMP results.
I couldn't figure out which searx instance to use and found no good way of knowing who to trust, most of the engines i used were broken and telling me to find another searx engine.
It looks like it's pulling most of its info from duckduckgo anyway.
Personally I'd rather trust a known entity than an unknown entity anyday, especially when the unknown entity is slow, complicated, buggy and broken in many places.
They’re pulling info from all the other search engines, not just DuckDuckGo. FYI DuckDuckGo is also pulling their results from other search engines, the only difference is that DuckDuckGo aren’t being blocked/rate-limited by them, presumably because they’re paying for API access.
If you hosted your own instance then it would be a lot more reliable since the IP wouldn’t send a suspiciously high amount of requests.
As for your trust argument, I couldn’t disagree more. You choose to trust DuckDuckGo, who happens to be closed source, because of their branding. The same way people trust/trusted Google/Apple/etc. because of theirs. This thread is a perfect example why being open source is the most important thing for any privacy service (because otherwise this privacy leak likely wouldn’t have been discovered, and people wouldn’t have known that the company so carelessly violate people’s privacy and fail to correct it when people point it out.. it should really make you wonder what’s happening in the search engines codebase).
As a DDG user I don't feel like I'm sacrificing anything. Two sets of results are better than one (I can see Google results by adding !g, which I do less than once per day on average) and, ironically, DDG bangs are the easiest way to use even Google services like Translate and Scholar.
For me the fact that they’re open source. This thread is a prime example of why it’s so important. We really have no idea what DuckDuckGo is doing because they’re closed source. For all we know they could be forwarding users’ IP to Microsoft/Yandex/etc.
If you want to market yourself as a champion of privacy, then the absolute minimum criteria should in my opinion be that your codebase is open source.
I don't really think is an ad hominem fallacy. An ad homihem fallacy is when you attack the person making an argument rather than the argument. But no one here is attacking the people making the argument.
The argument is "Trust DDG". That argument is being attacked as "DDG's founder has done bad stuff in the past, it's likely DDG will do bad stuff, so I won't trust it". That seems to be attacking the argument to me, thus not an ad hominem.
[My opinion on trusting DDG] - [Reason for my opinion on trusting DDG].[More information about the reason]
In this case, because I don't use the "personal attack" to reach my conclusion that the person was wrong, I don't think it would be a case of argument ad hominem.
But if someone read it like this:
[One reason for my opinion on trusting DDG] - [Another reason for my opinion on trusting DDG].[More information about the second reason]
Then I am using the "personal attack" to reach my conclusion that the person was wrong, so I think it would be a case of argument ad hominem.
My comment wasn't written very well, and I'll try to write better comments in the future. Not that adding an ad hominem to a valid argument makes it invalid, I guess?. But it's still good to avoid fallacies, and to write one's comments so they're likely to be understood the way one meant them.
That’s not an ad hominem attack. The comment didn’t say that the person’s argument is wrong because that person is silly. Clearly the comment is saying that the argument itself is silly.
The intent of the comment was probably not ad hominem, as if it were rephrased like "trusting ddg is silly" because then silly is modifying the act of trusting. But as-written, silly is modifying anyone (a person) without distinguishing whether silliness is the cause or the effect; if silliness is the cause of the trust then it's an ad hominem attack. It would be more clear-cut if instead of "silly" they said "low intelligence."
A common fallacy I see, though I don't know if there's a name for it, is assuming that just because what someone is doing can be described by the name of a fallacy that what the person is doing is fallacious.
> Argument from fallacy is the formal fallacy of analyzing an argument and inferring that, since it contains a fallacy, its conclusion must be false. It is also called argument to logic (argumentum ad logicam), the fallacy fallacy, the fallacist's fallacy, and the bad reasons fallacy.
I'm talking about situations where no fallacy has actually occurred, not situations where a fallacy has occurred but a correct conclusion has been arrived at anyway.
So what does any fallacy tell me then? Especially the fallacy fallacy? If it does invalidate the invalidation of fallacies it is in itself untrue...bz...recursion error..bz stack overflow
Privacy has to be scrutized constantly and be the top priority. If it isn't, then you're going to end up with another google.
A direct correlation exists between the revenue Google receives for selling data and the quality of its search. Google focuses completely on tracking and search, with privacy behind a far far away afterthought (if it's a thought at all).
I'm sure Google's thinking about privacy. They're thinking "will our latest privacy violation create enough of an uproar to affect our bottom line, or will the users just accept it like they usually do?".
The favicons on the duckduckgo browser are often worse than other browsers in my opinion. For example the BBC website where DDG interestingly enough just uses /favicon.ico and the other browsers use the apple touch icon. (Information I found from just looking at the pages headers)
Don't really understand why they do extra work to get worse results... This feels to me slightly worse than just a privacy concern, it's a misunderstanding of their domain which leads me to the question of what else do they not fully understand.
The good news is that you can have the DDG search engine as a default in other browsers.
(I understand that the DDG browser is probably not their main focus and any lack of knowledge can potentially be just on their mobile browser.)
BW already has all the URLs of services you use: you saved them yourself for them to keep safe. It's obvious you already trust them enough to know that sort of data.
So if favicons were gathered locally, then you would prefer that your own browser would reach out and make multiple requests to every (potentially dodgy) site listed on the search result page?
Note also that these are favicons for results that DDG has already given you. This isn't tracking your clicks. The list of sites that appear on the search result page is not new information to the search engine that just gave them to you.
EDIT: Like other commenters, I was not previously aware that DDG had a browser, and my comments were about this behaviour for the search engine results page.
Can you explain your edit a little more? I still only understand your original viewpoint. Just because there's an app doesn't mean they don't have to contact DDG servers at all, right?
We had already had created this anonymous favicon service for our private search engine. In addition, doing it this way avoids another request (and potentially multiple) to the end site.
The service is private as we do not collect any personal information (e.g. IP addresses) on any requests for this or any service and the requests are all end-to-end encrypted.
Your answer to all these criticisms is a lazy one. You are saying "this already exists for other things so we use it regardless of what the correct method is". Just because something exists in one form doesn't mean it's the correct choice.
I personally believe you, but this approach adds multiple unnecessary trust boundaries. Just thinking about how many separate logs are triggered outside of DDG is troublesome. And what happens when you guys think everything is secure, but one day find out it isn't?
I understand utilizing a single service/feature to accommodate multiple platforms is a big win from a programming standpoint. But if it carries the risk of losing the trust of your user-base while at the same time risks breaking the core mission statement of the business (privacy), it's probably not worth it.
Why not do it locally? It is one extra request to the favicon link in the header and cache it. Or cache it when the user visits the website. Or explicit button in the settings UI to refresh favicons locally.
What do you mean by "end-to-end encrypted"? Isn't this just a request to your service that will then go out and fetch the icon from the site, so it can't be e2e since you need to know which site to request the icon from.
Why use that term when it clearly can't be true or even seems applicable/relevant?
Yes, that might not be intentional and is used "just" for the favicon, yes they might not use the info on the domains you visit for tracking you today, but the data is there.
Why not use that data tomorow "just" to see what kinds of pages their customers (browser users) are visiting so they can better place their ads.. and then maybe some other idea.. this is a path that many such companies went ("don't be evil").
You either respect the user privacy or you don't - there is no middle "just for this little feature" ground
Seems a bit much, but k-anonymity could work here. Hash the domain, take the prefix, get a batch of favicons back. They won’t know which you visited, but still get the benefits of consistent favicon support.
When there's a miss, send the domain to the server so that it can fill the cache. You probably won't even have to do that as DDG could have already filled the cache with their search results.
Set duckduckgo.com as your default search engine with a blank home page. But you could also use @pkrumins home pages of https://techurls.com or https://finurls.com as nice home pages.
Use Mullvad VPN: https://mullvad.net/
(They are EVEN available on F-Droid now, which is AMAZING)
Advertising your Twitter for the advice of "switch to somewhat well-known browser X, install these very common extensions and use a VPN" is also a bit ... odd.
An ex DuckDuckGo employee recommending people use an alternative browser over the DuckDuckGo browser, in a post about the DuckDuckGo browser spying on its users, is about as on topic as you can get; after the current employee giving an explanation.
I haven't checked all links but some things need to be updated, for example Skimmer Scanner is gone from the Play Store and Yalp Store is abandoned and doesn't work anymore, you should be using Aurora Store. I'd also recommend Aegis or FreeOTP+ over FreeOTP for 2FA. NewPipe is better installed from this[1] repository until this[2] issue is solved.
Yeah, I do use Aurora Store. No longer use Skimmer Scanner. I have to update that post. Working on a new blog/site and a book (eventually). Thanks for heads up.
Stock Firefox doesn't actually have that much bloat, and it's not noticeably slower than Chrome on reasonably modern hardware (i.e.: most page loads are near-instant, same as in Chrome on a good connection).
... Until Google inexplicably restored it a few days later, but not before lots of accusations were thrown around.
---
IMO it's also worth noting that ungoogled-chromium is (obviously) an unofficial fork of Chromium. Google may at any time change Chromium so substantially as to either require Google integration at some fundamental level for even the most basic functionality, causing too much work for such a low-profile effort to continue, or just make Chromium closed-source. With Firefox, that risk doesn't exist because of the business motivations of the company that develops it.
As I said in other another comment, I'm a power user. And by power user I mean probably 10x the normal standard of what a 10x power user means. Tooling is everything to me; which is why I prefer this over anything else. I'm very biased.
You are probably seriously overestimating yourself and/or underestimating a large subset of your audience here.
People here are:
- early developers from majir browsers
- extension creators
- creators of major web properties
- people who spend their work days on the web
- etc
seriously. Don't underestimate HN.
Edit: and as someone who's been very into customizing and extending browsers since early Firefox: your ideas about Firefox vs Chrome tells me that you are either
- deeply biased
- use a subset of browser features that is too small to realize the problems with Chromium (in which case I suspect your 100x superuser is wild hyperbole)
You can come back when Tree Style Tabs including related sub-extensions works as well on Chrome as on Firefox.
When it comes to a browser, I'm curious as well. I often end up with 100+ tabs in firefox before I close them all. I don't think think I'd consider that being a power user, but would a 10x power user simply mean someone with over 1,000 tabs open?
I haven't used Chrome regularly in the last year or so, but it would get noticeably slow by the time I had 3-4 windows with 10-20 tabs each. Firefox hasn't really had that issue yet.
Years ago though, Chrome was noticeably faster than Firefox. That changed (IMO at least) at some point in the last few years.
I use Firefox day to day, but if I was a “power user” and picking apart web traffic everyday I’d probably use Chromium. The dev tools in it are way better than the ones built in Firefox.
That has fresh meme potential. How many 100x power-users do you need to make a 10x developer? And how many normal users for a 100xPU? Are 10 100xPU equal to a 1kPU? So many questions...
You can install uBlock Origin in 10 seconds from the GitHub release page (see previous link shared). This is vastly better than anything I've seen for ad-blocking.
Through better tech and with better developer usability, we can conquer these monopolistic false promises. Our work with the Lad framework is instrumental.
Also that just makes me trust them more – they are the only browser vendor that are trying to find business models that don't involve selling my data or selling out to someone who does.
The issue is that nothing is as pure as ungoogled-chromium is right now. I've also felt that Brave was slow, and not best for developers. As a power user I'm very biased, so Brave may work better for folks that are not as technically skilled and have no idea how to install from GitHub/brew/etc.
Edit: I feel it is slow because it has extra bloat added. Like I said, take my comments here with a grain of salt because I'm a very biased power user. I respect the efforts of these developers regardless of what project it is, the focus on privacy and building something different is truly awesome.
That's outdated information. Nowadays you can enable Encrypt All Sites Eligible and the extension will force HTTPS everywhere skipping the whitelist. It'll also prompt you to continue to the HTTP version if HTTPS fails to load.
Your "Security harden your mac" guide has a bunch of amazon affiliate links, it would be good to call that out when piggybacking on a post about privacy at least.
Thanks! I'm rolling out the following within probably the next 7 days:
- All new load balanced infrastructure
- Browser extension + API wrapper
- Support for pixel tracking blocker (opt-in checkbox or TXT setting)
- Smart alerting
- Globby/regex support
If you follow my Twitter or the GitHub releases you will get updates.
This is concerning because it indicates a lack of care in terms of privacy and understanding that the best privacy is achieved by knowing the least. Does this approach permeate their backend as well?
I'm not so sure. I think the privacy-functionality trade-off is understood and expected by the users, it woul be used by very few people if it were extremely spartan.
(But this instance, the favicon service, is not a good privacy-functionality trade-off)
What would a browser even need a backend for? The only valid use that I can think of is Google's Safe Browsing list, but if ad blocking can be implemented totally on-device, surely that can, too?
The use case I was going to suggest was a safe browsing list. Possibly also something like Have I been Pwned. So, yes, there are valid reasons to have a backend but they are very narrow and privacy is key when building them.
I was astonished by this at first, but I think you must mean "you're sending all searches performed with bangs to DDG". I worried that you meant somehow the browser search history was being sent to DDG, but that seems impossible.
well, this isn't something that I have given that much thought...
But yes, when I go to ddg and type in "!g <thing>" after having typed in "<thing>", I'm on some level hoping that this is somehow informing ddg that their search results for <thing> could be better.
Now, if they have somewhere stated that they are not going to do this, then yes, it would be bad.
Because it's the easiest (and sometimes only) way to get this functionality on major browsers and browser developers have no plans to implement this functionality natively.
I maintain a set of such keywords in my Firefox, that I use on iOS and Android too.
The easiest would be to just use Google btw, as it does a reasonable job to give you the website you're thinking of just by mentioning it in the search query, e.g "Frozen imdb" (though it does a perfect job in this case with just "Frozen").
If you use DDG, because I'm assuming you value your privacy, sending your search history to DDG when you could avoid it doesn't make much sense.
The main issue for me is that they only work in the url bar not the search bar. For the same reason people are upset about the favicon thing I don't want anything (mis)typed in the urlbar to be sent to a search engine. I've failed to find the correct setting to make this happen a few times before but just looked again and setting keyword.enabled to false prevents it from searching from the url bar (the setting is confusing and not related to the search keywords feature).
I did try firefox search keyword anyway once a while ago but it isn't that easy to configure and the setting were lost not too long after I started using it (I don't remember the details but I think my profile was corrupted by ecryptfs and the keywords didn't import to a new profile or something like that). Not importing them isn't a horrible choice (if that is what actually happened) considering there doesn't seem to be an easy way to even list them, but there should at least be a distinct import/export for keywords. The search page of preferences has a keyword field next to search engines but it doesn't seem to actually list them nor can you add a keyword from that page. All in all, it doesn't seem like a feature Mozilla cares about at all, although it would be a nice feature with a bit more work. Use from right click search on text selection would be another thing that would make it much more useful.
I tend to use ! in two circumstances, one as others mentioned when a DDG search didn't return what I wanted (and I am fine with letting DDG know that since it is the exact same search I just searched DDG) and secondly when I am looking for something on a particular site. Some of the second case would be best done via Firefox keyword search but the advantage of DDG is that they have a lot of obvious names going to the obvious place so rare specific searches are much more convenient than doing anything to set up a list. Since I very rarely use such searches, there isn't anything DDG could learn about me from avoiding just those searches going to DDG. OTOH, using site searches more frequently would be a win for privacy.
Haha, amazing to witness. This is the problem with catering to this crowd: your audience is suddenly full of people who just want to see you fail. Good luck, DDG.
I really like DDG - it works good, it is fast and it does not use my personal search for giving me "better ads" or "better search results" that put me in a filter bubble.
But there is a different issue here at play; because of errors like this the whole DDG brand gets a bad rap - and thats not only bad because of the risk of people losing a google alternative but because it is real easy to exploit situations like this for google-like companies to give an impression that "all this privacy thing is bs, all companies work in a same way". There are a lot of people that are not really sure is this "privacy thing" is worth the inconvinience of swiching to some other search engine/browser/app and situations like this one are not helpful in that regard.
Lot of folks are aware of this and are displeased for risking brand confidence of such a visible privacy-concerned company for miniscule gains like performance gains for fetching a favicon for the first time - just fetch the favicon after you display the rest of the page and cache it, maybe dont even try to fech it if the connection is poor - who cares really
Something is not adding up. Why would you go through so much trouble and over-engineer a favicon retrieval service? Really, favicon? Since when did they become so essential?
I'm pretty sure 90% of websites provide one in a standard way. If not, just draw a letter there, or anything.
But I don't know. I think that either there is more to this story, or DDG team completely lost common sense.
Nevermind privacy. How are favicons so complicated that they need a special service that understands edge cases. Just do it one standard way and if a minority of websites don't work, then exclude them. We've been through this mess before with all kinds of web standards devolving into mess.
Agreed, creating work arounds for non-standards compliant websites just eliminates the motivation and incentives to be standards compliant in the first place.
One major issue is that the "standard" favicon size has historically been 16x16 pixels... which, in the age of high density displays will render either comically small or comically blurry. There are other meta tags like Apple's "apple-touch-icon" which has some higher resolution options. But already you can see the logic here isn't trivial.
It still seems trivial to me. Parse all meta icon tags, prefer one that matches exact client resolution of current display (don't even have to download the icons - the sizes are defined as part of the meta tag), else use the largest-resolution one, else in desperate bid try site/favicon.ico, otherwise give up.
Really shouldn't be complicated enough to need a special service to handle "edge cases".
Another comment mentions web manifest - I guess try those first before meta tags, or whatever order the standard says to use. I mean, we're talking a web browser here, it's designed to do these kind of tasks.
I guess the problem is when you want to quickly provide a favicon with the search result (beit in the omnibar or actual search page), as you're typing and results are being displayed you cannot send off a request to the site, wait for HTML html to finish up downloading, send off a request to the manifest if its defined for multiple sites at once.
On a technical level of course its doable but in reality it's a complete waste of data and processing, not to mention it could take a long time to show up. I imagine they have these favicons all cached on their side so they can quickly send the right file down and/or do this processing if needed.
That being said maybe they should just not use a favicon if it's that big of a deal.
Oh, this is for putting an icon next to search results? Yeah, that changes the calculus considerably. I thought this was about showing the favicon in the browser for a site the user visits (per the issue title).
In that case, yeah, I don't think the icons are necessary to show at all...
Even if you wanted to implement this, the logic of the service could be directly embedded in the browser as an extension or similar. There’s no reason to depend on a network service for this functionality.
I feel like the only user benefit is to see the favicons of sites you are familiar with, so caching those locally after you visit sites is probably good enough.
I think that distinction needs to be made. I think DDG should treat this app as a web browser which means phoning home to this endpoint is unacceptable.
Whoever ever put their trust into American for profit companies which use slogans like "private secure and fast" should not be surprised at leaking all their data.
I never got how people trust companies like ddg or Brave. If you don't trust Google and Apple why would you trust a smaller company in the same jurisdiction. They will be forced to hand out all data as well regardless what they say.
It is a hilarious excuse for DDG to claim they are doing this for a favicon. Even if DDG is legitimately not using the data they are definitely collecting the data.
The problem with that is that it requires users to "trust" DDG, which is not how the world works today. If you are a company that collects info, and you expect users to trust that the info will remain safe, secure, and never get misuse that is downright foolish for anyone to believe a word of that.
We all know that DDG cannot claim it's impossible for them to get hacked and have all that data leak out. Hacks happen all the time and so the solution for DDG is to simply NOT collect the data, rather than collect and claim it's all secure.
And we all also know DDG has (or will) get a NSL (National Security Letter) from the NSA to secretly turn over the data anyway, and when that does happen the DDG employees are not even allowed to admit it ever happened.
It actually happens very often. People monitor their networks often, and pay close attention to such problems. Just search for “Apple phone home” and you’ll find many cases where people complain about Apple’s various services making worrying requests.
By now it has been sufficiently proven that it is physically impossible to even exist without sending surveillance data to someone on the internet. We should probably update the laws of thermodynamics to include that.
If DDG cannot fetch the favicon in different, reasonable way, then the question is whether or not the ability to display a favicon in search results is really worth it.
Since DDG owns the icons.duckduckgo.com service and the domain you were interested in is in the request to icons.duckduckgo.com, you've sent the domain to DDG's servers.
EDIT: I didn't notice that this topic was about the DDG browser (which I didn't know existed) and responded assuming this was about the site/extension. For a browser, yes, a client-side solution is possible and probably preferable. Please check and upvote other comment trees.
This makes sense to me and is not alarming. Getting favicons actually is difficult to do robustly; many applications and websites use Google's service to do so, which then leaks the request to Google: https://www.google.com/s2/favicons?domain=ycombinator.com
Putting this logic in the client is not feasible. You want to send requests directly to every shady site that shows up in your search results, load their pages in the background, work through network delays and HTTP errors, and parse out the location/format of the favicon files?
DuckDuckGo hosting this functionality themselves is also a positive. They have previously been burned when the Web of Trust service they were originally using was found to be farming data, and turned it off immediately once discovered. Processing, hosting, and serving the icon themselves prevents that from happening again.
"This is not to say that DDG is perfect: links you click do seem to be redirected through a /l/ page on their domain ..."
I am surprised the user is not complaining about this instead of the favicons. Their privacy policy goes on about the privacy implications of Referer headers and instead of calling out browsers for sending Referer by default, they instead give themselves power to record all the user's clicked results themselves. The Referer problem is something that can be solved by the user at the browser level through, e.g., using a client that does not send Referer, browser extensions/plug-ins that can control headers sent, or perhaps with a local proxy to remove the Referer header.
Unless DDG has changed, these prefixed result URLs are the default. It is possible to get unprefixed result URLs using the "lite" version of DDG however that is not the default. "Privacy-focused" search engine chooses less private default. News at 11.
I recently noticed that DDG has started redirecting queries submitted via POST to /lite/. The redirect is to the same domain. No explanation. I have a custom client that does not follow redirects and I now have to submit two sets of HTTP headers instead of one.
These guys are trying to make money from advertising just like everyone else. They have to be very particular in the methods they use to do it -- check the exceptions in their privacy policy -- but it is the same game. Ads and affiliate links. That sort of business and privacy are always going to be at odds with each other.
This is done only for browsers that don't support setting the referrer policy. https://caniuse.com/referrer-policy. It should not be happening with any modern browser, and (like the other reply) I don't see it in mine (Firefox).
Are you saying DDG is forcing you to send a User-Agent header?
DDG's privacy policy also goes on about the privacy implications of User-Agent headers combined with IP addresses.
So let's say I take what they have put in their privacy policy to heart and I stop sending a User-Agent header. In response DDG sends prefixed result URLs? WTF?
Using haproxy, for example, I can use a "modern browser" and send no User-Agent header. I still get prefixed result URLs.
You can disable this on the settings page. Frankly I'd infinitely rather send my request through a redirect served by DDG than send my referer to every site I visit. I think this is a perfectly sensible default for the average user, who both doesn't block / obscure their user agent and doesn't have referer masking software in place.
You're free to disagree, but in that case you can make use of the option to change the default that they provide for you.
To get prefixed result URLs, use GET not POST. Try duckduckgo.com/lite/ and lite.duckduckgo.com/lite/
If I recall correctly there is a way to turn off the prefixed result URLs without having to turn on Javascript, by adding/changing a URL parameter, e.g., kh. However that is not the default setting. Not very friendly toward the privacy-conscious user who has Javascript disabled and does not read HN to find out about otherwise undocumented URL parameter usage.
For certain queries, I can actually get a different first result based on the HTTP method I use.
I'm not sure what you mean by bringing up HTTP method. The two main ways I know to get to the results page are to enter a query into the website or the browser search bar, and both of those seem to issue GET requests. They also both produce real URLs without any intervention. What defaults are different for you than they are for me?
I don't see how this is a privacy violation in any way. The headline is that DDG is tracking the domains you visit, but favicons are loaded for every search result you see unconditional on whether you click them, and DDG already knows what those results are when they serve them to you.
Which other browser doesn't fetch favicons locally?
And I have no clue how search results come into this? Of course the browser isn't fetching favicons on search result pages, that's the servers job, but that's not what this is about.
It's really odd that this is the top comment right now, given that even the headline makes it clear its about the browser, not the search results.
For the search results DDG can do whatever they way. As far as I understand from the title, this is for the pages that you actually visit. For those pages it totally make sense for the browser to figure out favicon locally on its own.
As for complexity, I'm sure users of privacy oriented software would prefer to not have favicon in small percentage of corner cases rather than leaking browsing history.
> Putting this logic in the client is not feasible.
> You want to send requests directly to every shady site that shows up in your search results, load their pages in the background, work through network delays and HTTP errors, and parse out the location/format of the favicon files?
Looking at DuckDuckGo search results and visiting a page that you navigate to are two different things.
1. DuckDuckGo search results:
DDG already returns the search results so there's no privacy violation to return the favicon or the URL for the favicon in the list.
2. Any page that isn't a DDG search results page:
Use client side logic to locate the favicon. This means worse performance but better privacy - which aligns with DDG's goals.
If you want to optimise this then DDG could send the client some precalculated Bloom filters with info about known sites. The client could use these to try certain methods of favicon retrieval first.
I don't need the favicon for a domain I never visited just because it adds some visual cool to your UI. That is not a thing. Cache it whenever I end up visiting the page and be done with it.
> Putting this logic in the client is not feasible. You want to send requests directly to every shady site that shows up in your search results, load their pages in the background, work through network delays and HTTP errors, and parse out the location/format of the favicon files?
I don't see why not. Browsers are constantly dealing with these very issues. It's one of their core competencies.
I don't really trust DuckDuckGo, but I use their search service because I trust Google less... I still trust Firefox more for a browser although it won't take much at this point to make me switch.
If you do that, there's no evidence that Google is more privacy invading than DuckDuckGo, since you're left with taking their word for it. And frankly I trust a big, bureaucratic company more than I trust a startup. The ideal would be to trust technology (e.g. end to end encryption, open source) but that's not the case here.
Note that I also use DuckDuckGo in my Private Mode, as Firefox allows setting different search engine for Private Mode. I do that because it's better to compartmentalize your online personas, plus I keep my "web and app activity" in Google on, with deletion after 3 months.
Runnaroo is also a search engine that is Google based (for organic results). IMO it has an advantage over Startpage because of its integration of vertical search sources, like Stack Overflow [0]. I'm the creator.
Mojeek [1] is also a good privacy option that doesn't get talked about as much as DDG. They distinguish themselves by having their own search index.
What does Google have to gain from it? Google has pretty aggressive anti-scraping protections to protect against this exact behavior, so why would they allow Startpage to get away with it?
What does Startpage have to gain from it? Unlike DDG, they don't seem to have any core product, so they fully depend on Google's goodwill which is very shaky grounds when it comes to a long-term business.
> Unlike DDG, they don't seem to have any core product, so they fully depend on Google's goodwill which is very shaky grounds when it comes to a long-term business.
Isn't that exactly like DDG but Google instead of Bing?
My understanding is that DDG is using Bing results as well as their own. Whether their in-house results can stand on their own is another matter, but at least they're trying to reduce their dependency on Bing, where as Startpage is not.
> We also of course have more traditional links in the search results, which we also source from multiple partners, though most commonly from Bing (and none from Google).
So basically it's all Bing and I see no effort to reduce that dependency as you claim.
"You can’t beat Google when it comes to online search. So we’re paying them to use their brilliant search results in order to remove all trackers and logs."
I didn't know they had a browser. I'll have to give it a try. Can't be any worse than Google's browser. Or their OS. Or their video monopoly. Or their search monopoly. Or their secret partnerships with governments. Or their ad monopoly. Or their email monopoly.
Essentially a way to collect where people are visiting. I believe them that it’s anonymous, this valuable info wouldn’t need to be identifiable to be of value.
They should probably change the behavior to how it’s suggested in the thread, but I’m still going to use DDG over alternatives for the bang feature.
The favicon is acquired from DDG servers for the result you've just retrieved from DDG servers.
How is this leaking anything? What additional privacy would you gain from getting the favicons from the domains directly of search results delivered by DDG?
Everyone is missing the point here. Let me break this down as simple as I can:
1. End user does a DDG search for "food"
2. The "food" query returns a list of search results, these results have each have a link, DDG wants to display the favicon for each link.
3. To be clear, DDG does not store or log the IP address of the user doing the query. They do, however, know what was queried, so they know "somebody" somewhere searched for "food". They have to know this, they are a search engine after all.
4. Since DDG wants to show the favicon "privately", and they dont want to put that logic/work on the client side (which could leak your IP), so instead DDG finds the favicon internally.
5. A DDG server, completely separate from anything search-related is then tasked with finding the favicon for your "food" query results, lets say the #1 result is www.allrecipes.com, so a DDG server goes to www.allrecipes.com and finds the exact favicon location.
6. The "found" favicons are then stored in a cache, and displayed from the cache like this: https://external-content.duckduckgo.com/ip3/www.allrecipes.c... (and if no favicon is found in the local cache, you get a grey arrow by default)
7. I'd like to note, even with all this action, DDG doesn't know if you actually "visited" www.allrecipes.com, they simply know that some anonymous user did a search for "food", www.allrecipes.com was a search result, and a favicon was displayed. They dont know who searched for it because the users IP is not stored anywhere, they dont know if you visited www.allrecipes.com, they prevented you from leaking your IP to allrecipes.com since they didn't force the end user to load the favicon.
So whats the issue? What am I missing here?
PS: You know this works because after doing all these searches for food and seeing allrecipes.com (and even clicking allrecipes.com result in the DDG Mobile App or browser extension), guess what? allrecipes.com doesn't follow you around with re-targeting ads! Why? Because DDG prevented that from happening!
I’m new to this issue and happy to commit us to move to doing this locally in the browser and will have us move on that ASAP.
That said, I want to be clear that we did not and have not collected any personal information here. As other staff have referenced, our services are encrypted and throw away PII like IP addresses by design. However, I take the point that it is nevertheless safer to do it locally and so we will do that.