All these comments about metadata not being useful are missing the point. Metadata is incredible valuable and sometimes just as valuable as the decrypted data itself. Knowing what sites a target visits, access patterns, changes in behavior: all this can be fed into ML algorithms to come up with fingerprints.
You don't need to be able decrypt the data in transit if you know the endpoints and can somehow compromise the endpoints at a later date. And that is way easier. Breaking encryption is hard and time consuming. Identifying a site a user regularly visits and exploiting that is more straightforward.
It honestly worries me that this is the top comment on hacker news. Not because it is wrong (it isn't) but because of all places that website filled with tech workers and experts in the full software stack, full of people that work on and exploit meta data, it still needs to be discussed how important metadata is.
If we can't convince people with their ear to the ground, how does one convince the general public. Especially since it isn't intuitive how metadata is useful. Though the analogy I typically use is a private investigator following you around. Can't hear your conversations, but can see everyone you talk to, where, and for how long.
The people who visit this website are the people who are paid to create and administer all of this technology. They're not only the last people you would be able to convince of something that would affect their livelihoods, but even the ones who do understand feel like it is part of their duty to deceive the less technically adept about the capabilities and dangers of the technology that they're surrounded with.
The comfortable upper middle-class are the most conservative elements of any society; they're providing the management and expertise to implement any dystopia that's coming. Beneath them are the tradespeople and unskilled laborers who choose between working or starving, and above them are morons.
Nobody who has spent more than a moment thinking about it fails to understand the dangers of metadata, they just don't think they it will be a problem for them. Hence the most common response is something about how their lives are boring, and how they have nothing to to hide. "Who cares if I'm at Starbucks at 2 o'clock?" Technologists know full well what they could do with that information, that's what they're paid to know, and they're who are going to be doing it, or they're going to have to find another job.
> The people who visit this website are the people who are paid to create and administer all of this technology.
Exactly, some falsely assume all technologists somehow share an enthusiasm for morality. Many of the most successful technologists I know simply work for the highest pay from military/intelligence contracts.
The term "hacker" in "hacker news" is too misleading, especially those of us who use the more RMS-esque definition of it. Petition to change to something more appropos.
I personally agree that the more RMS-esque or Steven Levy definition of "hacker" is a good idea. However, we must acknowledge that "hackers" is not never a homogeneous group, and even more so today. I know some greyhat hackers who do brilliant technical works that rightfully entitie them "hacker", but they have questionable ethics. Normally I use the generic definition from RFC 1392 as the compromise.
> hacker: A person who delights in having an intimate understanding of the internal workings of a system, computers and computer networks in particular.
Using this definition, calling this site "Hacker News" is not exactly accurate, but also not too far-off. I think this website is 40% "Silicon Valley & Startup" News, 40% "Hacker" News, and 20% misc.
To add to that, even if the word "hacker" has a malicious connotation, today, the meaning of the word is closer to "the one who knows his/her stuff well and enjoys breaking them down and creating new things".
One of my favourite sites is the IKEAhackers.net (no affiliation). That site truly shows what a "hacker" does, in the furiture domain, but still, decomposing, redesigning, reusing.
Exactly like the RFC 1392 (aka "Internet Users' Glossary") as you mention. There is a distinction to the word "cracker" which shows malicious intent (what our dear friends on EF + NSA are doing).
This piece of news should also be a reminder to ALL, that these agencies that "protect" us (irrespective of flag) cannot and should not be trusted with/for anything. Especially not with the truth.
we should made labels by hashing publicly registered semi-precise definitions, and maybe add markers to indicate how closely what we mean fits said registered definition of the label.
(I don't actually think this is what people should do, but I do think it might be a cool expirament.)
Your comment implies that working for military/intelligence is automatically immoral. Nothing could be further from the truth. Believing that freedom is "free" is highly delusional. This civilization that we have, and enjoy, almost entirely depends on having more, better, bigger guns than the "bad guys" (in fact, having more, better, bigger guns is the best way to ensure you don't have to use them).
What you do with the guns is a different issue, but there's clearly many nations worldwide that focus on defense but not offense (maybe not US, but probably Switzerland).
> They're not only the last people you would be able to convince of something that would affect their livelihoods
Working in Silicon Valley, I would not agree with saying that engineers in general are involved in this for any immoral reasons. (One of the few exceptions is ad tracking experts, who dig like pigs for truffles through PII.) The reasons are not knowing history (ie. Crypto AG pwnage), and just a lack of intellectual curiosity.
I'd like to comment on the importance of metadata analysis.
This is not well-known, but before the British were able to decrypt German WW2 traffic, they used very detailed radio traffic metadata analysis to map everything they needed to know about ground troops. (The architect of that was given US citizenship after the war and built the US' system, but on a global scale.)
Details of that were classified long after the war in both countries. And it was just metadata.
I think that conservative means "in support of a (certain kind of) status quo" in this case, rather than the American line-up of hot-button culture war issues. The upper middle-class wants stability, whatever it is, because of the structure of their income stream. That's why upper middle class people are basically centrists and don't want to rock the boat too much: they're doing very well in the boat, and rocking it will have them taking on water. Other folks have much less to lose (which we should all remember over the next month).
Why do these points not apply to the upper class? I would expect them to be at least as conservative as the upper middle class, and all of these arguments fail to distinguish the groups. But the claim is that the upper-middle class is more conservative than the upper class. That's... weird?
Example: Upper middle class wants to limit immigration since it hurts their salaries. Upper class wants more immigration to get cheap labor into the country.
All the political terms - left, right, conservative, progressive, liberal, etc have definitions which vary greatly from country-to-country, over time, and by which group is using them. Clinton and Biden are representatives from American's dominant liberal party and represent views which are left-of-center when you do the sane thing and define center as "median voter in the country being discussed" and not "my group of friends" or whatever time or place you're imagining.
You might as well say 'Bill Gates is very poor, by any reasonable standard' (because unlike the rich people of 2,000 years ago, he can't raise an army that rivals that of his home nation-state or the rich of the future who take vacations on the moon).
I think the parent means that both Clinton and Biden are establishment candidates. Neither of the are boat rockers like Bernie or Trump in 2016. I think the reality of Trump’s presidency is far more establishment than his original campaign but still.
Liberal & Conservative in the US usually have to do with what changes a person wants to make but Clinton, Romney, Obama, Biden, hell Bush are all literally conservative in the scope and amount.
I wish you would not refer to arguments that you disagree with as tropes, it's a thought terminating cliché. Clinton ran on "America is already great" and Biden is running on a "return to normality" ticket under which "nothing will fundamentally change." The Clintons were the center of the DLC, who assassinated the candidacy of Jessie Jackson to bring on an era in which the Democrats would "End Welfare As We Know It," sign the crime bill, and deregulate everything. Biden got into politics as a New England Dixiecrat, of the kind that was extremely popular around Boston when white children there were threatened with being exposed to black children in school. His VP is a prosecutor who bragged about jailing the parents of truant children. Both supported the Iraq War. Biden championed the crime bill that was passed under a Clinton administration.
Yes, they wouldn't be conservative for Saudi Arabia, but they would both continue to closely ally us with the Saudis and aid them in murdering Yemenis.
> center as "median voter in the country being discussed
Both Biden and Clinton, issue by issue, are well to the right of the median US citizen. It's pretty dishonest to restrict the people allowed to have their opinions considered to the people who thought that the distinction between Republican and Democratic administrations was important, when the argument being made is about whether both candidates are conservative. The median eligible voter is barely more likely to vote than not.
I disagree with your claim that Biden and Clinton are "well to the right of the median US citizen". They are clearly to the left in literally every assessment - whether in the actual elections or polling done by any reputable organization. You're imagining that "median US citizen" is somehow far left of "median US voter" and that's just not true.
Additionally, everything you listed is just exposition on your initial claim. Saying that you will "End Welfare As We Know It" sounds like a potentially wildly liberal plan - perhaps UBI, government-guaranteed employment, housing and healthcare or some other fundamental shift. The victims of crime are disproportionately the poor, people of color and people who are structurally disadvantaged so removing the threat of violent crime from their lives falls well within the standard goals of liberals (even if the actual implementation of the bill you're referencing had more mixed results).
I agree with the general sentiment, but Trump is a very poor yardstick for conservative political beliefs. Out of Reagan, both Bushes, McCain, Romney, and Trump, Trump is the clear outlier.
You're right that Trump is not necessarily representative of other Republicans. But I think the skew is actually the opposite direction of what you're saying. I think Trump is actually less likely to be favored by wealthy people vs say Bush.
Look at this polling of support of Bush vs Kerry by income level[1], as income rises support for Bush almost always rises, and support for Kerry almost always decreases.
I didn't mean to suggest anything about the directionality of the skew, just to mention he's not a classical conservative and there's likely some skew between supporting Trump and having conservative political leanings.
(On a side note, there's some smaller skew between voting Republican and having conservative political leanings. I'm conservative in fiscal policy, foreign policy, and favoring action at a local and state level, but liberal regarding most social policies, criminal justice reform, and environmental regulation. I think government enforced price transparency plus a German-style universal healthcare system via private insurance decoupled from employment is preferable to either a US model or a Canadian/UK-style single payer system. I've always registered as a Democrat.)
Your behavior is characteristic of a terrorist/pedophile/drug dealer. An automated computer system/bureaucracy, the workings of which are too complex for a human to intuit or critique, decides on the basis of this "fingerprint" that you should be dealt with. You are bombed by a drone / disparaged in authoritative media / shot by police along with your kids and dogs / have your possessions taken by force / have your life ruined for a few months. Nobody at any step of this process is individually culpable, and nobody can identify with any certainty the actions which initiated this process.
That seems more than just the dangers of metadata. That's more of the dangers of giving machines the authority to drop bombs with no human oversight at all. That same kind of problem could happen if the government wasn't spying on anything, or if the government was spying on content, not just metadata.
Humans follow orders which are given by humans on the basis of data which is analyzed by machines and interpreted by humans.
If the machine says "dude is terrorist based on XYZ" and the human cannot realistically verify all of that is factually correct (perhaps the subject's phone was lost as the subject walked by a mosque?), then it is much easier for the human to say "Data says this dude is terrorist" than it is to say "Data says this dude is terrorist, but the data is probably wrong and we shouldn't..."
The existence of the data itself is a threat against every subject the data includes, at a minimum.
I believe the core problem there is still making extreme decisions without proper evidence. This could happen if the government knows much less about you (e.g. just the info on your driver's license) or much more about you. That is, the problem in these specific examples is not the existence of the data, but rather the willingness to throw caution to the wind and operating on shaky foundations.
Human's have a threshold where their confidence in the accuracy of something will determine their willingness to participate or take action. The machines/algorithms/authority structures and so forth are in place in large part to provide that confidence.
The issue today is that the leadership (in many areas of life from business to military to government), who make the decision to kill/censor/interrupt business/etc or not, are saying "we have to follow the data" without having any understanding of what that really means.
Ultimately, this creates false confidence both in the decision-maker and those that are following their lead. I find it unlikely that there would be anywhere near the same willingness if the 'intelligence' many of these decisions were based on didn't seem as rich and unmistakably correct as it often does.
Of course, the practical effect here is that leadership gets to blame the algorithm/model/data instead of having to accept the blame themselves. If only those pesky engineers and nerds in the lab were better at the job we'd bomb less foreigners.
As is being talked about elsewhere the social and interest graphs that can be generated are the most important aspects.
I'll give you an analogy that might help. Let's say that a personal investigator is following you. They have a GPS tracker on you. They can see where you go, who you talk to, for how long, what you buy, etc. The only thing is that they don't know what you are talking to people about or exactly what you buy (but they know where you bought it from). Would you feel comfortable with this person following you around?
I'm assuming not, because I don't know anyone that has answered yes. It feels like an invasion of your personal space, right? They can still learn a lot about you and your habits by doing this, right? But all they've gathered is metadata on you. So why do you feel uncomfortable?
It's not that important what you talk about and what are you doing somewhere for who those systems are used. Just knowing who talk with, and where you're is extremely helpful to law-enforcement to triage from millions of people in the country, to something closer on the order of thousands of people of interest.
Goal of metadata investigation isn't to directly target you, most of the time. It's to put you in the bucket of interesting people, that government will pay attention to.
It's exactly the same as ads on the internet - they maybe classifying you as a person potentially interested in computer security because you're visiting tech crunch. Are all people visiting it interested in computer security? Of course not. But you're many orders of magnitude more likely to be interested in it, than a random person from the internet.
> Even by that account, the scale of collection brought to mind an evocative phrase from legal scholar Paul Ohm. Any information in sufficient volume, he wrote, amounted to a “database of ruin.” It held personal secrets that “if revealed, would cause more than embarrassment or shame; it would lead to serious, concrete, devastating harm.” Nearly anyone in the developed world, he wrote, “can be linked to at least one fact in a computer database that an adversary could use for blackmail, discrimination, harassment, or financial or identity theft.” Revelations of “past conduct, health, or family shame,” for example, could cost a person their marriage, career, legal residence, or physical safety.
> Mere creation of such a database, especially in secret, profoundly changed the balance of power between government and governed. This was the Dark Mirror embodied, one side of the glass transparent and the other blacked out. If the power implications do not seem convincing, try inverting the relationship in your mind: What if a small group of citizens had secret access to the telephone logs and social networks of government officials? How might that privileged knowledge affect their power to shape events? How might their interactions change if they possessed the means to humiliate and destroy the careers of the persons in power? Capability matters, always, regardless of whether it is used. An unfired gun is no less lethal before it is drawn. And in fact, in history, capabilities do not go unused in the long term. Chekhov’s famous admonition to playwrights is apt not only in drama, but in the lived experience of humankind. The gun on display in the first act—nuclear warheads, weaponized disease, Orwellian cameras tracking faces on every street—must be fired in the last. The latent power of new inventions, no matter how repellent at first, does not lie forever dormant in government armories.
take a look at the history of the behavior of intelligence services through the 20th century and ask yourself how comfortable you are with this power being wielded by anybody.
Chekovs gun is a tool for better stories. It does not work in reality, because in the real world it is always a cost to doing things. And extreme action has extreme costs, so very seldom does any country do anything extreme. As an example, nobody has used a nuke in war since ww2.
What most people don't understand is that a targeted IP, with a stream size and a timestamp is enough to identify pretty every https page uniquely if it is accessible by a spider.
A headless chrome makes measurements of timings even easier these days. The order of how files are loaded, which file size e.g. jquery.123.min.js has, and where and when exactly in which order it is loaded from is very unique among all pages of a website.
I think that's more complicated than you realize. That list would be impossibly large to scrape and search, not to mention collisions and dynamic content problems.
>The order of how files are loaded
It's a good idea, but you don't know what files I have in my cache and when they expire, or what files my extensions are blocking. This'd only work in an ideal-case scenario.
An analogy I like is that they know you called a suicide hotline from a tall bridge in the middle of the night, but they don't know what you discussed.
I believe I read the same example in 2600 quarterly some years back, in which the metadata was described and scrutinized in greater detail. It was impactful enough at that time for me to recall even today.
In the lead-up to Australia's metadata retention legislation, some org posted the following examples. I found them reasonably effective at convincing normal people, but we failed to convince the decision-makers in Parliament:
- "What if your call logs indicated a 45-minute call to a suicide hotline made from a bridge. Do they need to hear exactly what was said?"
- "What if your call logs showed you receiving a call from a sexual health clinic, and that you then called a bunch of people in rapid succession. Do they need to hear exactly what was said?"
I think the lynchpin is not knowing. We don't know what all the data is used for or why it's taken. As long as we don't know we lack the ability to judge. We don't have the power to make a strong rhetorical argument without shared knowledge.
Colbert's white house correspondent's address covered it. Politicians don't talk about super depressing stuff like guantonomo bay and journalists have the courtesy to not try and find out.. We don't talk about data collection in any serious manner.
I think the weakness of data collection is when it gets in the average joe's way and it hasn't done that yet in a big way. The more we hammer home that it must facilitate movement through life and not hinder it, the better the middle ground will be.. maybe.
There is a requirement that both sides get really good at eviscerating lies and liars. Neither side of the fence wants fake data or betrayal.
> If we can't convince people with their ear to the ground, how does one convince the general public.
Convince them of what? Some of us don't believe the NSA are bad actors and and possibly we also believe they're doing their jobs and support them in that.
"The NSA" - who do you mean here? The org in its official function doing unofficial things without oversight? Or the individual working for the NSA spying on his ex-lover for blackmail material?
I mean, either you're saying "no one within the NSA has ever been a bad actor", or you're saying "the bad actions are acceptable collateral damage; no oversight needs to be applied to ensure the trade off between effectiveness and collateral damage is balanced", or you're saying "not ALL actors are bad" and leaving it at that.
And...none of those strikes me as a particular defensible position to take.
You're one of the people this [0] comment is talking about.
Also, how can you possibly believe that the NSA are not bad actors? Between trying to hobble encryption, spying on everything, and enabling bad individual actions, and having a horrible success rate [1], what is left to defend?
NSA's competence or success rate doesn't invalidate the need for such an organization. Individuals part of the organization that behave badly don't either. Other states have organizations like the NSA and in order for the USA to defend itself from them the USA also needs one. We don't question the need for a military because one lieutenant burns down a Vietnamese village, we demand justice and changes, but we continue to support the need for a military. That's my thought around the NSA. I support private companies protecting their customers by utilizing encryption and I support the NSA to do whatever they can to amass all the information they can when private companies fail. Why? Because our enemies are doing the same thing. A moral position that loses doesn't last, and for sure there are some moral positions I would stand by them even if it meant losing, but this one isn't one of them. Not for me anyway.
This comment doesn't stand up to any kind of scrutiny whatsoever.
One, handwaving institutional corrupt and violation of the constitution by claiming it's only a few "bad actors" ignores how high up that corruption emenates from. Two, it assumes that the violations are needed for national security, a claim which can easily be dissected by understanding what William Binney has told us about thinthread, just as an singular example in a vast sea of examples. Three, it's a strawman to jump to arguing that because other countries are doing this, we need the NSA too. Very few people are actually calling for the dismembership of the NSA, and in general want accountability and a return to constitutional surveillance. Four, implying the constitution is a "moral position that loses" is absolutely a machievellian, realpolitik, ends-justify-the-means policy position that we and the world have suffered enough consequences and blowback of.
Your entire argument revolves around using the safety as justification for violations of their mandate and oaths, when all the evidence points towards the truth being quite the opposite: the surveillance program has failed to be effective for safety, and not only that, that failure is largely due to this very kind of thinking in the first place! By being willing to undermine the constitution the NSA (et al intel agencies) inherently reduce long term security and safety in the US by allowing bad actors in all kinds of sectors the ability to abuse the data they get.
The totalitarian surveillance system is about control, not safety, always remember that!
"Go again and see not just the film and the play but read the text of Robert Bolt's wonderful play "Man For All Seasons", some of you must have seen it - where Sir Thomas Moore decides that he would rather die than lie or betray his faith and at one moment Moore is arguing with a particularly vicious witch-hunting prosecutor (a servant of the king and a hungry and ambitious man), and Moore says to this man "You'd break the law to punish the Devil, wouldn't you?" And the prosecutor, the witch hunter, says "Break it?" He said "I'd cut down every law in England if I could do that, if I could capture him." And Moore says "Yes you would wouldn't you? And then when you corner the Devil and the Devil turned round to meet you, where would you run for protection? All the laws of England having been cut down and flattened, who would protect you then?" - Christopher Hitchens
Just chiming in here: it’s almost all about the graph. If you have the graph, the content is almost irrelevant.
This is why Signal hiding the graph as best they can, using SGX, is incredibly important work. Say what you want about Secure Enclaves, we know of no better way to conceal social graphs.
Yes there is still potentially some metadata analysis that can be done at the server to coordinate IP addresses but we know signal doesn’t keep those logs because of their response to the sealed subpoena (which they successfully sued with the ACLU to unseal):
>...we know signal doesn’t keep those logs because of their response to the sealed subpoena ...
That doesn't prove that. If Signal was, say, a NSA project they would have to respond to such things in that way to protect the signal intelligence value of the metadata they were collecting for their primary mission.
After Crypto AG we know it is a bad idea to trust any particular entity. Something like Signal can only be trusted as much as can verified.
If you are not trusting the people that are running these things, then Signal is just another siloed messenger where the servers are controlled by a single entity. There are certainly worse but Signal is not special.
Signal has open clients with reproducible builds. We know that they are keeping their promises wrt what information is communicated with the backends. That's a step above the other options in common use, and in fact does make Signal special.
> Signal has open clients with reproducible builds.
Not really. First of all, there is only one Signal client allowed to connect to Signal’s servers. And in the real world, the vast majority of Signal uses are getting their APK for that app from the Google Play store (the Signal team has said that they prefer you to use the Play store as well, instead of direct-downloading an APK from their website which they offer only grudgingly). That means that a state-level actor could possibly carry out a targeted attack to replace the Signal app on a given person's phone with a malicious build.
Also, Signal’s reproducible build system requires a specific version of the Android development kit. It has been pointed out that a state-level actor could be sitting on vulnerabilities in that, and not in the Signal source code itself.
Both these attacks indicate a problem that doesn't have anything to do with using Signal. If the actor can replace apps on your specific phone, then you're pretty fucked no matter what app you use.
If the attack is on the android dev kit, but not on signal, then.. the attack isn't on Signal, it's on the dev kit. Unless Signal's using an unusual version of the dev kit, your risk exposure to this attack is equal to any other app that you would use instead of Signal.
> That means that a state-level actor could possibly carry out a targeted attack to replace the Signal app on a given person's phone with a malicious build.
No, they couldn't. They would need the Signal developers' key. Android requires app updates to be signed with the same key as the original app.
A state-level actor can get the Signal key either covertly or by simply marching into the Signal offices with either a warrant or (if that fails) guns. Now, whether that will actually happen is a secondary issue -- but I submit that you have a mistaken conception of what a "state-level actor" means in a threat model. The fact that Google Inc doesn't hold the necessary keys but Signal LLC does is not a meaningful distinction to a state-level actor.
That isn't to say "all crypto is hopeless", simply that you shouldn't consider Signal to be state-level actor proof.
Do you personally do that with every release (which happens every few weeks or so)? Do you know anyone who does that who is trustworthy? If not, it's a fairly useless form of protection.
That's all a smoke screen. Nobody is running an open client with a reproducible build, everybody is running whatever version is downloaded from their app store of choice.
Have you personally done that? Do you know of anyone who is doing that and publicly tracks said verification? It doesn't matter how trivial it would be to verify if nobody is actually doing the verification (not to mention you'd actually want many people doing it and publicly posting their verification, as well as you checking that your hash matches everyone else's before installing the APK -- and there is no automated setup for doing that on Android.)
I don't think any significant number of people do it. I don't use Signal specifically, but I don't even know that there is a way for me to actually do it and then track whether that matches the version the iOS app store loaded on my phone, at least not without jailbreaking the phone.
In contrast to Signal, Telegram doesn't end-to-end-encrypt messages by default (they get stored in plaintext on their servers), it also doesn't protect the social graph and even stores your contact list on their servers. Even WhatsApp is more secure than Telegram.
They are more or less the same thing if you are not trusting anyone. Both require you have to verify the key fingerprint for a particular contact (safety numbers) if you want effective end to end encryption. Both are silos. Telegram is better about distribution and can be gotten from places you might trust better (e.g. F-droid, Debian). Both have some sort of reproducible build thing going on. Both could get access to your connections to other users if they wanted where Signal also insists on access to everyone's phone number. Telegram works on desktop without also insisting you have the program running on your phone.
> Say what you want about Secure Enclaves, we know of no better way to conceal social graphs.
I'm not following. Secure Enclaves have nothing to do with protecting the social graph of Signal users. They're used to store the contact list (and other things) in the "cloud" in a safe way – things that weren't even shared / stored anywhere by Signal before Secure Value Recovery was introduced.
> Metadata is incredible valuable and sometimes just as valuable as the decrypted data itself.
Just adding an example for the people who don't see the value of metadata: WhatsApp is still a viable revenue source for Facebook even as they have no access to the text of the messages due to E2EE.
Knowing who talks to who, at what times, the type and approximate size of messages, the members of groups, and the contents of the phone book of every user gives enough information to keep their business model without exposing them to court orders asking for the plaintext (that's the reason they added E2EE to start with, there is no incentive to improve the service when they have a billion heads of cattle to milk).
> WhatsApp is still a viable revenue source for Facebook even as they have no access to the text of the messages due to E2EE.
A friendly reminder to everyone that even if the encryption that is used to send the messages in WhatsApp seems to be solid they upload your entire chat history as unencrypted dumps to the cloud.
Even if you turn it off your chats will still end up there as long as whoever you are chatting with doesn't also disable this.
I suppose the AFAIK (I do not use WhatsApp), it's Google backup services on Android. WhatsApp stores the local chat history unencrypted in the device and does not mark it as "do not backup", so the cloud sync service uploads it to the backup service. And Android does not encrypt this information.
For contrast, Signal does encrypt the local history and the backups (to the point that is a bit harder to backup the chat to outside in Android, you need to copy a randomly generate password manually to restore it. But it's a safe approach).
Additionally, WhatsApp heavily encourages storing backups in Google Drive as well, with semi-regular popups asking users to configure backups in GDrive and to set the backup interval, if not already done. Obviously this doesn't mean Facebook has the information and the straightforward interpretation is that it's the least involved way of creating backups without sending it all to facebook
My point is mostly aimed at people claiming WhatsApp is somehow very safe just because of the end-to-end encryption.
I'm saying end-to-end encryption is a really great idea and everyone should do that and still encouraging people to look beyond that and think about the entire threat model when deciding what is important for them.
> they have no access to the text of the messages due to E2EE.
Correction: they might not have access to the message text. It's entirely possible (if not plausible: FB doesn't exactly have a good track record) for FB to just self-MitM the E2EE and see everything that passes through their servers.
From their site:
> The verification process is optional for end-to-end encrypted chats, and only used to confirm that the messages and calls you send are end-to-end encrypted.
Even this process--which I'm sure very few people do--is fallible given the lack of authenticity: there's no way to confirm that the given keys are what's actually used for encryption.
Yes, this may come across as very "tinfoil-hat-y," but do you really trust FB to not be exploring every possible avenue to increase their data streams?
> It's entirely possible (if not plausible: FB doesn't exactly have a good track record) for FB to just self-MitM the E2EE and see everything that passes through their servers.
Why would they even need to MitM in transit when they control the endpoints? They can just analyze the raw text locally (in the app) and extract valuable information.
Knowing who talks to who, at what times, the type and approximate size of messages, the members of groups, and the contents of the phone book of every user gives enough information to keep their business model without exposing them to court orders asking for the plaintext
Similarly Google runs 8.8.8.8 so they know what services you use that aren’t HTTP that they don’t have bugged already.
> According to a former drone operator for the military’s Joint Special Operations Command (JSOC) who also worked with the NSA, the agency often identifies targets based on controversial metadata analysis and cell-phone tracking technologies. Rather than confirming a target’s identity with operatives or informants on the ground, the CIA or the U.S. military then orders a strike based on the activity and location of the mobile phone a person is believed to be using.
That's actually pretty good policing. They should apply that domestically for non-terrorist violent criminals. +1 they have my vote. (Obviously not on the "drop random bombs on them" part.)
In general, entities like the NSA need to treat metadata as important because that is often all they have. That is because most everything is encrypted these days. The NSA has known about the "going dark" problem for a long time now and this is the reaction.
So this situation can be considered a sort of a triumph. For most people metadata is no real threat to them. Generally it is already publicly known who your friends and family are and those are the people most interact with online. It is mostly valuable that no one else know what those interactions are even if they know when they occurred.
For the important instance of businesses the situation is much the same although sometimes there might be value in traffic analysis for larger businesses that have enough traffic to analyze.
I can't believe that anyone that was around here during the snowden stuff hitting the fan would even remotely say metadata isn't useful.
"Law enforcement agencies have claimed that metadata helps to eliminate suspects by revealing their networks and contacts. But there is no information regarding the use of metadata by government bodies that are not officially enforcement agencies within the meaning of the data retention laws."
Forget ML; just a queryable database where your analysts can plug in a known surveillance target and see who they're talking to has lots of value. (IIUC, that's Palantir's original core product, not anything in the ML space.)
Both you and these commenters are missing the point. They're not just collecting metadata. We know from the Snowden leaks that the NSA was able to decrypt most https traffic as well as most SSH and VPN traffic around 2013. Although protocol security has been beefed up a bit and many bugs have been weeded out since then, it's still naive to assume they've lost this capability.
> In the mid-1990s, the NSA had found out that somewhere under Copenhagen there was a backbone cable containing phone calls, e-mails and text messages from and to countries like China and Russia, which was of great interest for the Americans. Tapping that cable, however, was almost impossible without the help of the Danes, so the NSA asked the FE for access to the cable.
This was revealed in details about 20 - 30 years ago in the Danish publication "Månedsbladet Press". I remember they had an insert, looked like a small newspaper, containing all the details about this. I remember this because this was the first time I heard the term "echelon". I my memory serves me correct I think that insert in the magazine was the first story or source of Echelon operating outside of the US so there is some kind of historical significance to this story.
If I recall correctly they did this wiretapping in a underground bunker in the middle of Copenhagen. The bunker had no toilet and they got spotted by the journalists that wonder why more people was coming out of that bunker everyday to go the bathroom than people going in. Something like that.
I don't have the magazine anymore but I know that the main library in Copenhagen has a copy as I checked several years later.
If any historians around here want the backstory behind all this you should contact the library and get a copy of that publication. The publication stopped long ago, but that was the only newspaper-like insert they every had so it should be easy to locate in the archives.
This is really fascinating. I dug around a bit to see if I could find a digital copy of the article you mentioned. The Norwegian National Library have an impressive archive of scanned books and newspapers going back hundreds of years, but apparently no danish newspapers. Instead I found a reference to a book from 1996 by an author from New Zealand which seems to describe the same topic in great detail.
This is basically the very early, pre-internett version of what Snowden revealed. The book is made available for free by the author: http://www.nickyhager.info/Secret_Power.pdf
The summary is quite something: «In the greatest surveillance effort ever established, the US National Security Agency (NSA) has created a global spy system, codename ECHELON, which captures and analyzes virtually every phone call, fax, email and telex message sent anywhere in the world.»
They already abuse this, I would even go as far and say that huge portion of data collection is just for industrial espionage to prop up government backed companies.
If you are developing a novel algorithm or even a different approach to AI then you should absolutely setup an offline work office otherwise either USA or China will take your work and give it to a government backed company. It also doesn’t matter if your company is located in the USA or if you are an American citizen, so long as you don’t have governmental connections, you have to be 100% more careful as game is pretty much rigged.
As I have pointed out in a comment some time ago [0], there have been multiple instances where US (ab)used their intelligence for economic gains - be it getting a US company to file a patent (that the NSA "obtained") before the German company who actually came up with the idea does, or subverting Airbus so contracts go to Boeing.
This doesn't even need to be a specifically anti-US thing. I think countries should generally not trust other countries that don't have a strong and nearly binding interest in their stability, security, prosperity, etc. In a close multinational union like the EU, one could argue that this is currently the case, or perhaps with closely-tied countries like Australia and New Zealand. It becomes harder to justify as you move down the line, even if the US is far from an enemy of the aforementioned places. Consequently, in the other direction I don't think the US should necessarily trust Germany with sensitive data, or Switzerland should trust Canada, etc.
I don't think people are that naive. The reason for that arrangement is simply that it is only necessary for the NSA to collaborate with one Danish intelligence agency (foreign intelligence, which is probably forbidden from spying on Danes), rather than two (foreign and domestic).
In a larger sense, the arrangement might, to a degree, defeat the purpose of having separate foreign and domestic intelligence agencies, but that is a more abstract notion. I guess we'll see what the Danish public will think or do about.
Are you sure? I find it very convincing that NSA would not betray the trust, as it also says in the article.
> "I can not at all imagine in my imagination that the NSA would betray that trust. I consider it completely and utterly unlikely. If the NSA had a desire to obtain information about Danish citizens or companies, the United States would simply turn to [the domestic security service] PET, which would then provide the necessary legal basis."
I definitely think it would make more sense to give up on using the system on the citizens of a single small country, in exchange for not risking the extensive access to citizens of many other countries, including China and Russia.
If they want to investigate danes, they still have all their other methods at disposal.
A statement is not a scientific fact. The head of the NSA lied to the American public about their extra-legal monitoring. What has changed since the Snowden revelations? Why would Danes be more respected by the NSA than US citizens themselves? This makes absolutely no logical sense.
To your point, the IC trades in misinformation and distraction as much as it trades in truth and fact. When any of those acronyms make a statement it's fitting to wonder if there's a broader arc, a bigger picture.
Their responsibilities and mission are clear. There are no style points. They'll do whatever it takes to accomplish that mission. History is very clear about this M.O.
An interesting thing is no one is willing to say "lies" anymore. It's always "misinformation" because it gives the perpetrator a way out to claim it wasn't intentional. The NSA and CIA do LIE.
This has nothing to do with scientific fact? It's about risk/reward for NSA. If Denmark finds out that they were deliberately using the system on Danish citizens, the NSA risks losing access to the system. Not taking that risk in exchange for having to use other methods to investigate Danes sounds like a perfectly reasonable risk/reward calculation from the NSA's point of view, especially considering how small Denmark is.
I mean, sure, they can lie about it, but that always carries a risk of being found out. Especially considering it sounds like from the article that FE can see who the NSA is searching for (and even has to approve the search queries).
While I find the above quote hilarious, you're probably right since the article also hints that they have a similar setup in Germany and most likely also in Sweden, Norway and many other surrounding countries. And since most of their comms probably also go through those, they'll just eavesdrop on the next hop and bypass all that trust and save themselves any potential political repercussions.
German BND did nothing to verify any selectors asked by the NSA and just ran them without any safeguards. During the investigation into those practices it turned out the NSA used that trust to run over forty thousand selectors that are in violation of german interests. From what can be cobbled together from media reports that includes german industry, german government, other european governments, the EU itself, multiple european defense industries, every single foreign embassy in Berlin, delegates to the UN, NGOs like Human Rights Watch, and multiple Universities, among others. More then two thirds of all targets were friendlies within EU or NATO.
We joke that "german secret service" is the service that tells the usa about german secrets.
Are you looking for more oversight than is expressed in the article? It says the Danes check to make sure what the NSA searches for on that system does not include Danish citizen identifiers.
The data used by XKEYSCORE is stored on multiple servers, including the NSA's long term storage servers. How could the Danes be certain the NSA wasn't conducting searches from some cache they do not have access to?
At the end of the day, this will rely on trusting the NSA despite their dubious history.
Part of the Snowden revelations was that the German BND was also granting this access. The NSA sent so many keywords that the BND basically stopped checking, and looking at it in an investigative committee of the german parliament they found a massive amount of clear domestic espiomlnage (keywords like 'Siemens' or product/technology names). Basically the BND was too incompetent to realise they were enabling US espionage on their own companies. Insane but true. You could and should blame the BND but honestly what kind of partner is the NSA if you can't even trust them on such basic level?
Even more surprising that DK hasn't been paying attention to any of that apparently...
The very concept that people should have their basic human right to privacy protected or denied based on the happenstance of their location of birth is insanity.
Governments that gate human rights on nationality are fundamentally amoral, and should never be trusted to self-regulate.
"Intelligence is probably the least understood and the most misrepresented of the professions. One reason for this was well expressed by President Kennedy when, on November 18, 1961, he came out to inaugurate the new CIA Headquarters Building and to say good-bye to me as Director. He then remarked: 'Your successes are unheralded, your failures are trumpeted.' For obviously you cannot tell of operatives that go along well. Those that go badly generally speak for themselves.
The President then added a word of encouragement to the several thousand men and women of CIA:
'...but I am sure you realize how important is your work, how essential it is - and in the long sweep of history how significant your efforts will be judged. So I do want to express my appreciation to you now, and I am confident that in the future you will continue to merit the appreciation of your country, as you have in the past.'
It is hardly reasonable to expect proper understanding and support for intelligence work in this country if it is only the insiders, a few people within the executive and legislative branches, who know anything whatever about the CIA. Others continue to draw their knowledge from the so-called inside stories by writers who have never been on the inside."
One gets the impression his claim that all the CIA's successes are secret is a lie. Because there never were any. You would thing 50 years after the man kicked it that at least something would be come out. Not really surprising. If you look at how successful organizations are structured the CIA is not that.
What won the cold war was the West's consumer industrial base. Not the military industrial complex, and certainly not the incompetent people at the CIA with their cunning plans.
> In recent years, the NSA and the German BND have also been accused of massive illegal domestic spying. Thorough investigations have shown that was not the case, although their employees were sometimes careless and it was technically not always possible to do what was legally required.
Absurd statement from the article. What investigations? Internal from the NSA? It's like saying "No the NSA are not collecting data on Americans because Mr. Clapper said so to the Senate under oath."
So this comes two months after the head of Danish Defense Intelligence Service was relieved of duty for withholding information from watchdogs and violating Danish law [0].
I honestly don't know who should be blamed if there's not enough political capital to reign in these agencies globally. Either the electorate accept it, or don't know about it, which is bad enough in itself.
You can not share this link via Facebook Messenger.
Tried it and got a "You can't share this link. Your link couldn't be shared because it goes against our Community Standards".
Welcome to six years ago, where Americans where shocked that the NSA had spied on US citizens but did not give a shit about the spying on regular innocent people in other countries around the world.
As a Dane, I hope that the politicians a going to make it crystal clear the the FE is suppose to collect intel to protect the country, but not at any cost. Flat out lying and keeping secrets from the people who are tasked with the civilian oversight of the FE should be punished with prison time to make it clear, at all levels, that this will not be tolerated.
I don’t have much faith in any serious reprocaution though. At least two ministers of justice have told telcos to continue provide mass survilance to PET ( Internal police intelligence ). It’s clearly against EU law and EU courts have made rulings telling the Danish government to stop. They just don’t care. The police won’t even say how often it’s used. The general consensus seems to be basically NEVER, and it’s never the main intelligence source, still they refuse to stop.
>I hope that the politicians a going to make it crystal clear the the FE is suppose to collect intel to protect the country, but not at any cost.
Fat chance. Nasar Khader (conservative MP) is attacking TET (the body that observes danish intelligence orgs) on his twitter, and first whiff of any kind of top down investigation, launched Claus Hjort (former defence minister) in a series of attacks on the current minister for revealing intelligence secrets and harming the danish-US relations.
No politicians wants to deal with this, and some even seem to want unbridled spying to keep going.
Any politician that have had a part of this want it to stay in the dark. They know that it's explicitly forbidden in "Grundloven" (the constitution in Denmark), except if some law enables the authorities to do it. There has never been such a law allowing them to spy on all danish citizens, and if there was, we'd know about it.
As someone living in Denmark I wonder how do they classify a Danish citizen? Someone living in Denmark, having permanent residence, living in Denmark and being EU national, holding passport or being born here?
Also, from a fair amount of conversation with my Danish colleagues I find it odd how uncritical they are when describing the relationship of Denmark and USA, regardless if I myself find the specific view different than mine. The people in question are of the right political spectrum and not overly political but still.
If all we have to go off of is the slides and rumors, it's still not super clear exactly how it works. Even for HN users, which are largely engineers, it can be confusing. If you're a wikipedia editor, probably more so.
Agreed, though the big thing missing from the Wikipedia article is the root data collection method - cable tapping - which makes all the difference in explaining an ubiquitous surveillance system.
In the 90s I remember critical media linking Sandagergård to the Echelon Project. There were also rumors about foreign contractors operating at the facility.
I've always thought about this when I was near Sandagergård (it's situated in a beautiful place off the beaten path not far from Copenhagen) but as far as I remember the stories where mostly based on speculation without any hard facts. With these recent disclosures it seems that there might have been some truth to those old stories.
> it seems that there might have been some truth to those old stories
It's really weird to see confirmed, year after year, that all those "conspiracy theories" (first circulated on the '90s internet) turned out to be almost entirely true, when it comes to network-based espionage.
Echelon/five eyes? Check.
Massive network surveillance? Check.
Compromised ciphers? Check.
Compromised hardware manufacturers? Check.
NSA being fundamentally devoid of oversight? Check.
US espionage targeting allies for industrial gain? Check, check, aand check.
This shit was ridiculed back then, and it makes so hard to discern what is true in contemporary news reporting.
As someone who has heard my father re-tell the same story at least 20 times of the time back in the 90's when he was out walking his dog near Sandagergård and was stopped at a military checkpoint by soldiers talking American English to him this almost feels like déjà vu. Anyone who used the area knows Americans was there.
>The novelty of XKEYSCORE is that it enables analysts to find exactly those anonymous communications. For that purpose it reassembles IP packets into their original format ("sessionizing"), like Word documents, spreadsheets, chat messages, etc.
This sounds interesting. Does it mean they write a sort of serialization/deserialization routine for whatever format they can grab at hand? For example maybe it's possible to assemble a piece of binary data travelled through the Internet to its original, Borland database file format?
Many of our extractors are stateful - ie. They need to see previous packets in the session to extract keys, state, etc.
This is done by having front ends which direct raw packets to backends based on which session they are a part of - the obvious one being the tcp 4-tuple.
The backends then don't each have many sessions to look after - perhaps tens of thousands each. That means they can keep many kilobytes of state. Each time they get a packet, they process it by updating the state. Certain state transitions (eg. The completion of a credit card transaction in a finance feed session) will trigger another event to be emitted, which goes through the same system again, and might go into another extractor or be persisted for analysts to view.
Totally off topic, but I love that XKEYSCORE sounds like a Redis command. There has to be an opportunity for some sort of practical joke here but I can't find it.
> We should make a GPT-based generator of USA secret codenames ;)
I'd read somewhere (can't remember where, unfortunately) that program code names were randomly generated by selecting words from two lists. The idea is that, if (accidentally) revealed, the name wouldn't provide any information about the program's function. The thing I read said people would often re-query the generator until they got a name that they liked.
> Part of the agreement between the US and Denmark was that "the USA does not use the system against Danish citizens and companies. And the other way around". Similar words can be found in an NSA presentation from 2011: "No US collection by Partner and No Host Country collection by US"
At first glance, it sounds OK, but what if the US has similar agreements with some neighboring countries (and as mentioned in the article it has) and uses the data collected via them against Danish citizens and companies and vice versa? Everything will be perfectly legal, but in practice, no one is at safety and the creators of the system who have agreements with many countries have a huge advantage over their so-called "partners" because they are aggregating the big picture, but "the partners" have some guarantees only about the data which is transferred via them and have no guarantees even about it when it leaves their country.
Let's not get naive here. I don't know if they did say that, but even if they did, would you actually believe an espionage organization's "word"? They've clearly spent a lot of money and man hours developing it and reaped a lot of benefits from that system and it's not like they can uninvent the thing.
They might have renamed it or bumped up the clearance level to tighten up the circle of people in the know, but are most likely still running it and constantly improving it to keep up and make use of most new technological, bandwidth and storage developments. They certainly won't stop collecting everything they can, that's just how things work, for better or worse.
I take issue with the statement that "Snowden was driven more by fears than by facts". Snowden revealed that the NSA was collecting information about citizens (American and many other countries) indiscriminately, for example every call record, every internet search, every web page viewed, etc. The documents he shared show the NSA's mission to collect ALL data possible, terrorist or not. Additionally it showed that there was basically no oversight when it came to digging into people's private lives; an NSA analyst could type in an email or IP address, click a few buttons, put down a few words for "justification" and then get a live view of someone's internet activity, have access to all of their data held by Microsoft, Google, Apple, Facebook, Skype, etc. This is all bad, and it's real, not just a fear.
This was all over the news some time ago, albeit not for that many days. Happy to see it summed up, because I never did figure out what was up back then.
As a european , i d rather have my surveillance camera phone china than the US. Google already knows too much about where and what i m doing, they dont need access to my camera. China OTOH doesnt have any kind of legal jurisdiction on me, we don't have some the kind of alliances that we share with US. Like during the cold war, arbitraging between spies was a safe bet.
What would you do if you were in a position of importance and China had some things you'd rather everybody not know? Don't be fooled that nothing can come of it just because the legal jurisdiction doesn't exist.
Couldn't agree more. So many people are so scared of China (the new red scare) but in reality if you did something illegal who do you have to fear more gets their hands on the data: The FBI or some PRC equivalent? There are hundreds of stories of FBI (and CIA) working in other countries but I have never heard of any PRC people kicking in the door of someone outside the PRC.
As a European I prefer having Erikson and Nokia build our networks than unaccountable Chinese companies. 5G is a unique chance to get some tech sovereignty back as the two leaders (outside of state-supportrd Huawei and ZTE) are European.
I don't want to be the wise guy, but you know that technically, the People's congress was elected correctly, right?
And yes, they had multiple parties until the republicans had two elections in a row, and managed to influence the supreme court so much that they could gain total power over new arising parties (declaring them illegal from the start if they do not represent the congress's opinion), up until there was no way to get elected because the media was controlled by the very same laws.
See any parallels regarding Fox News and the Republicans or say, Dick Cheney?
No? Maybe do some research on your own and sleep over this.
China is actually the only country I would compare US's democracy with, because a lot of candidates have no choice but to join one out of two partied to even get considered to be elected. And it's not the 1st vote that decides this, because democracy in the US doesn't differ between party votes and candidate votes (whereas most other democracies have moved on, for like hundreds of years, and fixed this).
Thr problem I see here is that the US didn't have a revolution. Europe had to be crushed a couple of times in order to learn how to prevent their architectural mistakes in future.
Data collection and political system are VERY different. All governments collect data on their private citizens, but not all sell their organs for profit or do forced sterilization
I think the US is much better at collecting data. The US has been proven to collect data and plant backdoors, China has not, despite how much the US states that eg. Huawei has backdoors in their 4G/5G equipment. So either China is much better when it comes to privacy online, or just way more competent as they manage to avoid getting caught.
Answering to both responses to my post: as an European citizen I know that technically US is better at doing data collection, even more so because it's a "friend" country and we can't wait to give our data to them
But my question really is: does it really matter to me, provided that the data is gonna be collected anyway, who does it?
They're both, at my eyes, not doing it to my advantage.
I guess as an Italian I don't see China as a bigger threat and probably China is less interested in harvesting my data for the reason that they are not selling me anything by targeting me whenever I do something on the internet?
I speak English, I don't speak Chinese, my continent is watching the US elections tonight, it doesn't happen with Chinese politics, my pears stay awake at night to watch the Oscars, I don't even know if the Chinese equivalent exist, basically what US does is much more relevant in day to day life, what happens in China stays in China, so they are not really trying to buy my attention, which is the most valuable asset I own.
Well, until that democracy decides you have something interesting they want to take away from you.
In that case, regardless of whether you're a grotesque dictator or a quasi-peasant just getting by with your life, better start counting the days before something bad happens to you...
Good point. Democracies are horrible because they gave us nazi germany. Or how about we stop with the silly propaganda talking points and deal with the topic at hand?
As a non-US citizen, how can I be sure that all non-Huawei equipment is free of back doors, data-exfiltration and forwarding capabilities excluding the lawful interception feature set?
Internet was too ethical, naive and immature when NSA did that. We were happily using unencrypted connections to connect to forums, telnet based BBSes and such.
NSA, OTOH, intercepted the switches which would isolate high security networks (red/black separation) and bleed sensitive information with these enhanced hardware.
>As a non-US citizen, how can I be sure that all non-Huawei equipment is free of back doors, data-exfiltration and forwarding capabilities excluding the lawful interception feature set?
I wish people would emphasize this more. Bitcoin isn't anonymous, and with the legal requirements for reporting transactions, much of what happens on there can be corellated to real-world identities. Far as privacy goes, it's probably a step backwards even from bank accounts and credit cards since there's no warrant needed to access it.
According to google, about 90% of the web traffic on chrome is now encrypted [1]. Does it matter that they are tapping a cable? The metadata can’t be that useful.
That metadata is surely still useful when you know frequency and target IP addresses. In addition not everyone uses google services and browsers, and so its really narrow to say 'chrome is encrypted, so who cares?'... This metadata by the way is exactly how you can calculate correspondence and association of graphs of people and information. How is that not exactly a breach of exactly what people fear most? It doesnt matter what data you are looking at, it matters who else is looking.
No the 90% I was refering to is chrome web traffic, not google service (toward the bottom, “Percentage of pages loaded over HTTPS in Chrome by platform”)
Right, and chrome is trying _Hard_ to make that happen, whereas opera, firefox, etc, are not necessarily automatically upgrading to https:// where possible from http:// URIs, etc. Its highly unrepresentative of anything but a google chrome user experience.
At scale metadata is MORE usefull than the specifics of the communication (web, phone, messages, videostreeams, ...)
Who talks to whom, When do they talk, how longm from where. What sites do they access, ...
It enables you to pick a person, find his friends, where do they meet(online and offline), where do they keep money, where do they spend it. What news do they read, what music, what their political leanings are etc.
It enables you to see the big picture. (of either a single individual or entire group of people)
The content only becomes useful, once you have general idea what is going on.
Well the article focuses on backbone cable tapping, but they collect metadata of "who talks to whom" from facebook and microsoft, too.
Lets say someone is using a northern european vpn-server to connect to facebook and that vpn-server is not cooperating. Decloaking is statistical analysis against timing and rough size of packages, so tapping the cables on both ends of the vpn-server and recording metadata for all its connections is key, even if they can't break the encryption.
I've heard that metadata can be very useful (sometimes more so than the actual data). I don't remember the argument very well or where I read it, but it was something along the lines of analyzing the timing of activities to connect individuals to groups and events iirc.
Why can't the NSA just force a web PKI cert provider to create a fake certificate for them?
We know from the Lavabit case that once you start keeping private keys away from the feds they start making problems for you. Prove to me that every single root your browser trusts is not compromised.
Who says they don't? TAO - tailored access operations - is known to intercept physical delivery and re-solder chips onto hardware to get access. It is all a question of "how much do you spent on a specific target".
However every such change could tip the target off: if you replace the certificate and the target knows the key of the cert they expect, that will tip them off. Now a lot of these tools are about mass surveillance and big data: collecting metadata about everyone, not about some well defined target, then run big data analysis on it to discover targets. Like you have one person who is flagged and they talk a lot to this "HackerNews-Server" and so all others who talk to that server get an increase in score and now multiple of those people have a score above a treshhold and get flagged. Can't do that if you don't spy on everyone.
But they can't run active intrusion against every civilian ever without exploding costs and high chance of being detected.
These kinds of attacks are usually run by a major threat actor (i.e. nation state), targeted, and not run at large scale.
Certificate transparency is unlikely to help in this case. Key pinning was the more secure option.
For some issues see:
They maybe don't even need to force them. There are plenty of certificate authorities. Just look at your browsers list of trusted CAs or even worse the big number included in Android.
You can assume that some of those are at least in bed with TLAs or can't withstand an attack for stealing the keys.
There are hundreds of those.
If only one is compromised an attacker could issue valid certificates for whatever website you visit. They maybe not going to risk a root CA but there are plenty of intermediate ones. Some are directly controlled by states, so no reason to compromise anyone.
The NSA is harmless compared to social media and other privacy threats. Without tapped lines WWII wouldn’t have been won. Silicon Valley was founded with that budget. They aren’t 24/7 trying to change your spending habits like social media. And they aren’t stealing your company or reproducing your product at lower cost to make your inventory worthless. Is it concerning? Sure. But so is the speech enabled remote control (which you can drill btw) and phone you use that listen and watch.
You don't need to be able decrypt the data in transit if you know the endpoints and can somehow compromise the endpoints at a later date. And that is way easier. Breaking encryption is hard and time consuming. Identifying a site a user regularly visits and exploiting that is more straightforward.