"At least 80% of fibre-optic cables globally go via the US. This is no accident and allows the US to view all communication coming in. At least 80% of all audio calls, not just metadata, are recorded and stored in the US. The NSA lies about what it stores."
I think he meant that the NSA had access to the 80% of calls that are routed through the US, not the the NSA is recording and storing literally every single one of them. I think he was misquoted or misspoke.
William Binney hasn't worked for the NSA since 2001. Were they recording all calls back then? Did someone still there leak new information to him?
The NSA is recording every call in the Bahamas and Afghanistan. They can listen to any call made in the last 30 days. (Edit: Only cell phone calls, but still.) It's possible that a similar rolling-window system is in place for calls going through the USA.
Neither of those two places could be described as having a particularly high volume of calls though. Its an order of magnitude or more difference in sheer volume you'd be looking at.
Well, what would it take to record the whole US? One version says that we make ~5 calls a day on our phones [1] and ownership hovers at 80-90% of the 18+ population (and probably a bunch younger). The US is 311 million, lets say 80% of 300 million, so 240 million people making 5 calls / day.
The average call length is 1.8 minutes right now [2]. So we've got 240 mil person * 5 calls / person-day * 365 days * 1.8 min / call. So, about 788 billion minutes of calls / year at a flow rate of about 2 billion calls / day.
At MP3 compression of 128kbps, vocal data takes about 0.94 MB / minute [3]. So, close to 688 petabytes of storage data at a rate of about 1.9 petabytes / day. Seems within the realm of doable.
The problem of analysing this is in the "ridiculous parallelism" category, so they'd just be constrained by server farm capacity. Lets say they had a system with a conservative million nodes. Each day, each node would have to process ~2 GB of audio data looking for patterns. Not even challenging. If I were clever, I'd probably run a brute force audio to text on each node, then a text to symbol pattern analyser. I'd also have higher level net processes that look for patterns in calls spatially and temporally, but with far less processors.
Standard POTS audio is a single 3.4kHz bandwidth channel. Compressing VOIP codecs allow a wider range, but still sound great at 12kb/s.
So without any non-COTS tech and without sacrificing the lilt of grandma's sparkly voice, we're already at a 90% reduction of your numbers.
So let's go with 190 TB/day. Say we keep 30 days in hot disk storage, and spool it off to a digital tape robot afterward. We'd need a few thousand COTS hard drives on less than a thousand servers (triple it for decent RAID).
Then a few million-dollar LTO tape libraries, and a couple of guys to schlep tape cartridges all day long.
(Of course, since we don't require all that compute power, and because Gov't doesn't build datacenters like Google does, they'd just buy a few fat IBM z/OS boxes, and a few fat EMC cabinets for low $10's of millions total, and call it a day.)
Either way, it's well within the realm of eminently achievable, I'd say. Which means it's obviously happening. :)
Thanks for the correction on the compression that's possible (and others who offered comprehensible work in the 1 kbps to 30 kbps range). I didn't know the SOTA so I went for what I as a consumer thought of for audio compression.
On the final point, agree. This is in the range where its cost is round off error in some of the large security budgets. (+ probably far more for the labor, ops, ect...)
That was one of the main reasons I worked through it, because it sounds like a horrifically large task, but its really not even that crazy in terms of data rates and storage when you break it down.
And you can divide by two (person A calling to person B is just one conversation needing to be recorded, instead of two recordings for person A and person B), and you're at less than 100 TB per day. That's pocket money.
Your compression numbers are way off. 128kbps is for stereo music, not monaural voice recordings. If you stick with MP3, you can go down to 32 or 24 kbps without sacrificing any intelligibility. If you use a real voice codec like G.729, you can easily get by with 8 kbps. https://en.wikipedia.org/wiki/G.729
Here, have a data point: GCHQ had a sub-1kbps voice codec in the mid-80s (it sounds horribly 'squelchy'), and they used tape when they were doing it back then...
This is cool. Everyone always talks about the idea of recording and analyzing calls, but I've never thought about actual implementation. I appreciate the perspective.
It's got to be two orders of magnitude easier for them to tap phones in the USA though. They don't have to ingratiate themselves with anyone or hide what they're doing, just show up at AT&T with a FISA-signed warrant and install equipment. https://en.wikipedia.org/wiki/Room_641A They've had a lot longer to do the work, and they have a lot less distance to transport equipment.
The call in Afghanistan is just satellite calls - those satellites are all US owned. Tapping them is trivial given the very limited number of satellites.
US telecommunications is geographically disperse, and my original point anyway was that the volume of calls is enormous - storing and processing that much data is hardly a trivial problem since it's both storage AND CPU intensive. From the US black ops budget you could make some reasonable estimates as to the total size and capability of the NSA to do this.
"The National Security Agency has been recording and storing nearly all the domestic (and international) phone calls from two or more target countries as of 2013." That seems like more than just satellite calls.
Not around here. Cats are great meals for coyotes, foxes, birds of prey, feral hogs... The list goes on and on. But don't tell a cat that and burst their bubble!
I have no idea what the NSA stores, but it's certainly feasible to store all the audio calls if they want to, and the new data center they're building has far more capacity than they would need:
Very probably yes. GCHQ's retention goes back (very broadly) that far (it's inconsistent) - and they have some access to each other's networks (the NSA access GCHQ's network via example.gchq.nsa.ic.gov, specifically, so for example https://wiki.gchq.nsa.ic.gov/index.php would get you to GCHQ's internal MediaWiki if you're on the NSA network - GCHQ use an internal .gchq TLD). They can query selectors on each other's databases, and filtering of what's "not allowed" seems to be very often up to the analyst. (Of course, I don't know if they share everything. Probably not.)
The NSA's own BLARNEY initiative for this stuff dates back to 1978, although I think GCHQ beat them to the punch in effective mass telephony intercept - the first one I'm aware of started in the early 1980s, although that is offline, and probably no longer archived as the media (digital tape with low-bitrate encoded audio) would probably have degraded beyond all usefulness by now (and it sounded pretty crunchy in the first place, I understand, kinda like a bad squelchy lower-side-band transmission!).
Remember, however, that they didn't have the same kind of analysis capability back then that they've got these days (but they can probably go back and analyse old stuff they still retain). As Snowden's disclosed, you're seeing the newer systems having full-take ring buffers in nearline storage, followed by offline selection for recent access and analysis from that using a huge amount of distributed processing, and in turn automatic selected archiving out of that. It's pretty much the difference between microfiche and Google in effectiveness.
Back then the key words were "echelon" and "carnivore". People had suspicions, IIRC there was an EU inquiry, but only now do we have concrete evidence.
I don't think there is any doubt about the fact that NSA wants to store it all. Just look at the short interview/documentary with Binney that was linked from the article: https://www.youtube.com/watch?v=590cy1biewc
You're assuming that the 20% drop off is due to capacity. That isn't my understanding of the current situation.
The Utah facility gives them the ability to store calls for longer, it doesn't increase their scope.
The reason they don't record 20% of calls is, from my understanding, that they have internal rules against recording certain people important to the democratic process within the US (e.g. politicians (and their families), judges, election officials, etc).
Essentially the politicians were concerned that the NSA's resources would be utilised by whichever party was in office to gain an advantage, so they put themselves (both parties) onto a "no spy" list. But you won't find the law where it says that, because it is all internal-rules created by committee behind closed doors.
But wouldn't it be nice if they could use their data to get rid of some corruption in politics? I know, I know, how does the government as a whole police itself? Very tough problem. But at some level I don't really care that they collect data, it's what they do with it that's important. So who decides? What is the master agenda? How do they self police? etc... One can only guess at the answers to these questions. It may be nice to allow anyone in the NSA to spy on anyone else in the NSA to prevent abuse, but that would be a huge problem if a spy managed to get a job there. I just imagine this being a rather complex problem.
They are not allowed to spy on US persons without a warrant. Full stop. The law is quite clear. Further, the court ruled that metadata was exempt from protection because, in part, it was voluntarily handed over to a third party by the end user.
I'd love to hear how you came to your conclusion that only politicians, et. al. enjoy such protections.
Read your links, but found nothing contradictory with what I said... and I certainly found nothing to substantiate the claims of the post I replied to.
What exactly used to be true that is no longer true?
"They are not allowed to spy on US persons without a warrant. Full stop."
That's what you said that is no longer true.
"The FISA Court (FISC) today released a heavily redacted version of its July ruling approving the renewal of the bulk metadata collection on all phone calls from US phone providers under Section 215 of the Patriot Act. "
Then there is the "3 hops" rule that allows full spying of individuals if you are "3 hops" from a suspected terrorist, which is most people alive.
I'm not saying whether or not I support any of this, but I do acknowledge that the NSA was not allowed to spy on US citizens before the patriot act and everything we know indicates that they strictly followed this rule, but things have changed.
We're staring at the gradual but deliberate end of privacy and its scary. The large majority of the world's population either doesn't know or care how significant or dangerous this trend is, and those few who do will find their way into surveillance databases because they act "suspiciously" by encrypting their communications and guarding their privacy.
I don't see a powerful enough counterforce against this insidious trend anywhere around the world. "Inspired" by the US, other countries are joining a competitive surveillance race stoked by private corporations selling everything from GSM monitoring to big data.
I'm liberal as all get out (and voted for Obama), but this isn't limited to the right-wing. Greenwald's getting attacked by Democratic partisans a lot over the NSA revelations. Those same folks would've been outraged during the Bush years.
More or less, yes. However, the danger is really only to people outside the safe herd majority who might draw the wrong sort of attention. So I don't see it being a huge danger.
However, I don't think it'll ever devolve to the point where mass arrests or anything truly draconian takes place. There isn't any need in Democracy. You need to only control 51% of the engaged voters [e.g. the people who actually vote] and at least in the US the two party system controls that quite effectively.
I'm worried about this because it will lead to people getting stepped on and crushed between the massive gears of the "State Security" apparatus.
Problem is the next governments will inherit the surveillance system. Who says they won't be fascists and/or turn the country into dictatorship ? When this happens, we might see these mass arrests. A lot can change in only a few decades.
One would hope that the US population wouldn't vote in a party of fascists. And if they did, then they get what they deserve. American voters aren't known for being particularly bright (they voted in a drunk who couldn't put together a complete sentence, twice!) but voting in fascists is a little far fetched.
Obviously their running platform wouldn't be "I'm more of a fascist than the other guy". No, it's probably going to happen like it usually does- somebody gets voted in for being less of a nutjob than their opponent, something bad happens, and the elected official uses that as a excuse for all the horrible policies that follow. It's already happened at least once here with McCarthy; he was just a freakin senator, one of many across the nation.
It almost happened once again with 9/11. Bush basically gutted Habeas Corpus, since he decided that anybody arrested under suspicions of terrorism wasn't allowed the protections afforded under it. The government could arrest you, send you to secret prison, and that would be the end of it since you were never allowed a trial.
True, none of the above examples are true fascism. But, they are scary steps in the right direction. Fear is an excellent political motivator, and with our fear-mongering media we have, that's a more powerful tool than ever. Invent an enemy scary enough, and it seems people will actually hand you their rights while thanking you for it.
Yes but, for instance, if you lived under a fascist anti-gay regime and were gay you would hide it very actively. On the opposite gays today, through their correspondances, might be quiet easily spotted by the NSA or anyone with access to their phone calls / mails / texts. Once the regime becomes anti-gay, they will come for you and it is too late to do anything.
So what your saying is you'd be able to destroy all crawlable evidence on the internet and wipe the memories of all your friends and strip out every reference to a "gay bar" or the like from your finances?
I think you greatly overestimate how effective people could be in wiping their fingerprints from the internet. There is a reason people talk about once you post something on the internet its out there forever. You aren't magically going to be able to wipe out decades of fingerprints because of a regime change, regardless of whether the NSA was collecting them or not.
You can use this argument when you can wipe out all copies of a nude woman whose ex posted her to imgur or something. Until then, you are just 100% wrong because you know you can't do what you claim.
The parent comment is saying that those things wouldn't be created in the first place if the environment were already hostile to them. You can hide your present to some extent, but not erase the past.
Yes. I'm aware the parent comment wants to try to ignore reality and pretend you can nuke the publicly posted information on the internet in the event of a fascist government.
That isn't how the internet or public communication works. The Nazis were able to identify the people they sent to concentration camps because of past associations, even once it became clear what they were doing because people don't live in a vacuum.
By the same token, you aren't going to be able to nuke all your facebook posts, twitter posts, etc. that might obviously flag you as gay "magically" whenever a fascist type government comes to power. The evidence is already there and completely out of your control.
If bad people come to power you are screwed regardless of what the NSA is currently doing. At worst, this shaves a few months off the process.
The most hilarious part of this is, all of you arguing with me about this in a public forum. Do you think you can magically go back and delete all these posts if some truly evil people came to power?
Nope.
I'm amused by the downvotes and arguments because:
1) You've publicly provided information that isn't going to go away already to show which camp you'd end up in if the people arguing with me are right. This information isn't vanishing simply because its posted publicly on the internet. It isn't going to magical become invisible in the future.
2) I've said I agree with the basic premise in my parent comment and basically said it was people who were vulnerable/edge cases that are getting screwed. That is still going to the base regardless of what happens because you only need control of 51% of the electorate to remain in power. The US electorate has shown, consistently, it will believe lies.
http://www.washingtonpost.com/wp-dyn/content/article/2008/09...
"For now, there appears to be little political reason to back down. A Washington Post-ABC News poll taken Sept. 5 to Sept. 7 found that 51 percent of voters think Obama would raise their taxes, even though his plan would actually cut taxes for the overwhelming majority of Americans. Obama has proposed eliminating income taxes on seniors making less than $50,000 a year, but 41 percent of those seniors say their income taxes would go up in an Obama administration."
The ability to fabricate falsehoods and get 51% of people to believe it is a far, far greater threat than the NSA. The fact none of you seem to notice this or even care worries me more than the NSA does. If I can get 51% of people to believe any lie I tell them, I can rule any democracy in the world.
I am not ignoring the reality: public or semi-public posts (facebook) are not going anywhere and should be considered public information.
Reading my previous post you can surely infer that I disagree with the mass spying but not that I am (or not) gay.
This is not a coincidence: I don't hide this political view and accept to be vocal about it publically because I believe discussing it helps protecting the democracy I live in today. If I am to be accused of having that opinion later on, then so be it because that was an accepted risk.
On the other hand I don't accept that my privacy can be used against me, be it related to my sexuality or the opinions I chose to keep for myself or only share with close friends privately.
Overall I am not denying that the information can be found elsewhere or that the Nazis used past associations (I have no opinion about that) but the key point is that you should be able to dissociate between your public life and your private life.
I am well aware of the public traces I leave on forums and "social media" but I don't know to which extend my private conversations are infiltrated and I am much less careful when I write them. So I am not sure what extreme (or not so extreme) opinion I once formulated and was stored somewhere. That could be used against me by the next fascist government. There are two ways to avoid that: the first is that it is never stored in the first place, the second is that I never formulate such an opinion. I want the option that is compatible with a democracy.
Obviously. However, speed isn't relevant since you can't erase your fingerprints of your entire life on the internet.
> You can use this argument when you can wipe out all copies of a nude woman whose ex posted her to imgur or something. Until then, you are just 100% wrong because you know you can't do what you claim.
That is really what it boils down to. You can't erase all of the evidence on the internet, so you are screwed anyway.
I think the most amusing part of this is the fact I basically agree with the comment I responded to except to the severity and scope of the danger.
Vulnerable people will get screwed. Most people won't.
And you can spend the time you'd spend building new infrastructure making the existing infrastructure even more powerful.
Additionally, while the people capable of building this infrastructure might refuse to do it under a fascist regime, they're comfortable doing it now under a democracy.
> However, the danger is really only to people outside the safe herd majority who might draw the wrong sort of attention. So I don't see it being a huge danger.
Unfortunately, you're wrong here. The first trivial counterexample that comes to mind: You cannot have a democratic society if a government agency secretly records the phone calls of all (current and prospective) political representatives.
Your trivial counterexample requires blackmail on a large scale. The problem with blackmail is it might work 75% of the time, but the other 25% end up in jail.
In the case of a politician, from time to time, one of them would say "fuck it" and use their platform to say it to the world.
The subtler approach would be to take dirt on Candidate A and give it to Candidate B so B could win the election. However, in the real world, slinging real mud is just as effective as slinging fake mud in an election. [e.g. Swift Boat Veterans for Truth had no actual evidence of their allegations and the majority of Kerry's unit said they were lying...but it still had a major effect on the election anyway] Tbh, the fake mud may be even more effective since you can create the best political narrative you can without being constrained by verifiable facts.
So I'm really not that convinced that the costs of such political meddling [clearly exposed villainy that may force politicians to take action against the Security Apparatus or get voted out of office] is not cost effective. It is far, far more likely that the Security Services would simply behave themselves and make it clear that they wanted a large budget and virtually unlimited powers. They pretty much are at that point already.
(Other than that, the argument that the NSA collecting data is less dangerous because in the U.S. political debate, facts don't matter anyway, doesn't sound too convincing to me.)
You are talking about people attempting to break the law...
"The first batch, published a week ago, included a conversation between Marek Belka, the chief of Poland's central bank, and Interior Minister Bartlomiej Sienkiewicz, in which they discussed how the central bank might use its power to help the government win reelection in 2015. Mr. Belka said that his condition for help would be "the dismissal of the finance minister" in favor of a “technical and apolitical” one. They also discussed the creation of a law that would allow the central bank to buy government debt on secondary markets, a practice known as quantitative easing.
Under Polish law, the central bank has to be independent from political interference. But Jacek Rostowski, the finance minister, did indeed lose his job in November last year and was replaced by Mateusz Szczurek, an economist with no political experience. And new laws that include a provision allowing quantitative easing are also now in the works."
Why yes, if you conspire to break the law and get recorded, this is a bad thing for you. I'm not sure why you cited one of the few cases such a recording would be perfectly acceptable.
That is actually the valid and good purpose of the media and the security apparatus. Exposing political corruption and bringing it to justice wherever possible.
Because such leaks will be used selectively, against anyone the data collecting agencies see as a worthwhile target.
And no, blanket recording of all phone calls is not the "valid and good purpose" of the security apparatus, even if it occasionally exposes political corruption.
> Because such leaks will be used selectively, against anyone the data collecting agencies see as a worthwhile target.
So because they don't remove all corrupt officials, just some, its a bad thing.
Okay. I guess I don't really see that as any different as now and is completely unrelated to the quantity of recordings.
> And no, blanket recording of all phone calls is not the "valid and good purpose" of the security apparatus, even if it occasionally exposes political corruption.
That isn't what I said. I said exposing corruption was the valid and good purpose. It had nothing to do with recording all phone calls, merely some conversations in restaurants which is what you were talking about being recorded is a valid and good purpose.
You don't get to just flip flop around and re-arrange the context of the subject you brought up [recording people having private meetings in restaurants involving illegal corruption] and [recording everyone's conversations].
>So because they don't remove all corrupt officials, just some, its a bad thing.
Yes! Absolutely! That's the worst-case. It means that the likes of Keith Alexander and James Clapper (and others who they may be in cooperation with) get to choose the government (to a degree sufficient to suit their needs).
The story in the Guardian by Antony Loewenstein certainly reports that William Binney's personal opinion as a former employee of NSA is that NSA is gathering up and recording whatever it can and that NSA has a "totalitarian mentality." That is a very important issue, if true, but it is at least debatable that NSA is really that thorough in its actual practices and really that generally blatant in disregarding the civil rights of Americans or even of people in other countries. For one thing, Binney also points to NSA intelligence failures in the same article, and if NSA is missing major activities of other countries (the Russian intervention in Ukraine) and nonstate terrorist groups (the "Islamic State" capture of much territory in Iraq), then surely NSA doesn't have the time and resources to analyze all of the data it gathers, and maybe it is not gathering as much data as some people claim.
Several comments posted before I read all those comments and read the fine article write about NSA blackmailing politicians. I don't believe NSA blackmail can or will happen in general, for reasons I have mentioned before here on HN. One of the most common kinds of comments here on Hacker News about issues like this is a comment that ASSUMES that if government leaders are under pervasive surveillance they are all afraid of blackmail. But I don't believe that, because some government leaders and some political candidates are essentially shameless. Even after they are caught (by old-fashioned journalism, or by a jilted lover or some unrelated criminal investigation) doing something unsavory, they are still willing to run for office, and SOME ARE REELECTED. United States Senator David Vitter was reelected in 2010 even after a scandal involving behavior that I would consider shameful,[1] and the antics of former DC mayor Marion Barry[2] are probably still notorious enough that they don't need further discussion here. In short, I call baloney on the idea that NSA can keep politicians on its leash simply by knowing their secrets. Some politicians have PUBLIC lives full of dirt, and still get elected and influence policy anyway.
The other reason I don't believe this HN hivemind theory of politics is that I by no means assume that everyone in politics lacks personal integrity. Some politicians, I am quite sure, could have all their secrets revealed only to have voters think "Why is that person such a straight-arrow? Why not have some fun once in a while?" The simple fact is that there is value system diversity in the United States electorate, and there is personal conduct probity variance among United States politicians, and there isn't any universal way to unduly influence politicians merely through even the most diligent efforts to discover personal secrets. If politicians think that NSA is going too far (as evidently several politicians from more than one party do think), then they will receive plenty of support from the general public to rein in the surveillance. (Obligatory disclaimer: Yes, I am a lawyer, who as a judicial clerk for my state's Supreme Court used to review case files on attorney misconduct, and, yes, some of my law school classmates are elected officials, including one member of Congress. I am absolutely certain that there are enough politicians ready to mobilize to roll back NSA surveillance programs if they really think the programs are excessive in their scope.)
First of all, whether they are shameless or not doesn't really matter. If sexually explicit pictures of a politician end up on the internet, his reputation takes damage. It's beyond his control. Plenty of politicians are willing to go to great lengths to avoid this. So what does it matter if he's shameless or not?
Second of all, politicians and other powerful people might have something to hide that isn't just embarrassing but is also illegal. It's certainly not beyond the capabilities of the NSA to target powerful people that oppose the NSA while ignoring those that support them.
For example, in the news you might read about a CEO that gets convicted of insider trading. Nothing to worry about, right? Well, what if the NSA purposefully targeted him because they don't like whatever it is he's doing? Using parallel construction, it wouldn't even have links to the NSA.
Nacchio was convicted of running a pump-and-dump insider trading scam that netted him ~$100MM at the expense of common public shareholders. If there's an award for "most obnoxious implication of NSA's wrongdoing", it should go to the attempted rehabilitation of people like Nacchio.
Here's the indictment. It's quite straightforward.
"No later than December 4, 2000, through and including September 10, 2001, NACCHIO was aware of material, non-public information about Qwest’s business, including, but not limited to" [litany of distressing concerns about Qwest's bottom line which ultimately proved dispositive in valuing Qwest].
Nacchio claims to have believed that secret national security government contracts were going to rescue Qwest from their financial problems (note the implicit concession that Qwest had problems from which its financials needed to be rescued). One tie-in between Nacchio and NSA is the notion that by refusing requests from NSA, Nacchio lost those contracts. Stipulate that this is true; it's a plausible complaint. Nacchio still took the money and ran.
P(insider trading | telecom CEO) is what I meant. Iff Nacchio was singled out for illegal but commonplace behavior, then it matters less to me what he did and more why he was the only one prosecuted. But if every insider trade has equal likelihood of getting caught, regardless of refusing NSA taps, then Nacchio is far less interesting.
I don't know if every inside trader has an equal likelihood of being caught, but Nacchio's case appears to have been particularly brazen. The facts I presented aren't disputed: he sold over one hundred million dollars of stock in a time period where the future of his company was very much in doubt, taking advantage of information his company didn't share with common investors.
The tragic thing about core human societal issues like this the same issues keep coming back, again and again- Quis custodiet ipsos custodes?
The truth is people with uncountable power will use that power unaccountably. I think often the person who gains the power originally isn't an abuser, but rather one of their successors.
The NSA needs accountability. But better yet, is removal of unnecessary power.
It might have sounded like paranoia in the past, but without evidence to the contrary, I will side with history and assume the abuses described above are happening.
> It's certainly not beyond the capabilities of the NSA to target powerful people that oppose the NSA while ignoring those that support them.
I had to think something similar when I read this in your parent post:
if NSA is missing major activities of other countries (the Russian intervention in Ukraine) and nonstate terrorist groups (the "Islamic State" capture of much territory in Iraq), then surely NSA doesn't have the time and resources to analyze all of the data it gathers, and maybe it is not gathering as much data as some people claim.
That is, how do we know for a fact the NSA missed this? Could it not also simply have not acted on it, for whatever reasons? E.g. I know on the surface it's in the interest of the US to have "stability" in the Middle East, but even former CIA people have been warning since the buildup to the Iraq war, that some strategies and tactics can lead to more terrorism, more instability. And then exactly that happened, because their warnings were not heeded. Am I to believe that was incompetence? I know this borders on tinfoil terrority, but I really don't mean to outright claim what my questions might imply, I am just wondering and genuinely curious/clueless, I don't read news regularly, so I have no idea what the NSA having no idea about the Ukraine and ISIS stuff is referring to.
>but it is at least debatable that NSA is really that thorough in its actual practices and really that generally blatant in disregarding the civil rights of Americans or even of people in other countries.
At this point in the debate it has been observed on a number of occasions that ranking officials at the NSA are dishonest. They have made comments both sworn, and in general press releases / statements (on the record) that have later been proven false; and we have reason to believe those officials knew they were making false or misleading statements at the time. So in short, the NSA, and the intelligence community in general have destroyed their credibility.
>Several comments ... write about NSA blackmailing politicians. I don't believe it will happen in general, for reasons I have mentioned before here on HN.
I personally suspect that it already happens. We know that it has happened before (it was a favorite tactic of J Edna Hoover). I see no reason to think that it can't/isn't currently happening.
>I call baloney on the idea that NSA can keep politicians on its leash simply by knowing their secrets. Some politicians have PUBLIC lives full of dirt, and still get elected and influence policy anyway.
I think that is a naive outlook, and that naughty habits of conduct actually serve the purposes of an extortionate NSA/FBI quite well. The NSA does not need to "keep politicians on a leash". They may merely need an occasional favorable vote or motion on the floor. Knowing the internal communications of congressional staffers (and their normal voting habits), they wouldn't need to risk extra exposure to get a marginal vote over the hump. Not to mention getting people like Ruppersberger, Rogers, Feinstein, etc. appointed to the appropriate committees.
The fact that some politicians run successful campaigns despite imperfect, even scandalous personal lives isn't proof that every politician / official is indifferent about their reputation, or that they are invulnerable to blackmail.
I can't really address your last points as they are your personal opinions but I think that most Americans have very naive and uninformed notions about how governance actually works, as opposed to the textbook version of it we were all taught in school.
At this point in the debate it has been observed on a number of occasions that ranking officials at the NSA are dishonest. They have made comments both sworn, and in general press releases / statements (on the record) that have later been proven false; and we have reason to believe those officials knew they were making false or misleading statements at the time. So in short, the NSA, and the intelligence community in general have destroyed their credibility.
Do you include William Binney, 30 year veteran of the NSA, in that cohort?
1) It doesn't require a revelation to be career ending for the person involved to be believe it could be damaging (and therefore be susceptible to blackmail).
2) It doesn't require all politicians to be be lack personal integrity and give way to blackmail, even a few can tip a balance.
3) Selective/distorted leaking could also tip the balance in places as to who gets elected.
4) Calls could be monitored to take advantage in negotiations for example by knowing what the real bottom line is in a negotiation, knowing which parts of a coalition can be picked off and exactly what they have been offered.
The question is not could the NSA take absolute control of all politicians but how many they would need to cause real corruption of the process. And I'm not saying that they are doing it now but that they shouldn't be allowed to get to a position where they could.
Regarding this in your disclaimer: "I am absolutely certain that there are enough politicians ready to mobilize to roll back NSA surveillance programs if they really think the programs are excessive in their scope."
Firstly there is no sign that they knew anything of the scope until Snowden, secondly "why aren't they doing it?" or failing that "if what we now know isn't excessive what would be?"
Indeed. CIA director lost his job when the FBI spied on his mistress' email account. No.2 Admiral of Strategic Command lost his position too for some gambling scandal. Certainly possible to coerce or blackmail other positions of power that are unelected. Not everybody is shameless like Anthony Weiner
The question is if they were spied on to get that info about them to force them out due to them being at odds with some other party. Who was the party, what were they at odds over -- and finally, how worse off is the nation when these people are potentially pushed out via this tactic.
>United States Senator David Vitter was reelected in 2010 even after a scandal involving behavior that I would consider shameful,[1] and the antics of former DC mayor Marion Barry[2] are probably still notorious enough that they don't need further discussion here.
This is sex dirt and drug dirt. Money dirt is crucial to the functioning of politics, and I'd like to see a similar list of politicians who have survived evidence of kickbacks, bribery, money laundering, and misuse of campaign funds. Sleeping with someone other than your wife while being videotaped smoking crack will not put you in prison.
>I by no means assume that everyone in politics lacks personal integrity.
For one thing, Binney also points to NSA intelligence failures in the same article, and if NSA is missing major activities of other countries (the Russian intervention in Ukraine) and nonstate terrorist groups (the "Islamic State" capture of much territory in Iraq), then surely NSA doesn't have the time and resources to analyze all of the data it gathers, and maybe it is not gathering as much data as some people claim.
I do not mean this as an ad hominem, but just by this statement, the rest of your arguments comes off as absurdly naive. Have you considered that perhaps the NSA and all the other "alphabet" organizations don't actually care much about accomplishing their stated goal, but are instead working tirelessly to amass power and influence through the machinations of the state? The linear way in which you assess personal and organizational motivations is simplistic to the point of uselessness.
Very few people or organizations with power are doing what they say. Instead, you must ascertain their intentions by what they do. This is politics 101. There are countless examples of this throughout history, but now is neither the time nor place to enumerate them all.
The real question to ask is, how in the name of Alan Turing's ghost did the NSA, CIA, FBI and the rest of the spooks with all their capabilities not find Osama, or Saddam's WMDs, or the Russian invasion, or...? Hint: it's not incompetence.
TL;DR: First order analysis of intention based upon a person or organization's stated intent are rarely correct. Instead, empirical modelling of behavior over time is required to ascertain true intent.
I'm wondering if the large scale data gathering (and I take your point about the organisation perhaps not being able to process all the data it has) is actually detracting from more focused action on appropriate targets, i.e. actual enemies of the state who represent a real and present danger of death or injury?
I've witnessed the assumptions that senior managers sometimes make in medium to large organisations that what they call 'IT' will somehow improve things or answer vital questions when in fact it is the observation of a business process with steps in detail that is needed to find the bottlenecks.
Perhaps I'm being cynical but the 'selling' of large volume data collection (even if a lot of that data has no predictive value and remains static on the hard drives in the data centre) to the politicians might be seen as guaranteeing budget.
> if NSA is missing major activities of other countries (the Russian intervention in Ukraine) and nonstate terrorist groups (the "Islamic State" capture of much territory in Iraq), then surely NSA doesn't have the time and resources to analyze all of the data it gathers
Or maybe foreign intelligence is simply far more difficult.
I can't imagine its easy to intercept high level government communications in Russia where they can simply order US bases companies to turn over user data.
I legitimately believe the greatest threat to most controversy scandals is individuals like you, who with long posts of obfuscation and obscurant arguments attempt to meddle with any bit of chance that any consensus or unity will form. To people like you, once we have enough evidence that would satisfy you, much would be lost.
I see no reason to trust you after posts like these. You have said so little with so much text. It shows that you were a lawyer.
Discussions surrounding threat mitigation and risk mitigation are uniformly one sided on HN, and even the most delicate walk down the line of acceptable mitigation is shunned.
The more interesting point is the inverse position of the community in regards to network and systems security, in which all measures of threat mitigation are encouraged or "required".
Discussions of threat mitigation are irrelevant so long as the threats and their countermeasures remain undisclosed. There can be no productive debate when one side knows everything and tells the other side nothing.
It doesn't matter if surveillance is used only for good (evidence suggests otherwise, though); if the surveillance organizations have all the power to operate in secrecy and distort the truth when questioned by the legislature, their "trust us" assurances are wholly untrustworthy.
It's naive to the point of absurdity. It's basically saying:
The NSA/CIA/FBI/etc. couldn't even see Ukraine coming! They were on their coffee breaks, those silly spooks! They never get anything right do they? So don't worry about them collecting information on you! They couldn't even find Osama! Those slackers. Don't worry be happy!"
The stupidest assumption one can make about a powerful organization is that it's incompetent and thus benign. Crack open a history book to see how benign power really can be.
Observing everyone isn't about targeting everyone, it's about targeting anyone. So even if they really did miss Ukraine, that doesn't mean we shouldn't care about them recording everything else.
You can collect lots of data without finding the terrorists, because to find the terrorists you have to do a good job analyzing all that data. But even if you're not doing good analysis, you can still keep the data in your back pocket to target anyone you wish.
As to whether scandals are effective, one counterexample is Eliot Spitzer.
Whether or not the data is analyzed in a timely manner is irrelevant to its' legality or morality (speaking solely on spying within the US.) One would assume that both competent terrorists and Russia have had proper NSA counter measures in place far before Snowden's leak. Indeed, bin Laden was able to remain hidden for a very long time. What we have seen is plenty of stories of dumb Westerners attempting to become jihadists.
NSA blackmail is far fetched. Of immediate concern is NSA employees inappropriately spying on other Americans. This has already happened multiple times that we know about: http://www.bloomberg.com/news/2013-08-23/nsa-analysts-intent... Lapses at the companies which conduct bulk top secret clearance checks should only raise more alarms.
The most recent doc leak demonstrates that not only was a contractor such as Snowden capable of searching and extracting data but it was done so without the NSA having any idea. No paper trail. How did someone get certain information on you or did they? No one knows. Blackmail hardly necessary if politically charged attacks reap large rewards by crippling a citizen running for office or an attorney general investigating a corrupt corporation. Anyone could be targeted: a sitting congressman, someone running for office, a federal judge, friends, children, family. Conducted in a veil of secrecy with secret oversight guarded by a national security mandate leaves no accountability: a total abandonment of America's foundation of checks and balances.
In the longer term, this behavior is more grim than that posed by a rogue individual such as Snowden. If the existence of the NSA and its spying on Americans is fundamental to national security anyone who opposes this behavior is as big of a threat as a bin Laden. It becomes appropriate to target not only Americans but also sitting government officials who pose a threat. Congress's muted reactions could leave one to believe this is already the reality. At the least, the fear is there.
Integrity and values are shallow and attach only individually to a mortal man. If an institution or government is built without transparency and without a balance of power, eventually, at some time bad people will gain control. They may be intentionally bad, seeking nefarious and self-motivated gain, or it may be unintentional: simply an absence of knowledge in a certain area which produces disastrous decision making. The Catholic church abuse scandals presented a mixture of both. Because the institution was deeply opaque and without proper checks and balances, widespread failure occurred. Even lives devoted to continuous moral contemplations, recitations and displays of charity could make up for this.
...surely NSA doesn't have the time and resources to analyze all of the data it gathers, and maybe it is not gathering as much data as some people claim.
That's just as speculative as absolutely every other bit of speculation that's flying around, though.
If you're in the NSA's shoes, and it turns out your budget is insufficient to analyze the mass of stuff you're recording, you don't stop recording stuff. You'd use this "analysis crisis" to justify additional budget for expanded analysis efforts, especially if it takes the form of automatic systems developed and sold by contractors located in the districts of selected legislators.
My own speculation is that yes, they're recording and keeping (for a little while, at least) absolutely everything they can get their hands on, doing massive damage to the ideals of freedom and democracy and America in general in the process, but surprise surprise, all that recording simply turns out to be much less useful than everyone had hoped.
"The police," broadly speaking, have argued for (often successfully) more and more high-tech spying capability for many decades now, but it's not obvious that such capabilities have made any big improvement in public safety. We may all fantasize about magic wands that could make our jobs easier; for some reason, American authoritarians actually believe they can buy such a thing.
Power corrupts, with all this unchecked surveillance somebody is going to use it against their political rivals it's inevitable. There could be lower level corruption too with DEA agents splitting the spoils of busts with NSA analysts in exchange for intel. Lobbyist payoffs for industrial espionage or intel on protester action against their pipeline plans. If none of this is reigned in who knows how long democracy will last, all the ingredients for a totalitarian state are ready.
The "shameless" ones are usually working for NSA. The shameless ones don't tend to be the "good guys", but the corrupt/easily compromised and bought politicians.
Thanks for the several comments about my comment. I upvoted quite a few, because they made thoughtful points. I appreciate thoughtful discussion of public policy issues, especially from people who disagree with me, because that helps me learn more about the world.
Some of the thoughtful points brought up in discussion of my parent comment of course address important issues that I didn't discuss at length in the parent comment. I should make clear that I fully agree with everyone here that if the common people of the United States (or of any other country) don't work together zealously and with mental toughness and solidarity to defend basic civil rights, then we are all hosed. I've seen one country's transformation from stark dictatorship to democracy and rule by law first-hand,[1] and that experience makes me very eager indeed never to see the United States slide into dictatorship. Similarly, I'd be happy to see the "tyranny of the Taliban"[2] reported on by the Guardian newspaper (the same newspaper that is the source for the article we are all discussing here) fall everywhere, and never return. People who have the dictatorial impulse try to spread their control as far as they can reach, and it takes concerted action to fight for and maintain freedom, which is why I frequently recommend that my fellow Hacker News participants study the writings of the Albert Einstein Institution,[3] based on the experiences of people power movements that have overthrown dictatorships.
Simply put, I am one of many Americans who has not only participated in public protests about NSA activities in view of my local police and TV cameras,[4] and I will continue to decry and protest about interference of my freedom or my family's freedom by anyone. Along the way, I am trying to engage in what one thoughtful comment described as discussions around "threat mitigation and risk mitigation," to understand better what private actions on my own part, what governmental actions by my government, and what actions by other actors will best maximize your freedom and mine.
>The story in the Guardian by Antony Loewenstein certainly reports that William Binney's personal opinion as a former employee of NSA is that NSA is gathering up and recording whatever it can and that NSA has a "totalitarian mentality." That is a very important issue, if true, but it is at least debatable that NSA is really that thorough in its actual practices and really that generally blatant in disregarding the civil rights of Americans or even of people in other countries.
As a European, with a much more turbulent and close-up sense of history (including actual dictatorships in our recent past, in several countries, or the "democratic" governments involved even in actual bomb attacks against their own citizens, usually with the support of the US too: http://en.wikipedia.org/wiki/Operation_Gladio ), I'm always amused by this kind of non-cynical naivety.
I guess it's the same way a non-criminal black person in the US, given their historical experiences, comes to understand that "justice" largely caters to some privileges and biases, whereas some protected suburban white child might have a much more naive approach.
What I'm getting at is: when it comes to those kind of operations, always assume the worse. And the so called "abuses" are seldom done by some "bad apples", they are built into the system.
>The other reason I don't believe this HN hivemind theory of politics is that I by no means assume that everyone in politics lacks personal integrity.
Another case in point. My take is that tons of people "lack personal integrity", and a certain loss of it is a pre-requisite for a succesful career in politics.
But in an a higher level, it's also that whether people in politics have "personal integrity" or not is besides the point.
The system can operate without integrity, even if most of it's players have "personal integrity".
For one, personal integrity is totally compatible with doing bad -- it only requires that you believe that the "bad" has to be done for an ultimately good cause (e.g surveillance and safety).
Plus, ethical systems (that guide "integrity") have many levels, strict conformance to the law and professional ethics is just one of them. For example, a cop that plants evidence on some guy, perhaps does not do it because he "lacks personal integrity" (e.g to pad his arrest record), but because he really believes the guy is guilty and that it's his duty as a good guy to put him in prison. That the guy could be innocent, and what he's doing is morally wrong, doesn't cross his mind, because he is convinced.
There are not a lot of things they are not allowed to intercept, and it is very unclear (and classified!) what their interpretation of those rules are.
Is that a serious question, or are you offering up a snarky "yeah, and so what if they did, who cares" style argument?
The answer is: a lot
But really you need to possess the capacity to conceive of at least one example of some selfish desirable evil, before you can grasp the vast potential this potent advantage of pervasive eavesdropping offers.
The elementary starting point in all these eavesdropping shenanigans is to imagine someone cheating at a simple card game. If you're the dealer, and your opponent is betting real money on the outcome of the game, would it be fair for you to look at his hand, or worse, stack the deck in you favor before the game even starts?
But population control goes way beyond the elementary sneaking of e-mails and phone calls. Total population control would carry profound and harsh realities with it, like eugenics, human experimentation, blood sports and human spectacle, secret prisons, human trafficking, sex slaves, disappearances. Trivial things like stock market manipulation become absurd, meaningless anachronisms, as corruption is endemic to the scenario. Extrajudicial killings, summary executions, sovereign leaders can wave their fingers and have people put to death.
What's it like to be a slave? That's what the ass end of total poulation control looks like. It's like being an olympic athlete for Iraq, and then coming home to be tortured by Saddam junior when you lose.
For those that benefit from total population control, every decadence awaits.
But, perhaps your rhetorical question is something along the lines of the "human rights are inalienable, and it would be impossible for them to do special terrible things, beyond the terrible things we normally fear like chemical warfare and nuclear annihilation"
Actually, no. Under a regime of technologically enabled total population control, there would emerge a class structure so rigid and inescapable that essentially a very small number of people would be born into a lavish, but paranoid existence, and the rest of the world would suffer a permanent crushing misery, that starts with ordinary poverty and gets worse. All of it enforced by geosynchronous satellites and atomic clocks with nearly the precision of spinning neutron stars.
Keep in mind that this would be a new age of technologically enforced, drone enabled population control. A North Korean gulag is bad, and it probably doesn't get much worse than being an elderly uncle to the dear leader, and still meeting an end by kangaroo court trial and an execution where your ripped apart by dogs.
Unless of course, the dogs are robot dogs manufactured by Boston Dynamics (oh, wait I mean Google). But maybe they're just drones armed with ever more accurate hellfire missiles. And that pretty much sums up the blue sky possibilities, if we fail to preserve the old-world checks and balances that are quickly disintegrating in the face of advances in modern technology. Completely obedient robotic systems enabling a truly air tight subjugation of humantity beneath an ever watchful panopticon. And wealth disparities that look like an abyssal cliff separating hollywood visions of the future from the stone age, and sociopaths enforcing the existence of the cliff perhaps for sheer amusement, narcissism, or whatever irrational human foible (bodily fluids, mandrake) worms it's way to the surface of the human factors that control these things that now influence our lives.
How many drones can be controlled from a single data center? How many drones would we need to enforce a permanent world-wide police state, and surveillance apparatus?
Technology is a force multiplier, and it permits fewer and fewer people to achieve a great deal, and possibly much more, far beyond what we've considered "super human" in the past. But worse yet, it can mutate innocuous well-meaning intentions in perverse ways. Thermonuclear bombs and missiles have kept a lid on serious global warfare for more than 50 years, but only by virtue of two psychotic power structures promising to vaporize each other at a twitch. Is that "good"?
But let's say the worst happens, and an evil megalomaniacal cabal implements a true robot armageddon. Wouldn't such a terrible sociopathic technocratic regime just cut to the chase, and kill off every last human being that dissents? Probably yes. But only if artificial intelligence proves to be a stalwart companion, otherwise much of the general population of humanity would be permitted to live, if for no other reason than fear of lonliness and inbreeding, but undoubtedly humanity would still be subjugated at the whim of the technocracy.
Anyway, to sum up, part of the story would be a short violent transistion phase, followed by a miserable but infinitely stable, robot-reinforced tyrrany. Like a marvel team-up of Kim Jong Il and son along with Saddam, Uday and Qusay all chanting "O'doyle rules!" while eating hot dogs and drinking Big Gulps and watching robots rape political disidents to death in a stadium filled with their genetically engineered clones and sex slaves.
Whether that is the NSA leadership's goal is anyone's guess. However, with the road they're going down, they will have the ability to exert considerable control over people around the world, particularly in the United States. Give that sort of capability it is inevitable that it will be abused. They're already abusing the power they have now.
They might not even have a specific goal. It could be one of those cases where they want to track and store the data, because it's available. It would be similar to being a developer, and wanting to track user information (user ip addresses, when they last logged in, which articles they viewed, how many times they viewed them, etc). Even if you don't have a need for that information now, it might be a key piece of data for an upcoming feature you develop a few years down the line. It's difficult to predict the future, but saving the data you have available is a good way to be prepared.
Now, I don't agree with what they're doing, but I could easily see something like this happening.
He was an (important) employee but did not speak for, or set the NSA agenda. He's also disgruntled. This makes the quality of the source less than ideal. Now if he had some sort of memo or recording that would change the source, and could provide more credibility.
But here's the rub - who still uses audio calls? Its a point of deminishing intelligence. That's like saying that 80% of all 8-tracks are now in 1970 cars (as a statement in 2014). Its just not relevant.
Uh, essentially everyone? I have a small VoIP company and we were handling around a billion calls a week. You have a really terrible selection bias if you think people aren't using audio calls. In fact, audio calls are on the RISE because of easy and cheap international calling. We have clients that have customers that literally leave their phones open to their families 8 hours a day. Just on and walking around the house.
There have been at least 15-20 trillion constitutional violations
It is an amusing, even throwaway line, but it has a horrific message - the US administration is happy to sail so close to the line of totalitarianism that it will possibly violate the constitution a trillion times. In the UK we jus found out similar legislation is unconstitutional and we are hurriedly writing another law to get round it. _sigh_
What happens when a country that really cares about it's constitution has to rush an amendment through or face civil rights violations from everyone?
I think he meant that the NSA had access to the 80% of calls that are routed through the US, not the the NSA is recording and storing literally every single one of them. I think he was misquoted or misspoke.
William Binney hasn't worked for the NSA since 2001. Were they recording all calls back then? Did someone still there leak new information to him?