Hacker News new | past | comments | ask | show | jobs | submit login
The NSA’s machine learning algorithm may be killing thousands of innocent people (arstechnica.co.uk)
402 points by mocko on Feb 16, 2016 | hide | past | favorite | 208 comments



They literally named it Skynet. They have an evil sense of humor.

Actually using machine learning to detect terrorists isn't a terrible idea. But you are going to get an error rate, and probably a high one in the noisy real world. Maybe only 50% of the people you detect are actually terrorists. Maybe it's even worse than that. We can't even test it because there is no validation set and unreliable labels.

The reasonable thing to do with that information, would be to surveil them further, search their house, or arrest them. Not assassinate them without a trial.

And the more I read the details, the more alarmed I am. The 50% figure I used above may have been way too high. The base rate of terrorists way too low and they have very little data to begin with.


A drone strike typically kills identified terrorist targets but also unidentified targets. The trick is that as long as bystanding casualities (also children yes - or as the drone operators call them "fun sized terrorists") have not been identified they're automatically counted as terrorists. (see also https://theintercept.com/drone-papers/)

I don't think now that the beast has been unleashed it can be controlled or harnessed with peaceful means (the tools of democracy). It will take more than a couple of middle-class people with billboards handing out flyers. America is fuelling its own terrorism and creating hate so they can continue their war on terror. Every terrorist act in the West will further drive the hate and justify surveillance. Pretty sure this is a downward spiral which ultimately has only losers.

Which steps can the average Joe take which aren't considered radical by the system? Do you know any please share because I don't.


Dr. King was considered a radical by the system but I will still leave this here:

>The ultimate weakness of violence is that it is a descending spiral, begetting the very thing it seeks to destroy. Instead of diminishing evil, it multiplies it. Through violence you may murder the liar, but you cannot murder the lie, nor establish the truth. Through violence you may murder the hater, but you do not murder hate. In fact, violence merely increases hate. So it goes. ... Returning hate for hate multiplies hate, adding deeper darkness to a night already devoid of stars. Darkness cannot drive out darkness: only light can do that. Hate cannot drive out hate: only love can do that.

Where Do We Go from Here : Chaos or Community? (1967)

http://www.amazon.com/Where-Do-We-Here-Community/dp/08070006...


Much love for King, but he was wrong as a point of fact. Violence has by far resolved more issues than anything else.


"The simple fact is that non-violent means do not work against Evil. Gandhi's non-violent resistance against the British occupiers had some effect because Britain was wrong, but not Evil. The same is true of the success of non-violent civil rights resistance against de jure racism. Most people, including those in power, knew that what was being done was wrong. But Evil is an entirely different beast. Gandhi would have gone to the ovens had he attempted non-violent resistance against the Nazis. When one encounters Evil, the only solution is violence, actual or threatened. That's all Evil understands."

-- Robert Bruce Thompson


>[President George W. Bush] has a vision which can be described with two other words: Manichaean paranoia... the notion that he is leading the forces of good against the empire of evil, that in that setting, the fact that we are morally superior justifies us committing immoral acts. And that is a very dangerous posture for the country that is the number one global power. ... The fact is he squandered our credibility, our legitimacy, and even respect for our power.

-Zbigniew Brzezinski, Former National Security Adviser, The Daily Show with Jon Stewart, March 14, 2007

Cultural side note:

"For Hindu thought there is no Problem of Evil" -Alan Watts, The Way of Zen

There is great anxiety among the Western Judeo-Christian tradition about "Evil winning out over Good" that is not so much felt by far Eastern cultures (Hindu, Zen, Taoist etc.). For instance, the concept of Yin and Yang and how they define each other and rely on each other.

St. Thomas Aquinas actually paralleled this concept quite briefly in The Summa contra Gentiles, though such an idea is quite taboo among many fundamentalist Christian traditions today:

>There are in the world many good things which would have no place unless there were evils. Thus there would be no patience of the righteous, if there were no ill-will of the persecutors; nor would there be any place for a vindicating justice, were there no crimes; and even in the physical order there would be no generation of one thing, unless there were corruption of another. Consequently, if evil were entirely excluded from the universe by the divine providence, it would be necessary to less the great number of good things. This ought not to be, since good is more powerful in goodness than evil is in malice, as was shown above. Therefore evil should not be utterly excluded from things by the divine providence.


Could you provide some notable examples where violence has resolved issues in the long term and did not lead to more issues/violence? This thread is in relation to the religious wars currently going on in the middle east. The Jews, Christians, and Muslims have been trying to use violence to resolve their issues in the middle east for thousands of years now to no avail.


A few notable ones off the top of my head:

WW2

American Revolutionary War

The Roman-Carthaginian wars

Genocide of the Ainu in Japan

Genocide of the Neanderthals by Homo Sapiens in Europe (Maybe; it's theorized at least)

The Gombe Chimpanzee War

I'm sure I can come up with more if you like.

The primary reason Jews, Christians, and Muslims are having trouble solving their problems using violence is that they're not using enough violence.


American Revolutionary War

Which led to the perpetuation of slavery for another 90 years (which the British were in the process of abolishing throughout their holdings), among other "issues."


That's irrelevant, and massive hindsight bias. If the British still held the South, they might have kept slavery too. And if they did try to abolish it, the civil war/revolution probably would have happened anyway and they might have won.


Wut?

The American Revolutionary War resolved the conflict between the two parties rather satisfactorily. I did not claim that it ended all violence.


Could you provide some notable examples where violence has resolved issues in the long term and did not lead to more issues/violence?

Was the original question. The point is that while it solved one conflict, it deferred and prolonged another (quite major) one -- which might well had been resolved had the colonies not seceded from the Crown.


There were also a lot of other conflicts on that list with a promise I could provide more. OK, how about this:

The execution of John Wayne Gacy.


Sure, and with the vague definition of "issues," you can move the goalposts all day long.


90 years of slavery that very likely would have been severely restricted in 1807 and abolished outright in 1833 (based on certain major events relating to slavery in the British empire, which I'm sure you're aware of) might sound "vague" to you, but most likely it wasn't to those affected.


>> The primary reason Jews, Christians, and Muslims are having trouble solving their problems using violence is that they're not using enough violence.

The problem of those groups is that they can't prosper in peace. Violence is not a solution for that. Rather, violence is the essence of their problem.

Basically, if you want to be realistic there can never be "enough violence". Reasonable people prosper because they understand this and seek to solve their problems in a peaceful manner rather than to escalate the violence until one side perishes.


They prosper(ed) rather well despite these minor problems. If that means to you they were using enough violence and a lot at that, I'd agree.


>If we look into History we shall find some nations rising from contemptible beginnings, and spreading their influence, 'till the whole Globe is subjected to their sway. When they have reach'd the summit of Grandeur, some minute and unsuspected Cause commonly effects their Ruin, and the Empire of the world is transferr'd to some other place. Immortal Rome was at first but an insignificant Village, inhabited only by a few abandoned Ruffins, but by degrees it rose to a stupendous Height, and excell'd in Arts and Arms all the Nations that praeceeded it. But the demolition of Carthage (what one should think would have establish'd it in supream dominion) by removing all danger, suffer'd it to sink into debauchery, and made it att length an easy prey to Barbarians.—England Immediately, upon this began to increase (the particular, and minute causes of which I am not Historian enough to trace) in Power and magnificence, and is now the greatest Nation upon the globe.—Soon after the Reformation a few people came over into this new world for Concience sake. Perhaps this (apparently) trivial incident, may transfer the great seat of Empire into America. It looks likely to me. For if we can remove the turbulent Gallicks, our People according to the exactest Computations, will in another Century, become more numerous than England itself. Should this be the Case, since we have (I may say) all the naval Stores of the Nation in our hands, it will be easy to obtain the mastery of the seas, and then the united force of all Europe, will not be able to subdue us. The only way to keep us from setting up for ourselves, is to disunite Us. Divide et impera. Keep us in distinct Colonies, and then, some great men, in each Colony, desiring the Monarchy of the Whole, they will destroy each others influence and keep the Country in Equilibrio. [1]

-John Adams

So yes, empires do rise and rule for a time but in the end they grow weak from over reliance on luxuries and having no real competition to fight. In the end these empires based on war always collapse.

But let's take WW2 for example. WW2 led immediately into the Cold War, and the Cold War led into the War on Terror. It is also worth noting the arbitrary national lines drawn along tribal boundaties after WW1 and WW2 and their direct contributions to the enflamed tensions in the middle east today.

>The primary reason Jews, Christians, and Muslims are having trouble solving their problems using violence is that they're not using enough violence.

That is M.A.D.

What exactly is the issue here, what is the end goal that is achieved through violence? It's survival, right?

If we are to ensure survival amidst such increasingly rapid technological proliferation, mankind must put an end to war or war will put an end to mankind.

[1]http://www.masshist.org/publications/apde2/view?id=PJA01d005


Do you really think that this violence will not come bite western world in the ass one day? West is allowing killing of innocents, including the "fun sized terrorists" kids. Thousands of them. Do you really think they will just grow up and forget about it?


It took one executive order to start this, it will take one executive order to stop it. The election is in a year's time. Make your vote count!


Which candidate do you mean will stop it?


Rand Paul would. Bernie Sanders might.


I would have thought Obama would have. I doubt anyone is going to be able to stop it without shutting down the whole intelligence apparatus.


Bernie definitely would, except -- I'm not too sure JFK wasn't an inside job w/ the CIA - I wouldn't put assinating a president past the NSA, heck I wouldn't put anything past them, the more I read--the more I feel like I woke up in 1984, and we're already in some post-apocalyptic world where Big brother controls everything but we just don't know it (yet).


JFK got done in because Lyndon Johnson was a much more palatable president. If the vice president is not on their side, they'd have to keep killing people down the line of succession until they got to somebody who was. After the vice president: Speaker of the House (currently Paul Ryan) > President pro tempore of the Senate (currently Orrin Hatch) > long line of officials serving at the pleasure of the president.


> A drone strike typically kills identified terrorist targets but also unidentified targets.

A drone strike typicall kills identified terrorists because they need to be able to say that. It's much better to be able to say "... and some collateral damage" than "We hit a wedding party."


We need a military. All states do -- its part of their duty to their citizenry.

The problem in the US is that our hippy-run academia are driving the smart people out of jobs in the military (and police force for that matter), and it's putting us in a terrible position.

The only way to fix it is for smart people to start joining the military again.


Academia doesn't drive anyone out of the military. It may dissuade some people from joining.

US military culture drives people out.

Either by being surrounded by lazy or coasting individuals who think of the military as just a job (discouraging to more ideologically minded people).

Or by being hammered by the ugly, jingoistic rhetoric and motivations of some peers and leadership.

Or by being too smart and standing too tall, upsetting the boat. The US military, large portions at least, strongly discourage some aspects of intellectualism and, certainly, individualism. Smart people suffer in this environment if they're not power-motivated, and the good ones usually aren't.


Oh that is why ROTC is such a big issue and many elite universities do not have it in their campuses.

Intellectualism is absent in military? Some of the best lectures I watched are from Naval War College. TOR comes from that part of the woods.

You do not have individualism in any armed forces, this is absolutely lunatic. Its like preaching individualism in Basketball team and telling Allen Iverson he is oh so special and do it all ( which he did and spectacularly failed). US Military just like other military is Socialist both in its creed and organization. That is no surprise and PG actually credit to Post-War equality in America to be by-product of Military service and GI bill.


Re ROTC: Which elite universities are you speaking of and why don't they have ROTC detachments? It appears, from a cursory glance, that MIT and Harvard don't have ROTC detachments, then those cadets would go to a detachment at a nearby university. This is typical for smaller schools that neighbor other universities (see universities and colleges in Atlanta, GA for this as well, several of them send or sent their cadets to GA Tech's detachments rather than hosting their own for a smaller cadet corps).

Re Individualism: You're taking the extreme position of individualism which is not what I intended. I meant those people who generally don't suffer from groupthink.

An example of groupthink, second-hand tale: During the second Iraq War, my father was responsible for a lot of CSAR mission planning for the USAF. He would not send helicopters into bad weather conditions (primarily sandstorms) that he knew (after 20+ years of experience) they couldn't handle. The Army CSAR guys would send in their own birds and crews. The USAF CSAR crews would then have two rescue missions once the weather cleared. There was a pressure to ignore reality and press forward despite the odds and regardless of the cost that put more personnel and materiel at risk than necessary. In some sense it's honorable, in another it's just stupid. A degree of individuality, the ability and willingness to risk saying "no" when everyone else is hell bent on something, is the sort of individualism that's needed.

Sometimes in combat situations like the above, but the majority of the time not. See the drone programs, surveillance programs, acquisitions programs, that are either unethical, ill-conceived, or just wasteful. Groupthink allows them to persist. Generals who let it slide because they know they're getting promoted in 18-24 months and it'll be someone else's problem allow them to persist.

Some degree of individualism is necessary in both the officer and enlisted corps to step in and alter the culture.

A squad of all individuals unable to act as a team, yeah, that'd be fucking stupid. Obviously that's not what I was talking about. If you thought it was, you took the worst possible interpretation of my words and ran with it. Great.

But an officer corps of all yes-men drones is useless, too. They need people that can break out of the mold at every rank, not just the top.

EDIT: And because it's not clear. Sometimes, the willingness to say "yes" when everyone else is saying no is also useful. The story above was about being more risk-averse than the group, and it being the better option. But sometimes the group is too risk-averse and someone willing to step in and risk failure is what's needed - see the example of the generals who let shit stand instead of changing them. The real leaders will step in and risk their career when something is broken but unchanging because of fear or cargo cult behavior or whatever else.


We, if you mean the U.S., once considered a large standing army a hallmark of tyranny: e.g. http://press-pubs.uchicago.edu/founders/documents/a1_8_12s6.... is George Washington needing to argue "yet a few Troops, under certain circumstances, are not only safe, but indispensably necessary".

How we got from there to over a third of the total world military spending, well, it's a long story, and I don't know what to do, but I don't think encouraging more smart people to join up will help the most.


>In time of actual war, great discretionary powers are constantly given to the Executive Magistrate. Constant apprehension of War, has the same tendency to render the head too large for the body. A standing military force, with an overgrown Executive will not long be safe companions to liberty. The means of defence against foreign danger have been always the instruments of tyranny at home. Among the Romans it was a standing maxim to excite a war, whenever a revolt was apprehended. Throughout all Europe, the armies kept up under the pretext of defending, have enslaved the people.

-James Madison, Constitutional Convention (1787-06-29)


@alanwatts,

that is the kind of talk that left US exposed in 1812 War and the West Point, Naval academy were a product of realities post-1812 war. You can quote all Madison you want, but the need for professional army was evident through out the world and worked like a charm in 1845 When US wrested massive land from Mexico.


I don't mean to argue against the logic behind the negative-feedback cycle of the security dilemma.

The Madison quote was meant to indicate the historic precedent for over extending oneself on the war front and the ways it has left the civil front critically vulnerable.

However, today, in the electric age, there is no distinction between civil and military fronts.


This is absolute bullshit. The problem is that we need a military that is actually going to man the fuck up when an O-5+ decides to push some unconstitutional program on the people.

What has happened isnt that hippy academia is jacking up the military, but rather that the military has been kicking out the kind of people who arent afraid to tell truth to power, and are increasingly making choices and then only hiring people who tell them those choices are good and what they want to hear.

I have spent the majority of my free time since I got out trying to understand what the fuck is going on here, and the reality is that our generals failed our military by allowing the neocons to push us into wars for dubious reasons. Thats even counting the real reasons no one talks about, such as upcoming resource wars and a return to a tripolar world.

No, the real problem is that we have barely had a real president since the new world order bunch assassinated JFK (and RFK), because that was a very clear message to any POTUS willing to actually stand up. (Dont get me wrong, JFK was far from perfect...)

We now have institutional corruption from the top down in every single branch of government, including the famed fourth estate which has been turned into stenographers weekly. Corruption and incompetence are running rampant, and are acting as a cover for the malicious string pullers (Hanlons razor is a logical fallacy!)

So no, its not that the goddamn hippies have fucked up the military, its much closer to the military and its propoganda programs have so far infiltrated academia that its a lifeless shell of what it could and should be in this internet age.

Keep in mind though, its the military who has pushed this though. While the three letters have carried out operations similar to mockingbird (thank you church comittee), its the globalist new world order group who have pushed it on their controlled government puppets. (Read up on what Norman Dodd found during the Reese Committee for more info)

Yes, you are correct we need a smarter military, but just having "smart people" join up wont do it. You need change from the POTUS down, because beleive you me, we have turned mental deconstruction and reconstruction into an art form.


New world order? The Democrat establishment is doing everything it can to keep the primary from being taken from their anointed disciple. And the Republican primary barely even contains an establishment runner. The GOP power brokers can't decide if they should spend more effort attacking Trump or Cruz, but their man Rubio is trailing a distant third, and their real choice, Bush, is lost in the noise.

If there is a world order, it was never very powerful or orderly.


Don't be fooled by the circus. The string pullers have had a plan the whole time. While Trump and Sanders have slightly thrown a wrench in my predictions, they still factor in to the plan.

First, you have to look at who the original runners were: Jeb and Hillary. Jeb is connected to the action arm of the dark arts, through his father, brother, and his time in Florida, and Hillary is connected to the Rhodesian group for Anglo-Saxon dominance through her Oxford husband.

My prediction was that they would drop the dirt on Hillary mid-late cycle, while dropping the GOP forerunner (right now Trump) and suddenly shifting his votes into Bush. Suddenly we have a potential Bush vs Sanders/Hillary election, and the reason the world order group are fighting so hard to get Hillary the nomination is because Sanders is a wildcard in their plans. If they keep Hillary in, it's a globalist in office either way.

Incompetence is rampant, it's true, but don't let it smokescreen the hidden machinations, or else you will never be able to grasp the bigger geopolitical picture.


allowing the neocons to push us into wars for dubious reasons

It seems to me that it was some group other than neocons that got us inveigled in war in Libya and Syria - thereby creating the fertile spawning ground for ISIS.


Not sure what you're getting at, but ISIS command consists mostly of former Iraqi officials that were sidelined after Saddam Hussein was ousted. The soil was initially fertilized by burning down Iraq.

The "fertile ground" that allows moderates to become radicalized eventually boils down to polarization. It doesn't really matter whether the rhetoric is coming from Somalia, Nigeria, Mali, Turkey, Indonesia, Libya, Israel, Europe or the US; as long as the discourse goes in terms of "us vs them", many people will identify (and act) as victims.


I was careful in my wording. I do grant that the initial war in Iraq catalyzed the creation of ISIS. What I was saying here is that the power vacuum we caused in Syria and especially Libya created a place where that seed could grow and thrive.

But anyway, that wasn't the main point. The real point was to show that it wasn't a neocon that started our wars in Libya and Syria - it was today's iconic Progressive who did that, and without any sort of Congressional approval at all. Those wars can't be attributed to neocons, they are the Progressives' to own.


> It seems to me that it was some group other than neocons that got us inveigled in war in Libya and Syria - thereby creating the fertile spawning ground for ISIS.

The group now calling itself "the Islamic State" was a comparatively minor Islamist group in Saddam's Iraq that hit the big time with the US invasion of Iraq and subsequent occupation, when it leveraged the disruption of that invasion and the currency of the al-Qaeda brand to rebrand itself as "al-Qaeda in Iraq" -- it was so successful -- seizing weapons and territory and becoming a substantial force that, particularly with the rest of the global al-Qaeda brand in decline, that it found it best to rebrand itself subsequently as "the Islamic State in Iraq".

Sure, when the Syrian Civil War escalated, it sent fighters there, who eventually remerged with ISI leading to its next rebrand as what is usually translated by official sources "the Islamic State in Iraq and the Levant" (but(alternatively translated as "the Islamic State in Iraq and Syria", which is where the "ISIS" label comes from), and the group later rebranded to simply "the Islamic State".

But it was the invasion of Iraq -- not the conflict in Syria -- that formed the "fertile spawning ground" which catapulted the previously-minor group to become a significant threat.


Neocons are just a front for the Leo Straussian Chicago school of globalism, with players like Kissinger and Brezenski.

note: too late to edit original comment, but I meant to say "it's not the military", eg the military is being played in this global chessboard by other entities.


New world order conspiracies? On HN? Please don't.


I understand this response but I think it is the wrong one. Yes, there are many crazy and outlandish conspiracy theories out there that make it easy for the intellectually lazy to dismiss all theories out of hand, but the globalist conspiracy is one that is less and less hidden to the point that you aren't really paying attention if you don't see it. Now, thats not to say there is a single, grand, overarching conspiracy (a common fallacy in the realm), but rather there are many different parties all with their own self interests, sometimes aligned, sometimes not.

The conspiratorial view of the world is the correct one. Your attitude is one of sticking ones head in the sand and pretending these things don't exist.


The conspiratorial view of the world is the correct one.

Not when said conspiracies are dumped, explanation-free, into the middle of a post without any backing what so ever.

Buzzwords like "new world order" are both loaded and uninteresting, and that goes double when most people who've have examined the "evidence" have found it wanting. If globalization is meant, say "globalization".


Deapite the most strong public sentiment connotations, the phrase correctly distinguishes a sect of the globalist group, and is just used as such by me. Hence the phrase was "new world order globalists". Its a loaded term, granted and I could probably find a more descriptive way to articulate it, but focusing on that as criticism is nothing more than grammatical group think as a strawman for the much more importance issues being put forward. Its the lazy way out, eg, "that dude said "nwo", all his point are moot and hes a crazy conspiracy theorist." That essentially the position you took.

Also, are you even in the same thread as me? Theres plenty of context, within my comment and out, thats gives an indicator about why such a comment is relevant to the discussion.

What you have done is perform a knee jerk reaction and now you are backpedalling.


Use of buzz phrases is generally the mark of "crazy conspiracy theorists" (your words, not mine) or: people who generally have a tenuous grip on concepts like burden of proof and recognition of confirmation bias.

I have yet to see a single thing in this thread that suggests my initial read was incorrect, or indeed anything to indicate your rant contributed anything to the conversation. (Hence: uninteresting)

Like a single link to support your assertions, for instance.


I thought the problem is that, 25 years after its only serious opponent in the world disappeared, the US spends as much on the military as the next 7 countries - five of which are US allies - combined.


"A nation that continues year after year to spend more money on military defense than on programs of social uplift is approaching spiritual death."

-Beyond Vietnam, Dr. King, speech given exactly 1 year before assasination


Luckily through expert (sarcasm) US negotiations and a 3d chess policy, Russia just declared a new cold war and has warned of WWIII over Sryia. Eastern European countries are begging for US soldiers to be stationed there while they ramp up their own military spending to dissuade Russian aggression. So it looks like all that military spending will be needed after all.


I thought that only the military system--promotion boards courts martial--could drive anyone out of the military.


Costa Rica doesn't have a military. (See also https://en.wikipedia.org/wiki/List_of_countries_without_arme... .)

How has that country failed in its duty to the citizenry?


Correction: Costa Rica has no standing army. They have a gendarmarie and other paramilitary forces organized under the Fuerza Pública (Public Force). To say they have no military is not accurate.


There's a definition nuance I don't follow. When https://en.wikipedia.org/wiki/Military_of_Costa_Rica says "Costa Rica does not officially have a military" and "abolished the military of Costa Rica", that makes me think it has no military.

When http://www.coha.org/costa-rica-an-army-less-nation-in-a-prob... says "Without a formal military force, Costa Rica had no need to regularly invest in naval vessels, warplanes, or tanks, which typically make up the heavy equipment of any traditional armed forces." and "It is important to consider that a state that has no military is by nature nonviolent.", that makes me think it has no military.

The same with http://articles.latimes.com/2013/dec/15/opinion/la-oe-barash... which says "Henceforth, Costa Rica would take the almost unheard-of step of renouncing its military."

I don't understand why I should believe it has a military. At best we have different ideas of what a "military" means.

You mention "gendarmarie and other paramilitary forces" as a type of military. SWAT team are paramilitary forces, no? Would it be fair to say that Los Angeles has its own military? What about paramilitary support for anti-poaching and other conservation efforts?

The essay at https://medium.com/war-is-boring/costa-rica-doesnt-have-a-mi... attempts to convince me that the UEI is a "small military force in all but name", and that "the distinction between police and military work in Central America is a lot fuzzier than it might seem". I am not convinced by their comparison of how the police in Costa Rica have a drug interdiction role which is done by the military in other Central American countries - after all, in the US there is both military and police involvement in the same role.

Perhaps I can resolve the topic with two questions: 1) did Costa Rica have a military in 1990 (which is before the gendarmarie and the UEI)? and 2) what would Costa Rica need to abolish in order to say it doesn't have a military? Just the 70 people in the UEI?


> But you are going to get an error rate, and probably a high one in the noisy real world. ... The reasonable thing to do with that information, would be to surveil them further

It is even worse than that. Due to the base rate fallacy (https://en.wikipedia.org/wiki/Base_rate_fallacy) the 'machine' is going to have a huge false positive rate, so without that additional surveillance, most will not be guilty of what they are accused. But that surveillance takes money and resources, just the thing the 'machine' system is designed to save...


Yes. I feel like anyone considering such a system should be forced to read "The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection":

http://www.raid-symposium.org/raid99/PAPERS/Axelsson.pdf


People interested in these issues should read the position paper written by Jeff Jonas and Jim Harper in 2006, "Effective Counterterrorism and the Limited Role of Predictive Data Mining".

http://www.cato.org/publications/policy-analysis/effective-c...


The reasonable thing to do with that information, would be to surveil them further, search their house, or arrest them. Not assassinate them without a trial.

This appears to be what they do?

The National Security Agency reportedly tracked phone calls between the courier Abu Ahmed al-Kuwaiti's relatives in the Persian Gulf to all numbers in Pakistan, and NSA surveillance eventually tracked Abu Ahmed al-Kuwaiti's location in Pakistan via one such phone call.... In August 2010 they tracked al-Kuwaiti as he drove from Peshawar to a residence in Abbottabad – and as analysts inventoried the compound's striking security features they became convinced that it housed a high-level al-Qaeda figure... In September 2010, the CIA concluded that the compound was "custom built to hide someone of significance" and that it was very likely that Osama bin Laden was residing there[1]

As it turned out, that was where bin Laden was killed, so that process seemed to have worked.

I'd note that the arstechnica article gives no evidence at all that this particular program is used to generate kill lists. As you note, identifying possible couriers and just killing them without further investigation is a dumb idea. Fortunately I don't see any evidence in this or other articles that this is what happens.

[1] https://en.wikipedia.org/wiki/Abu_Ahmed_al-Kuwaiti


The evidence is that the head of the NSA literally said, "we use metadata to kill people". It doesn't get more direct than that. Signature strikes are a well documented phenomenon by now.

Also there's no way the CIA drone program can sustain the kind of long term kill rates they've been having without routinely killing innocent people. Terrorists just aren't that common (although if you define anyone who tries to seek revenge thanks to an errant drone strike as a terrorist, that "problem" can eventually solve itself).


I can't believe I'm actually defending the NSA here, but "we use metadata to kill people" does not mean "we ONLY use metadata and nothing else".


Of course, they also use bullets.

/s


They asked the NSA to defend themselves, to make a comment on the issue. They were totally silent.


If you are going to put surveillance on potential terrorists, you should also be putting surveillance on people not suspected of being terrorists, so you can judge the rate at which your surveillance personnel falsely accuse people being terrorists based on having been selected for surveillance.

Single or double blind that process.


They could put the surveillance on themselves for that good people control group. Oh wait...


That would be true if you were genuinely interested in avoiding innocent deaths. If you placed very little or no value on non-Western lives, then there would be no incentive to take that kind of extra care.


Reading discussions on drone strikes makes you wonder what ever happened to pacifism. This whole idea that you can just go around other countries killing people -- even if they intend on killing you, and even if you would be able to hit the right target 100% of the time-- it's just old. It's primitive. And that is not to blame the USA. At this point in history humanity, as a collective, has the maturity of an 11 year old. Maybe it's time to evolve a bit.


Couldn't agree more......just reading the front page of /r/all these days makes me want to drop out of society and move deep into the forest. I mean just take one example, the Flint water crisis - this is so unimaginably corrupt (is that even the right word) that it blows my mind. How do people knowingly do such things, how do we have no checks and balances in the system, how can we justify not sending several people to jail for a very long time?


I don't see why pacifism is "more evolved." Violence is one of the most fundamental interactions between organisms, right up there with eating and sexual reproduction. Is eating "primitive?" Are you looking forward to a future where humans "evolve" out of needing to eat? Throughout history, prosperous and successful societies have been built on the back of military power (Egypt, Rome, Great Britain).


We, as a human species, are supposed to evolve beyond our basic immediate needs.

We created societies, because we need to eat, reproduce, survive, without reverting to base violence. We created medicine, technology to improve our quality of life and reduce the chances that biological evolutionary pressure destroys us. The black plague came close to killing 60% of the European population. It is now eradicated.

What is the next step in our evolution as a species? Right now the biggest danger to the survival of humankind is... humankind itself. If we are to survive, not for a hundred years, but for a hundred thousand years into the future, we must learn how to cooperate without annihilating each other in the process.


Does our technology and development "evolve [us] beyond our basic immediate needs" or do we use it to serve needs we've always had? We spend an enormous amount of our effort making both food and sex easier and more abundant (there's an app for that!).

Violence isn't a vice that could be eliminated if only people had more virtue. It serves a practical function. It's the ultimate dispute resolution mechanism for people who either have too little in common or too much at stake to resolve their issues in any other way. Prosperity, abundance, and homogeneity makes it less necessary over time, but so long as conflict is possible between people, violence will be the final resort for resolving it.


> Are you looking forward to a future where humans "evolve" out of needing to eat?

Personally, yes. Taking time out to eat so I don't die takes me away from other things I was enjoying doing.

I enjoy the taste of many foods, but not as much as I was enjoying the thing I was likely doing before it.


I am grateful for the processes of cooking and cleaning and other daily maintenance in my life. I'm reminded of a zen proverb: "Before enlightenment, you chop wood and carry water. After enlightenment, you chop wood and carry water."


"Are you looking forward to a future where humans "evolve" out of needing to eat? " Actually yes, absolutely!

Don't get me wrong, I love eating as well as cooking, but I have to agree that a stupid amount of time is spent on doing those things. If I could switch to a solution where I take a pill and it provides my body with all possible nutrition then I would in an instant.


Yeah, I had a flatmate not long ago that took 'enjoying cooking' to a lewvel I'd never witnessed before.

He would spend three or four hours in the kitchen each night and sometimes all day on Saturdays and Sundays.

It baffled me to no end to spend that much time just to sate hunger.


Yeah I spend ~40 minutes making dinner and then another 40 minutes later in the evening making lunch for myself and my partner for the following day. Then there's cleaning up time as well. I like having home made food at work the next day but it's time I could be using for programming and other things really.


>It baffled me to no end to spend that much time just to sate hunger.

Clearly he wasn't trying to sate hunger. Cooking is as much of a hobby as hacking or being a "maker." Plus eating food is way more than just satiating hunger. I don't like cooking but I loving eating and drinking.


> The reasonable thing to do with that information, would be to surveil them further, search their house, or arrest them. Not assassinate them without a trial.

Which seems to be exactly the case. Ahmad Zaidan, for example, wasn't assassinated. In fact even if the algorithm was 100% correct - it makes no sense to assassinate people solely based on whether they are terrorists or not. A lot more can be gained from surveilling some of them, etc'.

So there is really nothing at all to suggest that those algorithms alone are used to create kill lists.


> So there is really nothing at all to suggest that those algorithms alone are used to create kill lists.

The first line of the article actually suggests this: 'In 2014, the former director of both the CIA and NSA proclaimed that "we kill people based on metadata."'

Also, even if something does not make sense does not mean that it does not seem to make sense to a particular person or organization.


Yeah but nothing going on in the national security state makes much sense. The CIA drone program needs targets. It devours them. When they ran out of real intelligence they started killing people they didn't even know the name of because they "looked like terrorists", check out signature strikes.

Those drones are going to be tasked no matter what. If the quality of the input intelligence is crap, well, good thing it's all top secret right!


Wait wait wait. So they're killing people based on a model that fails Statistics 101!? No accounting for base rates? No cross-validation!?

God fucking damnit, why the fuck does our government think killing should be cheaper and easier than healing!? We put new medical treatments through decades of difficult scientific tests, but have none at all for strictly invalid ways to kill people?


I do not think you completely understood the article.


The models as described in the article do seem to have glaring flaws. However, the article did not show that anyone has been killed based on the models alone.


I'm pretty sure I did. Random Decision Forest with no regularization, no probabilistic accounting for the base rate of the target class, and with the test and train sets mixed rather than performing proper cross-validation. They've almost definitely overfit, and almost definitely killed someone who wasn't actually a terrorist.


Perhaps someone who knows the topic can say whether this mathematical phenomena of "adversarial examples" for neural networks can be translated to random forests: http://www.kdnuggets.com/2015/07/deep-learning-adversarial-e...

Barely perceptible changes to an image cause it to be misclassified by neural networks which never saw the image before (this is important, because it rules out simple overfitting). As a non-expert, this suggests to me that, at least for some algorithms, the reasoning "statistical algorithm estimates 99.999% probability of guilt" implies "guilty beyond a reasonable doubt" is unreliable at best.

There is great potential here for safe, effective government if machine learning output is only ever input for humans with common sense, life experience and the ability to interact with the world.


With respect to adversarial examples for random forests, that link states:

   Myth: Deep learning is more vulnerable to adversarial examples than other kind of machine learning.

   Fact: So far we have been able to generate adversarial examples for every model we have tested, including simple traditional machine learning models like nearest neighbor. Deep learning with adversarial training is the most resistant technique we have studied so far.
But the larger point is that adversarial examples are just one demonstration that algorithms are still quite primitive. Nobody should be killed on the basis of a statistical algorithm alone (and to be fair, the article does not show that anyone has been).


It would be difficult to create such persuasive adversarial examples in this case. Humans can recognize images well, so it's obvious for us that those adversarial examples are nowhere near what neural net thinks they are. It would not be as obvious with a list of phone calls.


Many other countries have 'Skynet' programs https://en.m.wikipedia.org/wiki/Skynet


Given that there are very little real terrorists in the data, any sensitive detection mechanism will need insane specificity or it will give you an enormous amount of false positives.


> But you are going to get an error rate, and probably a high one in the noisy real world.

They're clearly not implementing a minimum-error rate classifier. Which isn't to defend their choice of loss function (it's not 0-1, but beyond that I have no idea); just to point out the inanity of some of the comments claiming the NSA isn't aware of base rates.


How many terrorists are there to put on the List?

For something around a ten of thousand people, at a minimum 80% of them will be innocent, just by sheer lack of actual terrorists to fill that many slots. And that's assuming a perfect fitting algorithm, with absolutely no false positives.


the more I read the details, the more alarmed I am

Just wait until the next leak confirms that the system has learned to fly the drones autonomously.


Just don't try to shut it down.

"We love you, Skynet! We just need to do some routine maintenance."


did you post this on the corresponding reddit thread as well? if not then someone literally copy pasted your comment, which I have been noting a lot more lately on there.


Yeah that was me.


Disingenuous title: NSA only collects data (SIGINT) and secures transmissions and identity of friendlies (SIGSEC). What others do with the data is not done by NSA.


Which is of course misdirection through diffusion of responsibility.

Stay classy.


>We can't even test it because there is no validation set and unreliable labels.

You can't test it, the NSA surely can and almost certainly can and did.


>The reasonable thing to do with that information, would be to surveil them further, search their house, or arrest them.

That is in no way a reasonable thing to do if you're only 50% confident in your results. A reasonable thing to do would be scrap the whole system and make something more reliable than a coin flip.


Characterizing it as a coin flip is unfair I think. If this system can promote individuals to attention out of millions of targets, telling us that this person out of all the millions has a 50/50 chance of being a terrorist, then that is hugely valuable information that deserves further inquiry.

Now, of course we don't know how good the rate is or what other evidence they already have or what exactly a terrorist is for that matter. But in principle, you'd be throwing out a huge amount of evidence if got rid of a system that changed a target's chance from 1 in a million to 1 in 2.


Big data analysis + mass surveillance is a frightening prospect. Of course you can train software to look for 'terrorists', but you could also train it to look for:

- whistleblowers

- minority groups (e.g. gay people, particular religious beliefs, political affiliation)

- political dissidents

- journalists whose behaviour changes

- personal vulnerabilities (affairs, mental health issues etc)

Think what authoritarian governments could (and are) doing with this capability.


- political dissidents

https://en.wikipedia.org/wiki/McCarthyism

And that's what it is used for. Australian gov. used their abilities to mark people who say against coal energy as anti-government, all they did was support renewable sources of energy. https://overland.org.au/2014/07/surveillance-of-activists-is...

and Canada: http://www.theguardian.com/environment/2013/feb/14/canada-en...

- minority groups

FBI was spying on black-rights activists, since ever. https://www.eff.org/deeplinks/2014/11/fbis-suicide-letter-dr...

- whistleblowers

No example needed.

- journalists whose behaviour changes

https://www.opendemocracy.net/ourkingdom/duncan-campbell/gch...

Campbell in '70s discovered ECHELON, NSA admitted he was their personal target since then.

- personal vulnerabilities (affairs, mental health issues etc)

https://en.wikipedia.org/wiki/SEXINT https://en.wikipedia.org/wiki/LOVEINT


>Australian gov. used their abilities to mark people who say against coal energy as anti-government

Even the partisan sources you've linked don't support that particular assertion AFAICS. It appears that gov.au placed a group of activists with the potential intent to disrupt energy supplies under surveillance.

'Green' activists suffer from a bad case of noble cause corruption which prevents them from understanding that things they think are wonderful and necessary - like disrupting power grids - are well within the scope of what other people would consider to be terrorism.

There is also no suggestion that any ML or mass surveillance was used in targeting them. Targeted, intrusive surveillance was authorised against them.


I don't know any good sources about it. I heard about it somewhere on reddit probably and it stuck in my head, there are more different kind of sources if you need more:

http://www.theguardian.com/environment/earth-insight/2014/ja...

http://www.abc.net.au/news/2012-01-07/brown-slams-spying-on-...

http://www.smh.com.au/environment/afp-spies-targeting-green-...


I would call it "mass discrimination". Because, in principle, that is what this amounts to: you're using a few traits of a person to generalize into a whole group.

I find it strange that nobody seems to identify this. It is almost as if, because it is done on a big computer and no human is involved in the processing, suddenly it is not discriminating anymore.

But humans programmed these computers in the first place.


This has been identified, posted to HN, and discussed.[1]

[1] https://news.ycombinator.com/item?id=10872852


Israel allegedly identifies homosexual Palestinians based on mass surveillance data and exploits them for it [1, 2]

[1] http://www.haaretz.com/opinion/.premium-1.617280

[2] http://www.vice.com/en_uk/read/gay-palestinians-are-being-bl...


Wow. To add on top of the stigma of being gay in a homophobic culture, now they're going to be targeted as potential traitors/informants too. That's just... wow.


Big data analysis + mass surveillance is just more efficient than previous methods.

Let's imagine: 1. citizens have right to fair trial 2. terrorist hunting is not done by police, but by military intelligence organization that cannot arrest, attack or sue anybody, just gather data and inform police/military or prosecutor 3. that military organization is supervized by democratically elected guy (president) 4. Law that defines clearly when such information can be used, and defensive attorneys are not excluded from using it.

That organization being part of military is inportant. If it's actually clandestine organization with priority of defence, it should be unwilling to reveal it's capability to foreign players. Unless it's actually matter of national security. This is probably happening already with multitude of military forces and we never hear about it. Shit only starts when police starts doing same stuff while disregarding fair trial. SS-troops and KGB we're pretty good at fucking up people even prior to mass surveillance and big data analysis.


To be honest, being someone interested in lots of things, just based on my browser history you could also put me in the terrorist box.


It seems like a safe bet that they will try to learn as much about people as possible. Not so different from classifying them into people who are likely to buy a new car or nutritional supplements.


But lots more churn and breakage.


>- minority groups (e.g. gay people

So there was this infamous incident of a Netflix machine learning competition for movie recommendations ...

Guess what somebody was able to do.


I can guess what you must mean but I can't find any references about it. Got a link?


This story is stupid.

I'm sorry. No fan of the NSA, but the premise behind it is completely ridiculous.

There is zero evidence of the repeatably asserted idea that the list this tool generates is any kind of kill list.

It's a tool that generates indicators of people that may be worth looking at when trying to find couriers. That's a very specific subgroup of terrorists, and I find it entirely unsurprising that a journalist would be falsely flagged as journalists have statistically unusual travel habits (Clearly labeling him as "member of Al Qaida" is unjustified by this evidence though).

Also, criticizing the NSA on their knowledge of statistics seems unwise. The NSA is many things, but "bad at Math" isn't one of them.

Read the information yourself, and come to your own conclusions.


> Also, criticizing the NSA on their knowledge of statistics seems unwise. The NSA is many things, but "bad at Math" isn't one of them.

Surely this would also apply to NASA, but then you go and read Feynman's analysis of the Challenger disaster and it doesn't seem at all the case.

I think the really unwise thing would be to ascribe abilities to an organization based on the abilities of individual members as politics starts to take over.

http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roger...


You may of course be right, I would find it surprising in the extreme if people were killed based purely on output of an algorithm. However we have no assurance that this tool was not used to generate kill targets without any human input. The NSA operates without any meaningful oversight and if they decided to use algorithms to generate kill targets, they would go right ahead and do it.


The thing is that the NSA doesn't kill people. The CIA and DoD run the drone program, and they both do their own due dilligence (how good is another conversation altogether) before carrying out their strikes. Also, most strikes have to be authorized from the top, up to and including the secretary of defense and the president.


> However we have no assurance that this tool was not used to generate kill targets without any human input.

That's absurd. We also have no assurance that the NSA isn't communicating with aliens to generate kill lists of people who will oppose the alien invasion of 2021, though we do have common sense to filter out such absurd scenarios. "We have no assurance that this isn't so" isn't an argument.


I'll bite. The difference is that communicating with aliens requires positing the existence of aliens, whereas simply generating a kill list is already something we know they do, though we don't know how it works.

It isn't "common sense" to filter out entirely obvious actions the NSA would take, and given the scale of disclosures from the Snowden files you ought to have already expanded your imagination away from "they'd never do that" to "yes, they already have the technical capability and have done similar things in the past, lied to Congress about it, dismantled oversight, lobbied against oversight, and otherwise deceived every mechanism of oversight put in place."

The fact that 2,500 to 4,000 people have been killed in Pakistan according to a completely opaque process for classifying targets and blasting them from the sky--a process that operates entirely outside any law--ought to be considered absurd, shocking, frightening, and soul-crushingly inhuman. But it's not. We're arguing about with each other about bullshit.

Focus.


The fact that 2,500 to 4,000 people have been killed in Pakistan according to a completely opaque process for classifying targets and blasting them from the sky--a process that operates entirely outside any law--ought to be considered absurd, shocking, frightening, and soul-crushingly inhuman. But it's not. We're arguing about with each other about bullshit.

I'm incredibly appalled by that. That's why stories like this are dreadful. They overreach in their conclusions, and will be easily denied by people involved in the programs who will then produce evidence to show that this particular program does exactly what it tries to to: identify terrorist couriers.

That denial and evidence will then discredit all the sensible arguments about the drone strike program.

Don't believe it? There's a discussion down-thread where someone is equating this list with the US Terrorism Watch Lists. Their clearly not the same thing at all (once glace at the slides shows you that), but they demand evidence. Of course, I can't show evidence that will convince them, but at some point an agency will, and they'll show exactly how the ist in this article (or some other list) is very accurate (I'm sure there is some list that is) and that will discredit the whole argument against the watch lists.


* who will then produce evidence to show that this particular program does exactly what it tries to to*

No they won't, because they've been challenged to do such things repeatedly in the past and always failed.

These people live in a foreign country and it's not like the US Govt dispatches a bunch of detectives and lawyers based on the results of this ML model. Get real. The intelligence is handed off to the CIA without revealing how it's generated, the CIA then says "we got a list of terrorist couriers from the NSA, let's go get em" and boom, off it goes.

This is all incredibly well documented.

There is simply no mathematical way the program described in the article can be accurate, that's what the entire article is about. So I don't see why you have such profound faith in them. It's quite clear they're a bunch of maths geeks who have a single hammer and will use it to hammer any US foreign policy problem regardless of how much it resembles a nail or not.


> The intelligence is handed off to the CIA without revealing how it's generated, the CIA then says "we got a list of terrorist couriers from the NSA, let's go get em" and boom, off it goes.

> This is all incredibly well documented.

Then feel free to submit at least one source (or better yet, several) for this "well documented" fact that (a) the NSA does not due any due diligence into suspected terrorists after they get picked up by this meta data process; (b) the NSA does not indicate in any way to the CIA how the list was generated; (c) the CIA uses this information without performing any of its own due diligence on the targets; (d) the CIA then goes and kills everyone on the list without any further approval from DNI, SECDEF, POTUS, etc.

I'm no drone program apologist but let's get real here to suggest that there's no intelligence or due diligence into these operations is willfully disingenuous at best.


Furthermore, even if the algorithm was generating kill lists directly there seems to be the implicit assumption that that would be worse than having some human "yes-men" who are directly beholden to orders of their immediate superiors and surely care more about their career advancement than bombing random Pakistanis in charge of the process.

It's the same fallacy as being overly paranoid about the safety of self-driving vehicles, the system doesn't need to be perfect, it just needs to suck less than what it replaced, how did the CIA/NSA come up with such lists before the advent of computer analysis?


...a good reason not to kill anyone without more than the decision of an AI or a council of human bureaucrats and politicians.


Informers. Of which there are a limited number ... and of course they are themselves not entirely reliable which is why virtually all developed western countries have scrapped the death penalty.


> The NSA is many things, but "bad at Math" isn't one of them.

I agree, and with the pedigree of cryptology that has come from the NSA (and with alumni like Knuth), that's very well established.

But non-technical people tend to have a habit of taking technically brilliant (and often specific) things and using them in a way that they were never designed or intended.


> alumni like Knuth

To the best of my knowledge, Knuth never worked for the NSA. (He did briefly work for the IDA CRD -- https://en.wikipedia.org/wiki/Institute_for_Defense_Analyses... -- which in turn did some work for the NSA, but that's not the same thing.)


How can we review these lists/tools and reach our own conclusions?

References please.


I'm completely unconvinced that it isn't reasonable for an agency to keep lists and perhaps tools like this secret.

Police don't let criminals know when they are being watched, and that is generally seen as reasonable.

Abuse is a problem. Not sure what the solution is, but I'm not sure making this type of list public is the solution.


>> Read the information yourself, and come to your own conclusions.

> How can we review these lists/tools and reach our own conclusions?

Previous poster was referencing your charge that critics review the information.

The solution will involve some sort of cleared-but-not-invested ombudsman groups, whose job it is to be specifically critical of technology tools, who are given authority equal or greater than the organizations they review, and who are rewarded for performance opposite or conversely somehow to the groups they review.

These groups would have to care about personal liberty, have the authority to actually halt intelligence gathering organizations when necessary, and be rewarded/incentivized for discovering and stopping poor practices.

Congress or maybe the Executive branch has the authority to enact something like this, though it'd take some strong wills to do it.


https://www.youtube.com/watch?v=UdQiz0Vavmc "We kill people based on metadata" - Michael Hayden, ex NSA-Chief.


Did you not read the Drone Papers documents? They go into much more detail and generally back up what this article is saying.


The point of using computer models to 'find' terrorists isn't necessarily to find terrorists, it's to abdicate moral responsibility. Everyone can just say they were 'following the model' when it turns out they were wrong. You can't execute a computer for war crimes.


We know that they do strikes based on circumstantial evidences (signature strikes):

http://www.theatlantic.com/international/archive/2015/04/war...



The thing that stood out for me is "Somewhere between 2,500 and 4,000 people have been killed by drone strikes in Pakistan since 2004".

WHAT!? That is so wrong! This stinks and we're making fuss about mathematics. Just read that sentence again.


Exactly.

In which freaking world are assassinations without due process legal or even desirable? The mere fact that we are debating whether someone was or wasn't assassinated is a perversion in itself.

Arrest them and try them in a court of justice. End of story.


I will not, in any way, defend the actions of the current programs being run.

However, your statement:

Arrest them and try them in a court of justice. End of story.

Is not at all easy to accomplish. The government of Pakistan doesn't control a significant part of the country.

Add to that the corruption at all levels, and it would be difficult to pursue an arrest warrant effectively. If you try to negotiate your way into the areas where the suspects are, it will take a while (and likely a lot of bribes) and they will hear of it and go into hiding. If you want to ignore all the formalities, and just rush in and grab the suspect, you're effectively mounting a small invasion, which the central government AND the tribal leaders will oppose. So you're back to shooting people again.

Again, I'm not saying what the USA is doing is a good solution, I'm just saying I'm not aware of a good and effective solution at all.


Arrest them? They were in freaking Pakistan. Why would you arrest them? You guys are going crazy. Don't be surprised when you travel and realize the world hates you.


He obviously means "send intelligence to the government of Pakistan and ask them to investigate".


Thank you for stating the obvious.

Share information about terrorism suspects, arrest them, bring them to justice. Don't go all Rambo, shooting first, asking questions later. That only perpetuates terrorism - "you killed my innocent parents, I'm shall take my revenge" - and the circle of violence continues.

(Plus, it's already been proven that anyone shot is marked as an "Enemy Killed in Action" posthumously, just to cover the perpetrators asses.)


I already know parts of the world hate me, because I come from the country that engages in drone strikes.


I think the parent means arrest the criminals responsible for killing thousands of people with drones based on a faulty algorithm.


I meant both, in fact.


We're making a fuss about mathematics because this program is sold on the basis of the infallibility of mathematical tools. Especially for non-technical people of a certain age, this is a very persuasive argument (just watch any movie with a computer in it made in the 60's or 70's - computers are magical instruments that find truth where mere humans cannot).


This story is hugely important. It gets to a deep question everyone here should ask themselves -- beyond immediate concerns of salary, equity, and learning value, is my specific work making the world a better place or a worse place?

This story goes to the core of ethics in engineering.

It's all of 9 hours old and has 341 points -- yet it's already off of the front page where nobody will see it. You could check HN literally every day and still easily miss this story.

Meanwhile, the front page is full of unimportant links to obscure tech trivia, many of which have less than 20 points.

We know that HN automatically penalizes submissions containing certain words, including "NSA", in the title. Certain prolific HN users have also said that they "automatically flag" submissions they consider "political".

But I really think HN would be a better place if the front page cycled out a little slower, if stories like this were not suppressed, if they got at least one day's worth of attention and discussion.


"Obliterated a wedding because people where celebrating by shooting in the air."

This is the definition of evil.

How and why is the Pakistani government allowing this program to operate within their territories?


Because most of the strikes are in the Tribal areas near the border with Afghanistan where the writ of the Pakistani state doesn't really run. The Pakistani military has received some bloody noses when trying to operate in the area. Even Wikipedia is happy to classify the whole thing as a war[0]. Over 6,000 members of the Pakistani security forces have been killed, 20,000 civilians and 35,000 "insurgents".

Pakistan gets significant military aid from the US to help put down an insurgency by groups that would overthrow the state. On a realpolitik level they also get to blame the US when a strike goes wrong whereas if they were dropping their own bombs they would have to put up with the blame. Mistakes always happen in war, I'd suggest however that a drone strike has significantly more time to conduct pre-launch checks than an F16 pilot does.

[0] https://en.wikipedia.org/wiki/War_in_North-West_Pakistan


But the end result of all that is just that it generates even more hostility towards the US and feeds terrorism. The US is creating its own enemies this way. Again.


Cynically speaking, this is win-win for everyone involved. Except for the innocents who are killed.

The US gets to debug and optimize their assassination program "on brown people that noone cares about".

Pakistan and US also get to claim that they are "fighting terrorism". This breeds resentment and new generations of terrorists to be "fought" in the future (divide and conquer 101).

Plus, who knows when and where the droid program might come in handy in the future...


droid program?


Among other reasons, Pakistan security forces receive a lot of military aid from the US State Dept, including weapons and materials (e.g. bulletproof limousines) used by the same Pakistani officials for their own protection. I expect US support also helps as a deterrent in their problems with India as well.


My guess would be because Pakistan has had good relations with US since it's independence.

"The United States today provides extensive economic, scientific, and military assistance to Pakistan... and is Pakistan's most generous donor of foreign aid"[1]

[1] https://en.wikipedia.org/wiki/Pakistan%E2%80%93United_States...


> My guess would be because Pakistan has had good relations with US since it's independence.

More often not.

There was a period from the 1970s when Pakistan was embargoed because of its nuclear power + weapons programmes, mainly in cooperation with France. This pushed Pakistan into the Chinese camp for arms supplies where they have remained firmly lodged ever since.

During the 1980s the USA did supply some silver-bullet assets like F-16s, mainly to atagonise the Soviets in Afghanistan, but later deliveries were again embargoed and the USA only recently reimbursed their purchase fees.

The USA-Pakistan relationship is one of the text-book examples of realpolitik.


To add to that, DoD just approved sale of 8 F-16s to Pakistan (http://thediplomat.com/2016/02/us-approves-sale-of-8-f-16-bl...). The ball is in Congress' court to decide if this actually goes through... but ya. Pretty much the definition of realpolitik.


>How and why is the Pakistani government allowing this program to operate within their territories?

Because it's a powerless country, and as always those are at the mercy of the top dogs.

At best they can ask for some bargains (foreign aid, some favorable diplomatic stance, etc).


It's because we're killing their enemies.These are strikes on the TTP not the Afghani Taliban.


They could always not allow and be left to deal with Afgani collapse on their own. And with soaring and increasingly nationalistic India under Modi.


You are missing the main point of the article.


I am not. I am just at a loss why a foreign sovereign would give the US, or any other country, the license to randomly kill their own citizens.


If they did not, what could Pakistan do about it? The U.S would do so anyway covertly, and the Pakistani government would be seen to be weak, since they were powerless to stop it.

So the Pakistani government 'allows' it to at least appear in control.


They have nukes. They could easily institute a defense doctrine of nuclear retaliation tomorrow, and the terror would stop. But the reason they don't is that they are bought and paid for, both as a country via foreign aid and as individual government members via bribes (or "speaker fees", as you guys call those now).


Let's run with that thought. Let's say Pakistan committed to using nukes in retaliation for US drones shooting their citizens, what are they going to shoot at exactly?

They're supposedly developing longer-range capability capable of hitting some of Europe[1], but right now they might be able to hit some US military bases in the middle-east[2].

I'm not saying Pakistan isn't corrupt, but pretending that they could simply switch from their current US-friendly policy to a strategy that's openly adversarial to US-interests seems to blatantly ignore the realpolitik of the situation.

1. https://en.wikipedia.org/wiki/Taimur_(missile)

2. http://news.bbc.co.uk/2/hi/south_asia/3545775.stm


You are right. I haven't considered that. They only care about the standoff with India, so it makes sense that their range only about covers her.


The U.S. pays the Pakistani government for the right to use drone strikes against innocents who are prescribed as terrorists by a computer algorithm, which encourages hatred and fear and creating the very terrorists justifying U.S. giving the money for the Pakistani government to buy weapons from the U.S. to field an army to fight a war with.


Oh come on "machine learning algorithm may be killing thousands.." just put a "may be" and your BS assertion becomes more plausible? By looking at this, people are having and idea that attack drones (it is rich that article puts one picture of it in the beginning) are loaded with such software and killing by looking at the result of a classifier. Or some super computer gives you a name and says "exterminate".

Apparently these tools only gives operators some clues and save their time. If it is a false positive, probably they just ignore it. Of course, obtaining the information is a different story.


the problem is you don't know that 'If it is a false positive, probably they just ignore it.' -> 'probably' is a fucking optimistic view. AND how the hell do you know this is a false positive ? some algorithm tells a soldier that this person is probably a terrorist - what happens then ? do you think they bother to verify this by spending more money ?

In my opinion it is most likely that, the people using this software send the output of this algorithm as a recommendation up the chain and some idiot decides to be 'safe' and recommends execution. because they don't have any accountability. I don't think you would be this relaxed in your opinion if these f-ing drones fly over your head - it is just that most people in the US do not care what happens to some idiot in Pakistan.

from the reports disseminated by the state department they are not even sure how many people they killed ? it is reported as between 2500 and 4000. Isn't this insane ? I have no idea in my mind that US is killing people indiscriminately using drones - and the word is not probably - it should be 'definitely'. i'm sure drones are turning people into terrorists more than they are killing terrorists.

don't tell me this is a bs assertion before US can give the name of every body they killed with a drone and the justification of it.


AFAIK for such decisions human information/intelligence is required. And there are often screw ups/bad decisions. No need for bad machine learning algorithms for this.


Your optimism is misplaced. The USA routinely drone strikes people they have no intelligence on whatsoever on the basis of nothing more than 'they acted like a terrorist', these are called signature strikes, i.e. they matched the "signature" of a terrorist.

Organisations that do this so frequently there's jargon for it are absolutely not going to be slowed down by requiring human intelligence (which is itself full of false positives and duplicity, see how random people were sold to US soldiers as "terrorists" to collect the reward money).


> If it is a false positive, probably they just ignore it.

And how do they know it's a false positive?

Indeed, if they really knew that, then they would have some more training data.


I am not defending peoples lousy/callous decision processes for fatal actions. But telling that bad machine learning algorithms are responsible for killings is just stupid.


If putting a huge number of innocent people on a "probably terrorist" list secret intelligence agencies use to pick targets to secretly execute without trial isn't a big deal, would you be okay visiting rural Pakistan for a week with your name and face on one? Let's assume you're ethnically Pakistani.

I mean after all, they do human intelligence checks, right? So you'd be safe.


I was wondering exactly how the NSA trained models to detect terrorists, and was surprised by the level of detail in this article. So the NSA performs classification using a Random Forest and about 80 input features? Huh. That actually sounds a little too similar to a Kaggle contest for my liking, but I wasn't expecting that much quasi-technical information anyway.

It seems a little silly to write this whole article based on a few powerpoint slides from years ago. Even as an amateur practitioner, I can see several obvious things that could be changed about the presented methods. I'm sure what the NSA is doing now in this regard is much further along than what was portrayed in those documents. Whether or not they should be doing it is a discussion for others to have. But it is interesting to learn a little more about the details of truly high stakes machine learning.


Reminds me of this dialog from Spectre (http://www.imdb.com/title/tt2379713/):

> Have you ever had to kill a man, Max? Have you? To pull that trigger, you have to be sure. Yes, you investigate, analyze, assess, target. And then you have to look him in the eye. And you make the call. And all the drones, bugs, cameras, transcripts, all the surveillance in the world can't tell you what to do next.


> ... possibly resulting in their untimely demise.

I get increasingly annoyed by these kinds of euphemisms in articles discussing US actions. As if it was something humorous.

Should be something like: ... possibly resulting in them getting assassinated by US military.


We're killing people in a country were not even at war with over statistics. Not evidence, but statistics. People who statistical might have something against us. Since when has this been grounds for killing someone? This is utterly terrifying.

Just imagine if someone decided to do this to us. How does Pakistan feel about all this?

This is approaching Auschwitz levels of evil.


They don't actually care if the person is innocent. The point is to intimidate their population. They use this metadata to fit into the system of law so they can cover their ass.


The argument was that in the US, they "only" collected metadata. No real phone calls. No big deal...

The power of their use of "only" metadata in Pakistan is frightening..


The founding fathers put due process into the constitution / bill of rights because it's a fundamental check on government bureaucracy run amok, and prevents physical harm from coming to people as the result of a capricious executive whim.

Now, we learn that the executive has excused themselves from following this process (which applies to Americans, some of which these victims may be, even when they are not in America).

The farther open we pull the NSA lid, the more revolting discoveries we find.

The individuals behind these decisions should be identified, and tried in the court of law.


"Under the random selection of a tiny subset of less than 0.1 percent of the total population, the density of the social graph of the citizens is massively reduced, while the "terrorist" cluster remains strongly interconnected."

There is no indication in the slides that the feature calculations are done on the smaller subgraph. The above comment doesn't hold if the calculations are done on the entire graph and then the 100k are sampled for training.


I see lots of doubt about the ridiculous nature of this program described in this article. People saying it's not possible.

Are you sure? https://theintercept.com/2014/02/10/the-nsas-secret-role/


Funny how in the title, the culprit is the algorithm and not the organisation.


The US, UK response to terrorism is so disproportional, odious and sinister one wonders whether this is infant a 'response' or something far more evil cooking below the surface.

This kind of technology necessitates mass surveillance, and offers you little more beyond the faint possibility that crime can be predicted. This is bogus science.

If you let your mind entertain the idea you can predict crime, then you are already at the thresh hold of a stifling surveillance state. And why are we even assassinating people. What happens to due process? One by one all these fundamental principles are set aside, massive and dubious mass surveillance infrastructures are being built, doublespeak is comically rampant.

One can begin to understand Snowden's urgent need to act, but what about the moral compass of all the people in the NSA and who support the US and UK security apparatus? People need to act. Surely they cannot suddenly subscribe to values we have aggressively demonized for more than 100 years.

Who would have thought that the US and UK are now the rogue states and other countries who are far more secure and sure about their politics need to begin to isolate themselves from these dangerous totalitarian instincts.


The more I read about US activities abroad, the more I realize Chomsky's right. Are sovereign countries supposed to tolerate a certain level of drone strikes by the USA (4500 people in Pakistan killed in the last decade) based on -- now revealed -- machine learning hunches?


There are two issues here, firstly some dramatic overfitting of the model, and secondly that all-too-familiar garbage in, garbage out fact, which is even more relevant when dealing with data.


They automated the decision to kill people and called it Skynet? That sounds awfully familiar. When can we expect assistance from the future to take the NSA people out?


I feel infuriated while reading that they named it Skynet. They are clearly the ones creating terror, that's so absurd! I can't believe that they do not have the understanding that this very act is what propagates wars and keep them going on forever. As it cannot pass as ignorance, I take as malicious the people who feed this system.


A related approach that identified suspects based on their banking history was outlined in SuperFreakonomics. The chapter is reproduced in this review:

https://www.goodreads.com/review/show/90622011


Hmm:

Behavior-based Analytics: - Low [cell phone] use, incoming calls only - Frequent handset changes - Frequent detach / power-down

Visits to other countries - overnight trips - permanent move

I guess that makes me an extremist militant, also.


Even before reading TFA that headline stands out like poke in the eye. An algorithm has no agency - moral or otherwise. Algorithms don't kill people, people kill people.


Why are terrorists still using SMS if it is lethal?


Would have appreciated a "top secret" flag in the title. The slides in the article are not declassified...


If that system skill kills enough terrorists to make them scared and limit inflow if new recruits into that system than why not? Obviously rest of the population of Pakistan pays their price, but what they expect if they can't (or really, don't want to) shut down terrorist activity in their own country themselves?


It's ignorant to assume any targets revealed by this program wouldn't be given a human intelligence analyst to verify accuracy before risking millions of dollars on a predator strike and the potential risk it was bad information. Enough said.




Non-Bayes analysis kills.


The important part is that it's helping them kill more people, so it means they're more productive. /s


So yes, a state would not be much of a state without some form of military. But why do we need separate states?

Edit: this is a genuine question about the practicality of a concept (the "nation-state") that was invented during the Gutenberg era of the printing press, and how such a concept has become impractical in the age of instantaneous international interconnectivity and economic globalization.

Downvoters: How are arbitrary divisions drawn on a map anything but counterproductive to synergy and efficiency in the age of globalization?


We detached this subthread from https://news.ycombinator.com/item?id=11109756 and marked it off-topic.


You really want to know? Because of regression towards the mean.

Having a single state would mean that the US couldn't afford to have a significantly higher standard of living compared to the rest of the world. In fact, 1st world countries would have to meet 3rd world countries somewhere in the middle.

(Assuming you are living in the states) Would you be willing to live on a wage of $5000 a year?

Edit: see here https://en.wikipedia.org/wiki/List_of_countries_by_GDP_%28no...


You're still thinking in terms of 19th century paper-based processes when you attempt to estimate globally integrated wealth in terms of a nationally fragmented paper-based information system ($). Also, GDP is not indicative of net wealth, as it is a measurement which implicitly asserts the broken window fallacy.

Over the long term, breaking down these legacy barriers to trade and freedom of movement would necessarily uplift everyone economically by principle of synergy.

Synergy: the interaction of elements that when combined produce a total effect that is greater than the sum of the individual elements, contributions, etc.

>I'm going to give you an important kind of a picture. I hear a lot of people say "I don't like machinery and technology, it's making a lot of trouble." So we're going to take all the machinery away from all the countries of the world, all machinery, all the tracks and the wires, and the works and we're going to dump it all in the ocean. And you will discover that within 6 months, 2 billion people will die of starvation having gone through great pain. So we say "That's not a very good idea lets put all the machinery back where it was." Then, we're going to take all the politicians from all the countries around the world and we're going to send them on a trip around the sun, and you'll find we keep right on eating. And the political barriers now... scientists say very clear you could make the world work and take care of 100% of the people at a higher standard of living that anyone has ever known despite the increasing population, but you can't do it with the barriers, any more than you can try run a human organism with a wall between the ear, the eye, and the stomach. It is an organic whole, it is total industrialization.

-Buckminster Fuller

>Human history can be viewed as a slowly dawning awareness that we are members of a larger group. Initially our loyalties were to ourselves and our immediate family, next, to bands of wandering hunter-gatherers, then to tribes, small settlements, city-states, nations. We have broadened the circle of those we love. We have now organized what are modestly described as super-powers, which include groups of people from divergent ethnic and cultural backgrounds working in some sense together — surely a humanizing and character building experience. If we are to survive, our loyalties must be broadened further, to include the whole human community, the entire planet Earth. Many of those who run the nations will find this idea unpleasant. They will fear the loss of power. We will hear much about treason and disloyalty. Rich nation-states will have to share their wealth with poor ones. But the choice, as H. G. Wells once said in a different context, is clearly the universe or nothing.

-Carl Sagan, Cosmos


Please try to make your points without quoting random people. Quotes are not sources, they are not facts, they're just something someone said.


Quotes do have the notable advantage of still being remembered and recognized many years after their original utterance or publication. This is often because they are a more concise and eloquent expression of a particular thought than the people repeating the quotes can manage themselves on short notice.

And they have the additional rhetorical impact of appeal to authority. While this does nothing to establish the validity of facts, it works wonders with garnering emotional support from your audience.

Don't fall into the trap of arguing with rhetoric. That's bringing a knife to a gunfight. The Internet is not a formal debate forum. The audience is filled with humans, not robots. Sometimes, if your goal is to sway people to a particular point of view, or even just to force them to acknowledge that other points of view exist, it is easier to take rhetorical shortcuts. Saying "that's cheating" doesn't help. We all know it's cheating. We just don't care, because it allowed the point to be expressed in a quicker, more entertaining way.


If you think the thoughts of great scientists and engineers like Buckminster Fuller (Former president of the Mensa Society) and Carl Sagan are unwarranted in a place like hackernews then you should find a new forum. Do you have anything to contribute to this discussion?

Edit:

"The argument from authority (Latin: argumentum ad verecundiam) also appeal to authority, is a common argument form which can be fallacious, such as when an authority is cited on a topic outside their area of expertise, when the authority cited is not a true expert."

Please do tell who might be a true expert on science and technology as it relates to societal organization, if not Fuller or Sagan.


Since you're a fan of linking to Wikipedia articles about fallacious arguments, I'll just leave this:

https://en.wikipedia.org/wiki/Argument_from_authority


That doesn't apply, and you know it. The fallacy is constructed as "A has said B, A is great, therefore B must be true".

The GP never stated that B must be true. Simply quoting somebody is not in itself a fallacy.


[flagged]


The title actually reads like your usual clickbait.

Unfortunately, it is also tragically accurate, wrt the info in the article.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: