Keep in mind that this DPI system, besides making it easier to monitor people's communications and even censor them, would also make it very easy for them to identify the type of traffic that goes through the pipes, so they can know exactly how to charge it differently, which brings us to another one of ITU's proposals, which is to kill net neutrality and charge for "premium services" like watching Youtube, or using other type of P2P traffic (Skype, WebRTC, torrents, etc):
They want to do this, they say, to help "grow the Internet" (hasn't the Internet grown fast enough without their help in the past 20 years?), despite the fact that evidence suggests that the sender-pays system would slow down the growth of the Internet, not make it any faster (research paper):
Note that a kind of sender-pays is already used in practice as all big content providers pay for CDN on a per GB basis, and the CDN company in turn pays for bandwidth.
Sure, but this was the case before CDNs as well. Unless you are a Tier 1 network you usually pay a fee to your upstream carrier.
However, the ITU would like to charge across many networks and discriminate based on the type of service provided.
So if data from your network A reaches the customer through networks B, C and finally D, then D would like to charge you to deliver it and not slow down things artificially.
But that is what CDNs do. A content provider, such as Apple, will pay both for their upstream carrier access, AND for the CDN access. And the CDN will effectively pay each network on the way to the destination, by renting their lines and hosting servers at their PoPs. CDN is a QoS service, it's just that the access networks feel they aren't getting part of that cake.
This is not worthy of "the sky is falling" levels of panic.
My experience with standardization efforts is that they generally run well behind the technology innovators. DPI has been around for a while. A DPI standard (or series of standards) out of the ITU will simply make public the baseline expectations of vendors and users of DPI systems.
On the other hand, CALEA has been on the books for over 15 years, and that is the kind of thing to watch out for -- it does mandate features that provide snooping to the government on demand.
(Speaking as someone who has implemented [shallow] inspection/filtering and CALEA-type features on comms equipment for markets both in and outside of the US.)
Speaking as someone who has implemented [shallow] inspection/filtering and CALEA-type features on comms equipment for markets both in and outside of the US.)
I'm not trying to flame you here, but I really must ask: How do you live with yourself?
I know how trollish that sounds, but I seriously don't understand engineers who voluntarily work against our own ethos. It's not like this is an industry in which implementing CALEA is the only possible way to feed a family. Job opportunities are practically endless.
I've implemented CALEA type features for a major ISP. I did it because it was the law that we implement it. I'm generally in favor of following the law.
Mind you, CALEA doesn't do anything that couldn't already be done with the law. And you get more protections via CALEA than you'd get otherwise.
For one thing, there are warrants that are delivered to a judge for review which then find their way to a company attorney. Only then does any interception take place. I can live with the checks and balances that are in place under this system.
I sometimes find it incredible that people are so quick to embrace lawlessness and then expect their duly-elected governments to follow those laws to the letter. Believe me, if enough people actually cared and went to their legislature, things would change. Of course, most people agree that we should pursue criminals using all legal means.
BTW, I'm in agreement that this system could be abused. Of course, that's always been the case. Laws are just words on paper unless there are people who will enforce them. If those folks look the other way, well, you're right back where you started.
And before you go into privacy rights, let me suggest to you that the millions of people posting their personal information on FB and getting their email via Google provide a pretty substantial counterweight to your position.
But the question was not why the ISP would conform to CALEA instead of breaking it; it was why you as a programmer would take on the job of providing a snooping system, instead of some other job that does not need a lot of explanation about why it's actually not really so bad. There are reasonable answers to this, but I think it's a fair question.
(I don't agree that 'we' should pursue criminals (or suspects) using all legal means.)
So your argument is that I should not be a part of this system? Because you don't agree with it?
As a person interested in PRIVACY and LAW and INTENRET TECHNOLOGY, who would you think I'd rather have working on this type of system? Someone else? Or myself: a person who knows what his motivations are, who knows what the laws are, who knows what the implications for others are, and who wants to see things done properly.
If it's all the same to you, I'd rather it was ME. Believe me, you're lucky to have a guy like me pushing back against law enforcement when their requests get over-broad.
Remember that good people are part of this system and use their judgement to make sure abuses dont occur. I trust my judgement.
Given how much worship Richard Feynman gets around these parts, I'm wondering how people reconcile that sentiment with the fact that he worked on the development of the atomic bomb. FWIW, he seemed to be pretty OK with his role.
As I said, there are reasonable answers, and I'm ignorant of your life, but if you want my view: you should not be part of that system, not because I disagree with it, but because it's wrong. I acknowledge that I may be wrong to think so. "Oh, you think so?" is a distraction from any actual points at issue.
If you've pushed back against particular acts of snooping, then thank you for that. I did not know you're personally involved in particular acts; that's a different moral question than writing an automated system. Note that "better you have ME" is what you'd expect Nazi collaborators to say. You're not a Nazi collaborator, but it shows the at-best ambiguous advantages of this sort of involvement.
Most people trust their own judgement. According to Dunning/Kruger that's weak evidence of a problem rather than positive evidence you're doing good.
In Feynman's autobio he said he regretted keeping at work on the bomb after the Nazis went down. They were all so invested by that point it didn't even occur to them to quit. IIRC that was part of why he chose to turn down work for the feds in general over the rest of his career.
I'd rather have good people involved in "evil" systems, to at least try to balance them, than have evil people involved in "evil" systems where there's nothing but external agencies to provide balance.
And whatever the system is I'd much rather have competent people working on it.
Couldn't disagree more. Every good person should avoid working with evil systems. We should also excommunicate any people who do work with evil from our circle as best we can so they can't grow as well in their profession.
So what? What's your proposal: do nothing because that's the high road, right? Right now doing evil pays pretty good. So long as there is financial incentive to do it and absolutely no downside, why would anyone stop?
Your comment is absurd. Every country has laws and if you break those laws you will be "crushed". Is the very law itself "a fascist viewpoint"? Please put more thought into your commenting as responding to this sort of nonsense is tedious.
I'd ask you to go back and consider the definition of a "just" law.
You just suggested that people who work on "evil" systems be "excommunicated". Exactly how do you do that in the context of a functioning legal system without the involvement of a substantial majority of the people who would have to live under a system.
To me, it sounds like your argument is that the laws of the USA dont meet your liking, even though those laws are developed as a result of a democratic process. You're arguing against your own point, which sounds like nonsensical thinking to me.
You were basically saying that holding people accountable is "fascist". We are in a situation where bad people want to do immoral things and pay well for it. There's no downside. The government isn't going to make a downside because they are the bad people so it's up to us. If no one does anything we're going to find our internet locked down and every aspect of our lives being monitored.
And /again/ I'd point you to the words you used in your argument that depend SOLELY on human judgement:
accountable
immoral
bad people
All of these words mean nothing until someone assigns them meaning. Your meaning might be different from another persons. I've already come up with my meanings of who "immoral", "bad people" are and I'm comfortable with the mechanisms for "accountability". And I vote. And my vote counts just as much as yours does.
It matters if you're trying to gain popular support for your political positions. The average joe doesn't see why wiretapping has anything to do with them. They don't understand or care.
I once quit a job because one of my employer's servers became infected with some malware, spread it to client's computers and the employer refused to notify and apologise to said clients.
I sincerely hope that you grow up and take responsibility for your own actions. They are the only things we truly own. I do not believe that you are evil for what you did but I most certainly believe that you are ignorant in a very dangerous way.
I think maybe you missed the point that I actually agree with the mechanisms that are in place. I don't have any disagreements when the framework is used as it is designed to be used. Namely, within the context of due process and rule of law.
More importantly: it's somewhat presumptuous of you to suggest I need to "grow up" or "take responsibility". I stood up in a ballroom full of law enforcement and telecom executives and advocated for the legal, lawful reasons why someone might want to use a prepaid phone without requiring identification. I argued that once you got past accounting, there was no reason to associate the usage details of a phone with a particular party. I even used examples of law enforcement abuse of these facilities to make my point.
There is a lot of misinformation in this thread about what "interception" really means and how it's done. And I suspect in no small way that this is because LEAs dont want to tip their hands as to sources and methods. I won't either.
What I can say is this: if someone is capturing your traffic and has a court order to do it, it's because there is strong evidence that you're using that traffic to conduct illegal activity. A judge is the final arbiter and looks at the evidence (not collected traffic) to support that conclusion.
So...
Don't try and sell Adderall on Craigslist.
Don't steal credit cards or trade secrets via bots that "phone home".
Don't kidnap children and then send pictures of them to your friends.
The Fourth Amendment protects you against UNREASONABLE search. The reasonableness test is left up to the courts to decide on.
The problem with due process and the rule of law is that those things are enforced by humans. People invariably suffer from corruption, in particular, those in power. The less they are capable of, the safer everyone is. Governments and corporations have done orders of magnitude more harm than smaller entities like gangs (though from an absolute perspective, the separation between a gang and a government is mostly ontological). The criminals are (for the most part) not the ones we should be worrying about.
The legal system we currently have is broken at best and dangerous at worst. Many people have no faith in it's ability to be just or balanced. There are concrete reasons for harboring a distrust of the judicial system, e.g. http://www.scientificamerican.com/article.cfm?id=lunchtime-l...
Not to mention that the judicial system is 'dumb' in the sense that it's primary goal is to enforce laws, not to improve society. Having a machine which processes instructions in this way and which simultaneously has the power to ruin someone's life is a bad idea by all metrics. Those two goals (enforcing the law and improving people's wellbeing) are commonly at odds due to the nature of how human societies function and how politics influence things which they ought not to.
So how can anyone take this joke of a system seriously and expect it to be capable of policing itself when emotion is so deeply embedded in the judgements and actions it yields? The judge's decisions are emotional. The system as a whole is crafted out of an inability to deal with emotion (i.e. prisons being primarily revenge mechanisms as opposed to institutions which help people to stop being violent against others).
This is the bigger picture of the situation we find ourselves in. IMO, adding to the arsonal of weapons which this system has access to will serve to cripple, not improve society. In other words, this is all counter-productive and does not take the reality of human nature into account. It is an idealistic perspective.
So our legal system is a joke? That's news to me. I'd actually argue that it's been pretty damn effective in keeping our society from devolving into complete bedlam.
Your comments reflect all the certitude of someone who has never seen real evil up close. To suggest that a system that functions properly 90% of the time is a worthless endeavor isn't a realistic position I'm willing to argue with.
And I don't agree with your assertions. Statements like "Many people have no faith in it's ability to be just or balanced" doesn't jibe with the reality of our political economy. Every single day, hundred of millions of Americans go to work and get on with their lives. If the system was as broken as you claim, I seriously doubt we'd have the strength and standing among nations that you seem to ready to dismiss.
If you're getting all your information from magazines and wikipedia, you're bound to be misinformed about the reality of the task at hand.
And finally, statements like "The criminals are (for the most part) not the ones we should be worrying about" is simply indefensible. The justice system exists because people demand that it exist to protect them. They have agreed either explicitly or implicitly to the arrangement that we have today.
I know many members of law enforcement. They are by and large good people trying to do a hard job. A very hard job. Ask yourself if you have the courage to confront dangerous situations every single day, deal with persistent mendacity from nearly everyone you meet, and still maintain a level of professionalism and respect for individual liberty. That's an awfully high bar to set for a person and part of the miracle of our system is that it happens with such a level of regularity that we take it for granted. That's NOT the case in other countries.
> Your comments reflect all the certitude of someone who has never seen real evil up close
Real evil? Like Santa Clause and Heaven and God and Angels and Fairies?
I did not say that this system is worse than previous ones and I do not see how that ties into my argument at all. I was making a judgement on what currently exists - and it sucks.
> If the system was as broken as you claim, I seriously doubt we'd have the strength and standing among nations that you seem to ready to dismiss.
> And finally, statements like "The criminals are (for the most part) not the ones we should be worrying about" is simply indefensible. The justice system exists because people demand that it exist to protect them.
People also demand iPhones, junk food, drugs and violence. This is another logical fallacy (argumentum ad populum).
> I know many members of law enforcement.
I did not make a judgement on these people. In WW2, perfectly normal people committed atrocities because their culture and leadership dictated it. Normal people are capable of thoroughly horrible acts.
> I seriously doubt we'd have the strength and standing among nations that you seem to ready to dismiss.
You have your strength and standing because you are an empire with fingers in everyone's pie. The US has overthrown countless democractically elected leaders over the last century for profit. The world does not speak because it will be beaten for it. Why do you think the UN condems Israels actions but does nothing? This is not respect, it is fear. http://en.wikipedia.org/wiki/Covert_United_States_foreign_re...
Disagree all you want but the facts speak for themselves.
The US isn't perfect, but there is no other place in the world I'd rather live. For all our faults, people still bang on the doors to get in.
I'd put our record up against that of any other nation in the world. ANY nation. Look hard enough at everyone else and you'll find all the reason you need to hate their countries too.
Everybody is trying to get citizenship in the west because the west has destroyed half of the world. Read about why your country (and the west in general) is so rich in the first place. Read about why places like the DRC and Latin American countries are so poor and conflicted. All of this prosperity is built on the blood, sweat and tears of countless people. All empires are the same - rotten at the core. So it was with Stalin's Russia and Britain's empire and Hitler's Germany and Spain's empire and France's empire and I could go on and on.
So, sure, everything's peachy if you just keep your eyes on your house. As soon as you start to look around you'll find that your smile will drop through the floor.
You're talking to someone who has parents from Central America and Germany. Believe me when I say that there is so much blame to go around, it's unlikely that any unbiased reading of history would place it SOLELY at the hands of the west.
I'm well acquainted with the history of the world and my point still stands: the oppression of the "other" is a HUMAN problem, not a USA/West problem. As long as there are people, there will be these kinds of problems.
Tell me what country you live in and I'll list all the reasons why you are not the USA.
> it's unlikely that any unbiased reading of history would place it SOLELY at the hands of the west.
You are responding to a point that I did not argue. I asserted that in the context of recent history and current events, the west, and the US in particular, has done inordinate amounts of damage. I did not claim anything more than that. I did not state that they are the only ones causing damage, just that they are currently the most effective at it. This was in response to:
> I'd put our record up against that of any other nation in the world. ANY nation. Look hard enough at everyone else and you'll find all the reason you need to hate their countries too.
Really? China has a pretty good run going so far. How many Chinese people were imprisoned/starved during the cultural revolution? Have you taken a look at Central/South America lately? Don't even get me started on Africa.
I'd challenge you to consider the following thought experiment: if any other nation in the world was currently the sole military and economic superpower in the world, which one would you choose and why? Whose record would you suggest makes them a better candidate for that role?
Yup. People are ignorant of history and therefore of the need for privacy rights. Thankfully encryption doesn't yet depend on the public not being ignorant and politicans not being corrupt.
IMO it is the responsibility of every citizen to ignore laws that are stupid. Civil disobedience.
I realize this would mean some people might say "going 30 by a school zone is stupid!". So be it. If you disobey laws that most people believe are right, then you lose and face the consequences. If you disobey laws that most people will realize are stupid nothing is likely to happen to you.
Every feature I've implemented has been security/stability related. Inspection/filtering/shaping/limiting are absolutely critical on ISP networks. Taps/mirrors are critical to troubleshooting. If ISPs didn't deploy all kinds of filtering, the Internet would be mostly unusable.
I work at an ISP, and this is absolutely true. Sometimes our mail servers get hammered, and we need to modify our blacklist to include servers, netblocks, and/or entire countries(!) at a time. In order to know what to block, we need to be able to know who is emailing whom. Sometimes, it's one of our customers, and we can call them up and tell them their box is owned. This kind of intrusive access is only used for maintenance, and without it we literally couldn't keep the mail servers online.
If the traffic is terminating on your own servers you in no way need deep packet inspection to determine the source of traffic and its nature. Even if the traffic wasn't terminating on your machines, you don't need DPI to determine src and dst ip:port tuples. Which is all you need to do what your suggesting.
My comment isn't about DPI specifically, but a whole range of intrusive monitoring policies at ISPs. In order to determine the originator of an email, you have to read (at least) the email headers. The IP address of the last hop is not that useful in routing email.
If you need DPI to determine what traffic to drop, you are running your pipes way too hot. It's the users traffic, why do you think you're in the best position to decide to drop one website's traffic over another?
Note that I said in my original comment that my experience has primarily been implementation of shallow inspection.
Subscriber-connected edge gear is often oversubscribed. Lots of little pipes coming in from households; one or two medium-sized pipes headed to the core.
> It's the users traffic, why do you think you're in the best position to decide to drop one website's traffic over another?
All too often it's not actually the subscribers' traffic that causes problems; it may be malware.
User traffic is user traffic, if they're paying for it it should be all treated the same. I'm not saying you don't need to prioritize some traffic with QoS. I'm saying you don't need DPI to run a network.
Sincerely,
A network engineer
User traffic is user traffic is whole lot of different kinds of traffic. So there should be QoS, but how? You cannot depend on the IP Differentiated Services Field, so perhaps heuristics to deduce what the traffic is? Thats prone to misidentifying Netflix vs. CDN download thus ruining UX.
No DPI is needed.
If you treat your network's users them all the same, more power to you. Running the network and deciding what goes through it is still out of their control. I don't get why you're against those that do use DPI to run their network more easily, though.
It is not empty. I've refused a job I was head-hunted for (IT at a morally questionable firm) on a moral basis, even though pay and conditions would have been better than what I have now. If my current job started requiring me to do something I didn't agree with morally, I would leave (or refuse to do it and be forced out if necessarily).
So I think it is a perfectly valid question (albeit an uncomfortable one) to ask. There may be valid reasons for the OP to do/stay in the job that we aren't seeing. Or there may not. We won't know if we don't ask, and the OP doesn't have to answer.
It is well within the guidelines of HN, it adds to the discourse, particularly on a topic that is basically about morals/ethics. Just because it is an uncomfortable question, doesn't make it fall within what should be flagged.
I didn't read it to be empty, it actually displayed a point of view that is missing in public discourse. Questioning white wash may or may not look mean depending on the observer. It also proposes a very simple moral implication for this type of white wash. I think the white wash is real and it is very mean.
But like you, I also like spending time with what is really interesting to me. I hope I don't read void. Or mean-spirited.
No employee boycott is going to help, they'll just hire someone else. We won't see change until we start electing better leaders, or changing the system. Reduce the demand.
Probably because almost any tool ever created can do amazing things in the right hands and commit horrible atrocities in the wrong hands? And usually the positive outweighs the negative?
Can someone explain the problems with the ITU creating specifications? I thought I understood it, but all the recent excitement and anti-ITU sentiment tells me I must be missing something.
How is what the ITU does different from any standards body? They can propose standards for DPI, censoring, etc., but that won't magically make Level3 or Comcast or any particular ISP start playing with my packets.
What am I missing? Where does the stuff the ITU does somehow change the policies and actions of my ISP?
It let's individual governments "pass the buck" of responsibility. When the objection in parliament is brought up of "This seems like a bad idea" the response is "We're just doing what the ITU recommends"
I can see that working in countries who might be reliant on others for parts of their tech infrastructure. But would that actually work in the UK?
Here in the U.S., we like to be the ones creating recommendations for the rest of the world, not following them (at least not blindly). The excuse of "We're just doing what the ITU recommends" would never fly here.
Yes. This already happens with the EU; at least some of the unpopular things "forced" on the British government by the EU were actually requested by the UK in the first place.
That sounds plausible. I've heard the US do the same thing, making unpopular changes domestically by pursuing them through foreign policies, then bringing the US "world standard."
I have no specific knowledge of these negotiations, but you have to look deeper to see who suggested what.
You might find that (e.g.) AT&T (pipes) or Comcast (pipes) or Cisco (hardware) lobbied with a lot of countries and US government bodies to make this happen. Depending on the outcome, they might have a lot to gain; and doing it this way, they appear a helpless victim, just "taking orders from the UN", when in fact it was their initiative.
E.g. ACTA (and its son, the TPP) are pushed hard by Hollywood - but are mostly presented to the congress and the public as "this is an international treaty we must follow"
Poltiics and diplomacy make sure that real reasons are almost never reflected in newspaper headlines.
It absolutely works in the US. Look at some of the copyright extension stuff. There were complaints and the response from congress was "We're just normalizing with Europe"
I still haven't seen exactly how this would possibly be enforced. Just like products selectively choose features, even if the IETF or ITU says "mandatory", does not somehow create a law. The ITU can't just vote itself to tell an ISP how to handle traffic, even internationally.
They could create a standard and then let individual countries tell vendors "hey, you must comply with B.123 in order to sell in our country" -- but they can do that anyways. If a government wants snooping capabilities, you can bet every vendor will add it to get their business. It's still the government that decides if it's mandatory to turn on or not.
Again, I'd like to hear the full path from the ITU taking a vote, to my ISP suddenly snooping in on stuff. I can't figure it out.
It's not about enforcement. It's about deniability and ass-covering.
My sport (paragliding) has been destroyed by similar actions from the governing body in the last 18 months. They don't need to say "you should do X". All they need to say is "we think that maybe you should do X" and suddenly everybody falls into line and does X. It's not about enforcement, it's about not being seen to contradict a perceived authority.
ITU is lobbied by governments wanting some level of snooping.
ITU votes to allow some kind of snooping in the standards.
Government asks ISPS etc to follow the internationally agreed standard. "We'll only use the snooping stuff for terrorists and images of child sexual abuse, really."
Government uses this new compliance to the standards to get your ISP to snoop on stuff.
The governments take this circuitous route so that they as individual governments don't get attacked by local libertarians. Defeating a measure like this in one country is hard; defeating it across international treaties is very hard.
They are negotiating the underlying treaty, but I don't think this is a treaty provision. It's a standards document, adopted by World Telecommunication Standardization Assembly (WTSA), not WCIT, which is renegotiating the treaty. Basically, this is not a law, it's a spec.
I think it is a good time to start incorporating DJB's NaCl into ... everything. And also run HTTP Everywhere in the meantime. And set up opportunistic IPSEC.
Sad day.
On a related note, I suggest we stop calling the heads of state and bureaucratic organizations like the UN "Leaders" and starting referring to them by their real self appointed role, "Rulers".
Language shapes perception, and we've been using the wrong term for too long.
Isn't this an argument to to continue using the term "leaders"? Surely calling them "rulers" would, under your logic, push them further towards "rulership"?
They already know they are rulers. It's for the rest of the people.
I have heard people seriously say "Of course I prefer it with copy protection! It protects my copy!" back in the day when DRM was called copy protection. I'm sure they would have been wiser if the thing was called "Copy Restriction", because that's what it does. Similarly, I'm sure more people would realize what DRM means if it was called Digital Restriction Management (which is what it really is).
These things run deep. Who would dare oppose the PATRIOT act? Real patriots should, but none would. These terms run very, very deep.
Seriously, though, who are these people? I mean, how can I get a cushy gig like that?
You have to wonder about them as individuals, right? What path has their lives taken that they step into a room make a horrible decision like this and not retch. Is it a perverse sense of superiority over the people who are left out from the decision-making? Is it a complete lack of personality and individual thought? Or is it just an attempt to climb one more rung up the ladder, maybe move into that slightly nicer home in McClean or DuPont Circle?
From what I've experienced (as someone who occasionally rubbed elbows with such UN technocrats when I worked in international health), it's your last path that rules their lives. They are almost without exception climbers, both social and professional, much more anxious to climb the next step on the ladder than actually to build or change something for the better. Often, they start out idealistic and slowly change into jaded climbers. It's an outlook on life that would be foreign to many people here.
"US Rulers have recently made file sharing a crime carrying the death penalty" vs. "US Dictators have recently made file sharing a crime carrying the death penalty". I think Rulers sounds better (but I think "dictators" is still preferable to "leaders")
And of course you can't even read what they approved, because this extra-governmental body inexplicably restricts the text of their decisions to a nebulous list of "TIES users".
I note that they're not applying DPI to encrypted traffic per the spec, but they do note that unencrypted portions of encrypted packets will still be inspected.
The example they give is that if a PDU is encrypted, but all of the other sections of the packet are not, then only the PDU won't be inspected.
Appendix I & II are frighting in their casual use of major headings. Looks an awful lot like the fabled "tier-ed internet".
Heck, one of the diagrams even categorizes IP traffic in 4 levels: Gold, Silver, Bronze, P2P. I'm not an owner of tinfoil hats, but this has a lot of implications to a distributed web.
Appendix Examples:
I.2.1 Differentiated services based on service identification
I.2.2 Traffic monitoring
I.2.4 Traffic statistics and services-based billing
I.3.1 DPI used as a bidirectional tool for service control
I.5 DPI use case: Traffic control
I.5.3 DPI-based policing of peer-to-peer traffic
I.9.2 DPI engine use case: Simple fixed string matching for BitTorrent
II.4.11 Example “Identify uploading BitTorrent users”
II.4.13 Example “Blocking Peer-to-Peer VoIP telephony with proprietary end-to-end application control protocols”
If this gets widespread enough, they'll just inspect traffic when it leaves your VPN gateway/server. VPN is fine for public wifi, or connections between predetermined networks but you can't stretch it much past that.
Or the VPN-s connect to each other if it gets widespread enough and you get Overnet, where only minority of your traffic needs to go outside. And when that gets regulated there will be OverOverNet etc.
Only insofar as it talks about 1) partially encrypted traffic, 2) using local copies of the keys for decryption, or 3) flow identification of IPSEC. Properly done I don't see an IPSEC/L2TP VPN being vulnerable to DPI - although you will want a constant stream of "filler" packets going back and forward to thwart traffic analysis.
Otherwise, the whole thing is a disgrace and the engineers responsible for working on it need to take a long look at themselves. Dressing it up with examples of "Detection of Malware" is disingenuous, it's abundantly clear what the use case is here.
a recent Chinese paper (its author is the creator of GFW) suggests that the GFW is capable of using SVM to filter SSH tunnel traffic, at the success rate of 95%, without affecting normal SA use.
The hardest thing for me to understand is how every telco can complain of congestion, but they're perfectly willing to introduce unnecessary overhead for DPI.
Other restrictions besides bandwidth exist. For example, DPI requires more CPU and memory, which are some of the major constraints of networking equipment, especially when the equipment resides at the edge (borders) of a Tier 1 ISP.
from (http://en.wikipedia.org/wiki/Narus_(company)) we have the following "A single NarusInsight machine can monitor traffic equal to the maximum capacity (10 Gbit/s) of around 39,000 DSL lines or 195,000 telephone modems. But, in practical terms, since individual internet connections are not continually filled to capacity, the 10 Gbit/s capacity of one NarusInsight installation enables it to monitor the combined traffic of several million broadband users."
I think you're underestimating the limit to how fast DPI is or can get. The computations aren't complicated and like many kinds of algorithms you can trade space with time.
This statement might be correct (although I've never seen any evidence to support it), but it's still misleading because "forward the packet" is always going to require fewer resources than "read the packet, parse it using this set of algos, use the parse results to search your DB of shit you want to fuck with, optionally fuck with the packet, optionally forward the packet". An exception to this would be if you have big iron on the edge that protects resource-poor interior nodes. This situation is unavoidable sometimes (DDOS), but it's not what anyone should aim for.
They don't necessarily need to do anything with the packet then -- they could always pass a copy to cold storage, and then crunch the bits at their leisure in a massive data center.
A) this is probably true to an extent.
B) slurping everything to disk is impractical, and no one has that much storage, so you're back to parsing and deciding at the boundary. This "leisure" time doesn't ever happen when you save everything all the time.
C) this point seems to contradict the various claims of reasonable network maintenance I've seen; if it's trash you want to drop, you want to drop it on the floor not on your disk. If you're keeping actual traffic rather than just summaries of traffic, you're not doing reasonable network maintenance.
I think you're underestimating how much traffic can be out there.
DPI requires decoding application-layer protocols. This requires many layers of decoding. That's all well and good for one client (feature phones happily load web pages, after all), but to do it inline at the router means the router needs to be as powerful as the sum of all of the networking hardware of the hosts it routes to.
Source: at my last job, we were working on fixes for security holes that could cost billions of dollars of damages per incident, and the fixes were solvable in software, but we couldn't persuade customers of the value of buying beefy enough network equipment to actual block the attacks. Mind you, I didn't run any numbers myself on the cost of the hardware; that's just what my colleagues said.
Oh, it's obviously possible. Reading a little about Narus does not give me the impression that it's cost-effective, for any non-NSA sense of the word "cost". In any case, in my opinion our customers were not taking their problems very seriously, and their price sensitivity reflected that.
If you have any trustworthy source of ballpark pricing for a 10Gb/s DPI solution, Narus or otherwise, I'm curious. I'm talking BALLPARK here.
I looked up some of Narus's whitepapers, and what I'm seeing looks like Grade A Bullshit. That doesn't mean they ARE bullshit -- their whitepapers would look like this in either case. But I would like to point out that it is absolutely possible to build a company like this without actually delivering the goods, because many of your potential customers are largely incapable of understanding whether or not the goods have really been delivered.
ALSO, I guess I should remember that DPI for stopping malware, and DPI for spying on citizens, are really different things. When you're spying on citizens, 90% effectiveness is good enough. When you're trying to block malware (versus e.g. commited cyberterrorism), 90% is a little better than 0%, but not much, because your adversary can try many times and they only have to succeed once. So in that sense, the DPI being discussed here is a much easier problem than the DPI that we were trying to tackle.
well, i think you can look up the pricing for gateway nodes for cellular wireless equipment vendors (hspa/umts/lte/cdma etc.). my guess is that it should somewhere around 200-400k range. a sufficiently capable box can handle approx. 7 - 10m subscribers simultaneously doing data.
any cellular network that you send data on, already does this. standardization only levels the playing field.
Bandwidth is the measure of data over time. DPI is a process which inspects each piece of data and that takes time. Therefore, the bandwidth will decrease.
Latency != bandwidth. Also, one goal of DPI is to enable granular QoS policy. So the total bandwidth available would stay the same, but the bandwidth available to each app would vary depending on network conditions and the will of the network operator.
Wikipedia says, "In computer networking and computer science, the words bandwidth, network bandwidth, data bandwidth, or digital bandwidth are terms used to refer to various bit-rate measures, representing the available or consumed data communication resources expressed in bits per second or multiples of it (bit/s, kbit/s, Mbit/s, Gbit/s, etc.)."
All else being equal, increasing the denominator reduces the rate. It's just math.
And it's the wrong math. You claim that increasing latency will necessarily decrease bandwidth, and that is simply not the case.
If we assume that there are a fixed number of packets in flight at any given time, and we increase the flight time of all packets, then it will be true that the packets-processed-per-time will drop (and ceteris paribus that the bandwidth would drop). This assumption, however, is incorrect. One could (in theory, and within limits in practice) add extra pipeline steps to do extra processing, increasing the time taken to process each packet, without decreasing the rate at which packets traverse the processor (e.g. the router). This is hopefully what happens.
Of course, as I and others said elsewhere, it is OFTEN the case that as processing-per-packet increases, this becomes a bottleneck on packets-per-time, and this does eventually limit the bandwidth. (Jumbo frames sometimes help, but not if the processing is being done on the entire application payload, which of course is what DPI does, so jumbo frames ain't gonna help there.) So in practice, more DPI might well decrease bandwidth.
If you would like an analogy, imagine the hosts are cities, the packets are trucks carrying goods, and the route is a highway of fixed speed limit and width. (Yes, one weird thing about this analogy is that building new trucks is basically free and done on-demand.) Suppose we consider shipping an infinite amount of coal from city A to city B. The number of tons of coal shipped per day does not depend on the length of the highway. The amount of time it takes for each ton of coal to get there DOES depend on the length of the highway. Coal-per-day is like bandwidth, hours-per-truck-trip is latency. You can increase or decrease one without affecting the other, all else being equal.
Note: really, I shoulda done that example as a series of tubes, oh well.
You're right, I said something silly and invalid. What I was trying to say was simply that an in-series DPI implementation will increase round trip times and decrease bandwidth.
My personal understanding of DPI is that they want to do it in-series so as to be able to reset TCP connections immediately upon detecting a hit, but that might not be how they do it.
"In TCP connections, the large bandwidth-delay product of high latency connections, combined with relatively small TCP window sizes on many devices, effectively causes the throughput of a high latency connection to drop sharply with latency" --Wikipedia
It does equal bandwidth until everything is jumbo frames UDP
If you would have pasted a larger part of the quote, you would have included the part of increasing the window size (e.g. window scaling, selective ACKs, like almost everything out there supports) and the mention of satellite links to refer to high latency conditions. And that notwithstanding that, we are talking about additional latency of microseconds against a typical delay of at least 30 milliseconds. "drop sharply" just doesn't apply here.
If you want to argue against DPI that's fine but "it'll make the Internet slower" comes off as whining plus you're fighting against the exponential effects of Moore's law. A quick Google search tells me there are several products that'll do line rate DPI at 10Gbps.
There are much better arguments to be made against DPI such as privacy, a slippery slope to a walled garden, or just plain unfairness.
The main limit to bandwith is speed of routing the packets. This is why QoS typically fails for IP; it is usually cheaper to just use a faster dumb router than to use a slower smart router.
Monitoring systems are usually extra boxes that you place as either pass-through on the cables, or you tap the signals with attenuators on electric cables, or you bend the fiber enough to tap about 10% light off them, enoufg to regenerate the signal. Other solutions such as just using mirror ports on an ethernet switch exists as well.
They won't need luck, they have the rubber stamps of "National Security" and "We Promise We'll Only Use It For Bad Guys". I imagine that someone somewhere could find a way to apply the interstate commerce clause to let the US gov do what it wants there, too.
They've already been effectively wiretapping and storing a lot, if you believe some of the recent whistleblowers, and so far there's been no effective pushback.
I have heard that there is an age-old debate in the NSA and CIA about whether it is better to encourage consumer security so that we're less vulnerable, or to encourage the existence of known-to-the-government problems to facilitate signals intelligence.
Ever since 9/11 the signals intelligence side has completely won. And I suspect will continue to until the first major confirmed case of a major cyber attack on the USA that significantly inconveniences the general public. (Ideally an attack from a tiny power - something the size of Al Qaeda circa 9/11 would be perfect.) Then there will be much handwringing over how we could have let ourselves be so vulnerable.
It seems to be a balancing act. Too little secrets, and your law enforcement has a very hard time detecting threats before they happen. When people use VOIP instead of telephone lines, it's very hard to wiretap Dangerous People (and non-dangerous people).
It's easy to find ways that such things make it easier for people whose job, goals, sworn duties, etc are to Protect us, or our nation. Many people join the armed services and civil service (of any country) for that reason. I rather like the idea of our intelligence agencies finding out ahead of time about genuine threats (whether from foreign states or from terrorists and the like), even while at the same time I am frightened by the potential slippery slope of where this could lead if unchecked.
At some point, you really do have to decide whether you prefer safety or liberty. Part of me wants to shout "liberty!", as it's a founding principle of our country, but as a parent and citizen it's very easy to also want safety.
While that choice seems to make sense on a short term, I don't think giving up liberty can lead to safety in the long term. At some point your liberties will be so restricted that you're at the mercy of the rulers without much chance to influence how they rule.
There are so very, very few Dangerous People, and so many, many non-dangerous people. The latter class also includes people with at least some money, so en masse, the non-dangerous people constitute a large amount of money. I reckon that an overwhelming majority of the wiretaps are to make money, rather than to catch Dangerous People, statistically speaking.
A lot of the reasoning behind extending surveillance stems from the need to find the people who enable the dangerous people to do dangerous things.
One example: when I got robbed a couple years back (an armed guy stole my laptop), I had the opportunity to discuss the strategies with the detective in charge of my case. A lot of resources were devoted to catch the guy with the gun, but very little to catching the person (or organization) buying the stolen goods (in this case, a laptop) and reselling them. If you catch the robber, it's easy to replace him. If you disrupt the chain at the receptor, you will do more damage. OTOH, if you catch the unsuspecting buyer of a stolen laptop, he (or she) will gladly point the authorities to the store where they'll find a convergence of many such value-chains. This is where most of the money is and where the most damage will be done to the system. That's why now I have the serial number of my laptops written down and all their labels photographed and stored. And all sensitive information encrypted, in case they don't want my laptop, but the data on it.
Having said that, catching the people who support the really dangerous extremists, the drug-dealers, the pedophiles and the slave-traders involves catching who, at the surface, seems rather harmless, making donations to religious organizations, smoking a joint at a party, buying porn online and groceries from Walmart.
On one hand, we may want to make our technology difficult to abuse, but, on the other, we may also want to find people who are very good at protecting their tracks, and do so through people who really don't know how to do it.
Not too be too difficult, because I'm actually interested in whatever rationale behind extended surveillance exists, but can you cite something official for me to read about this "reasoning"? Because it seems to me that much of the rationale for ridiculous amounts of surveillance is "Hey! Terrorists!" That's certainly what's behind the War on The Unexpected, as manifested by the TSA and DHS.
One way to think of this is as a cat and mouse game.
Another way is that some people are developing high-bandwidth, distributed, and anonymous internet software and some other people are writing tests that this software is expected to fail presently but that the developers must pass in order to make it to the next level of hardened security and reliability.
You know we're about to have a bad time when the highly tech savy crowd on HackerNews is left asking "what does this mean?" All over the comments in the thread...
We need to start establishing a DPI/MITM resistant secret sharing protocol, and a related IP protocol.
E.g., a series of simple recaptcha-style tasks that need to be solved within 3 seconds each to be considered secure - in order to avoid active MITM with either machine or manual labor. Once a secret key is established, it can be used from that moment on.
Otherwise, opportunistic encryption can be MITMed and becomes useless.
Less secure: Require committing the equivalent of 10 seconds of a modern i5 processor to establish the shared secret within 20 seconds. That would be acceptable for two side of a connection, but not for a server or a MITM attacker.
Can someone explain what this means from a practical point of view for telecommunications end users? Does it mean that NGN traffic will all undergo deep packet inspection in transit? What exactly are NGNs?
There's a lot of misconceptions on DPI going on here. Without taking position, let me just share a few facts (I work with this...).
- DPI is used in most networks in the world, and is mainly used for throttling P2P or for traffic analysis. Legal interception e.g. classical wiretapping is a different thing, although it could use same hardware.
- DPI does cost some resources to the operator but the impact on end users is negligible (except for malfunctions or improperly dimensioned DPIs). It only adds a tiny amount to latency, which puts an upper limit on bandwidth. In many cases, the bandwidth is limited somewhere else on the link.
- DPI hardware works in several stages, typically: analyze IP flow (shallow inspection), if not enough to decide, to DPI (analyze HTTP headers etc), if not enough to decide, rely on heuristics based on traffic pattern etc.
- DPI is not very good at all at dealing with encrypted traffic, and most DPIs will not be able to do anything else than shallow inspection. (some claim to do traffic flow analysis, and some (normally transparent proxies) can break the HTTPS flow in two, but it would generate client errors).
- ITUs specifications won't have much impact on what ISPs do at the moment (as they already do it) but I guess could be a part of the wider regulatory discussion.
Historically the ITU was a forum through which Ma Bell marketed her switching equipment to national phone companies outside North America. Presumably this is more of the same, except Ma Bell was divested of most equipment manufacturing and it's all IP now so other companies (USA, Japan, Germany, with a few others) are driving. The capacity for spying on and shaping the internet use of customers/subjects is appealing for many decision-makers. Among "first world" nations, we probably shouldn't discount the persuasiveness of the various American TLAs with their various evil projects. Don't you want your national constabulary to buy the same crap the DHS buys?
I suspect that if all of this was for reasonable network maintenance then the ITU wouldn't need to be involved. The equipment manufacturers have the researchers; they don't need the ITU to tell them how to shut down open mail relays.
What makes me less concerned about these proceedings is that all this is happening anyway, whether we're told or not. Yes Virginia, there are bad people in the world, and some of them run phone companies, while others spy on the citizens who pay their salaries. Eventually they'll all be routed around, and we have encryption anyway. Sure key management is hard, but if your opponent controls national network operators then you'll use something better than TLS.
I do find that description a bit odd, since all the early ITU specifications were made mostly by Europeean vendors, and were incompatible with the American versions of protocols, etc.
Anyway - a lot of these specs are made by the very researchers of the various vendors that you talk about - ITU is just a forum they use to collaborate.
If this were some form of physical information transit it would have never passed. While ill be the first to admit I do not know what will come from this, or really even the basic understanding of DPI, this can not be good for those who value there privacy.
I imagine that forcing encryption over all connections would be a counter measure to this? Going about getting all websites to offer encryption might be another story...
Against big governments, nothing is stronger than the little men and women getting together.
If this is so bad (which I am not technically capable of understanding...), what shall we DO about it?
Signing the petition is probably not enough. Get people to have a minute of 'no internet' across the world? Similar to the Anti-SOPA movement? Suggestions welcome.
Are they recommending Deep Packet inspection to the governments of the world in the name of "security" or are they recommending it to the ISPs as a means of packet prioritization?
I'm unsure if this is "requirements on how to implement deep packet inspection, if you choose to do so" vs "requirements to perform deep packet inspection."
Context is that Google says it impacts profit^W^W^W it's evil, so it must be. Because hey, standardizing what every government is doing already must be evil.
IPSec by itself isn't going to really help. If they are doing DPI, they can MITM your IPSec connections. You still need a key management system, and I am not aware of any large-scale systems that are in-place to just "switch on" IPSec, that is, suddenly provide you with the certificates for every IP you want to connect to.
You're right, it could be a passive inspection. But... if they are your ISP and have access to your packets, chances are they can rewrite and inject traffic too. Sure, they might need a bit more hardware to do so, but it's not exactly difficult.
But you're correct, DPI doesn't necessarily imply MITM capabilities.
http://itu4u.wordpress.com/2012/10/25/proposal-for-ict-and-i...
They want to do this, they say, to help "grow the Internet" (hasn't the Internet grown fast enough without their help in the past 20 years?), despite the fact that evidence suggests that the sender-pays system would slow down the growth of the Internet, not make it any faster (research paper):
http://mercatus.org/publication/do-high-international-teleco...