I'll tell you a dirty little secret of the protocol design.
Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs, send window and retransmissions. Guess what? If you don't follow windowing semantics of TCP, then one of two things will happen on saturated links - either TCP will end up with all the bandwidth or you will.
So - surprise! - you have no choice but to design a TCP clone.
[ EDIT ]
That said, there is a fundamental problem with TCP, when it's used for carrying secure connections. Since TCP acts as a pure transport protocol, it has no per-packet authentication and so any connection can be trivially DoS'd with a single fake FIN or RST packet. There are ways to solve this, e.g. by reversing the TCP and security layers and running TCP over ESP-over-UDP or TLS-over-UDP (OpenVPN protocol). This requires either writing a user-space TCP library or doing some nasty tunneling at the kernel level, but even as cumbersome as this is, it's still not a reason enough to re-invent the wheel. Also, if you want compression, it's readily available in TLS or as a part of IPsec stack (IPcomp). If you want FEC, same thing - just add a custom transform to TLS and let the client and server negotiate if to use it or not.
I mean, every network programmer invents a protocol or two in his lifetime. It's like a right of passage and it's really not a big deal. Unless it ends up attached to a brand name, in which case it starts attracting snarky remarks like this one :)
> Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs...
> So - surprise! - you have no choice but to design a TCP clone.
Some use cases don't really want TCP or a TCP clone, for example multi-player games might only use a semi-reliable loss detection system which has ACK's but does not attempt retransmission on failure to deliver. This works a lot better than TCP for that use case, and is not really a "TCP clone".
The really sad thing about this though is that "the internet" no longer works for stuff which isn't TCP. I guess it does, but not for your whole target market.
The lowest common denominator is users behind an NTLM proxy. Mass market products don't generally work unless they have a fallback which is "I can make it tunnel through proxies". Which is HTTP or CONNECT :( Which means no teams will do something which isn't TCP anymore, since it's not a thing they can tunnel :( Interested in counterexamples though.
This is the same reason why a UDP replacement is a great idea but a waste of time for most people developing a product. This makes me sad.
You have it the wrong way around. People who cordon themselves off and only leave HTTP working aren't on the Internet.
They don't want non-web apps to work. Those networks deliberately rule out networked games, uTP, VoIP, IPsec, WebRTC, etc.
If you want to worry about people teetering on the edge of the Internet, you should worry about people behind NAT and its prevalence to the point of people confusing NAT boxes and routers...
(Re your last sentence, it's not a UDP replacement, see other comments.)
// Defines for all types of congestion feedback that will be negotiated in QUIC,
// kTCP MUST be supported by all QUIC implementations to guarantee 100%
// compatibility.
enum CongestionFeedbackType {
kTCP, // Used to mimic TCP.
kInterArrival, // Use additional inter arrival information.
kFixRate, // Provided for testing.
};
Actually, there are also "scavenger" congestion control algorithms that use leftover bandwidth. The most high-profile example is LEDBAT used by BitTorrent's uTP. uTP is indeed a TCP clone, but I had my chance to develop a protocol which was using LEDBAT and it was NOT a TCP clone at all (it later became IETF PPSP).
So, TCP eating all the bandwidth is not an issue sometimes.
uTP is designed to let TCP eat all the bandwidth on saturated links, and generally defer to TCP whenever possible. For BitTorrent, that's exactly what you want. For web browsing, you probably want to be more aggressive. I skimmed the source code, and it looks like they're using the same CUBIC congestion control algorithm that Linux's TCP stack has been using by default since 2007.
I agree with your original point, but none of the stuff after [EDIT] makes much sense to me.
"Since TCP acts as a pure transport protocol, it has no per-packet authentication info and so any connection can be trivially DoS'd with a single fake FIN or RST packet."
I guess you mean if you're sharing medium or your adversary has MITM? You're generally screwed for DoS in that scenario anyway, no matter what encryption thingy you use. Otherwise, any modern TCP stack should be able to reject blind FINs or RSTs as out-of-sequence.
Generally, the reason people avoid TCP-over-TCP is not DoS, but basically "timer fights", as per: http://sites.inka.de/~W1011/devel/tcp-tcp.html. That said, everything-over SSL is still popular for practical reasons, such as clientless VPNs which need to work around proxies, firewalls etc.
No, not the MITM. A simple denial of service - I can terminate your TLS-over-TCP connection at will by sending a FIN or RST packet.
More generally speaking, I can alter the state of your secure over-TCP connection with an unauthenticated packet. Compare this to ESP where every packet that can potentially alter the state is first authenticated.
Blind, or with some way to observe the sequence numbers?
If it's blind (e.g. window guessing) I'm interested in what your techniques are.
If it's not blind and you don't have MITM and you don't share a medium, I'm interested in how you're doing that.
Challenge: I have the following TCP connection open (oh man I'm really asking for it, but what the heck :)
tcp 0 0 192.168.0.10:52352 216.240.155.220:80 ESTABLISHED 16284/nc
If you can terminate it with a FIN or an RST in the next hour, I'll post proclaiming your victory. If either the box or my local link drops, we'll call it a draw.
EDIT: Just downloaded the sample code and it looks like it won't quite work out of the box, but I'm sure someone with more time could through together something with scapy or similar pretty quickly.
Well I did post the source port, which reduces the effort by a factor of tens of thousands... also, the up-thread did say "with a single fake RST or FIN packet"... which is why I mentioned window guessing...
Also, crap, I'm an idiot. Let me do that non-natted ;)
tcp 0 0 216.240.155.220:4007 60.225.131.226:43972 ESTABLISHED 18694/nc
Sorry for the beginner questions, but how did the remote port change from 80 to 4007? Are you NATed on both ends? And, how did you go about getting the IP:port of your local router? Are you using some tool, or do you have root on your office router?
PREFIX: this is going to sound really cynical. Also, I haven't read RFC5961, so thanks :)
Yeah, actually it might have been. But I prefer to give HN the benefit of the doubt.
Umm I try and be gentle. But let's talk about the fundamental problem, since you appear to understand it.
TCP-RST DoS is like the worst denial of service ever, since it has such massive negative amplification. You're spending like something like 30k * (required window guess packets) to interrupt one connection, which requires like say less than 10 packets to re-establish. Unless you're talking about BGP, which shit, we normally use TCP-MD5 for anyway. Even commodity home bullshit throws most of your packets away for free.
I take things quite slowly. The normal HN methodology is to go "Incorrect. blah blah blah.". Reality is nuanced and complicated. I try and avoid doing that, because honestly most statements are actually right when you squint at them the right way. Unfortunately, I get the most fake internet points when I say stupid and ultimately unsupportable bullshit while drunk. I still try to avoid doing that :/ But ahhhh here I go anyway.
The nuanced version of this is that most people's devices are going to get DoS'd way harder by fake ipsec traffic which forces checksums in CPU than it is by out of sequence packets which get handled in hardware.
But why don't we live in this fantasy world where we can all terminate TCP connections with imaginary sniper packets and ipsec solves everything.
..., most of it, is filtered out by the replay protection at virtually no processing cost. The HMAC verification is not exactly a CPU killer either, even less so if there's some sort ASIC/accelerator involved, which won't be that unusual if there is an IPsec involved.
To not multiple the comments - what I said above about a single TCP packet obviously implies being on the route between the parties.
> ..., most of it, is filtered out by the replay protection at virtually no processing cost
Really? Think about that and reply, given the ability of your adversary to send arbitrary traffic.
> To not multiple the comments - what I said above about a single TCP packet obviously implies being on the route between the parties.
How is this seriously distinguishable from MITM? I mean what the hell. Come on you're stretching definitions here.
Uhhh anyway, I think in about 5 years you might be right. Practically speaking no consumer hardware has ipsec embedded but hopefully everyone will in the next few years.
The last time I was in Taiwan working on ASIC hardware none of the white brands did (and most of the big brands are just buying that crap and re-labelling it). Yeah designed in $country? Nah. Taiwan is super good at design, they just need the vendor and the label on the box. Anwyay, that was 2 years ago though, and I was just doing consumer bullshit.
In case you're interested (and you're probably not) the ASIC NAT hardware was the only there coz it enabled the anemic CPU to NAT at the box rated speed. Unfortunately it also turns out, that's why most home users can't seriously torrent or do nice things :( Damn ASICs.
People wonder sometimes why "open source" version is more expensive, and in terms of consumer routers, it's because the pure linux versions don't have the secret sauce. You gotta use more CPU and memory. It's a worthwhile trade-off for not putting up with a real RTOS. But also, good luck MIMO 450 Mbps wireless N using just the broadcom CPU at the rated Mhz :) I'm not even sure if they license the drivers for open source now, I know they definitely didn't before.
There are some really hilarious things which happen where like $marketing from $vendor decide that not only they need a custom enclosure (everyone needs that), but that the box would look way better if the antennas were arranged in certain way. This triggers multiple all-nighters from the engineers who have to relayout the boards and redo the EMC tests.
Wait, I'm way off topic. I think my original point is that you probably don't have ipsec in hardware at home. I await brutal smackdown, coz the community is knowledgable.
"Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs, send window and retransmissions."
Maybe I'm missing something (likely) but it seems to me any potential replacement for UDP specifically doesn't care about packet loss, acks, etc. The whole point of UDP is speed over reliability, yes?
TCP has more than a few other issues as well, not the least of which is that signalling for those windowing semantics doesn't effectively distinguish between congestion and packet loss.
This is more than merely UDP with encryption. Rather, this seems to be a reliable stream protocol that aimed at dethroning TCP. I don't think it intends to replace UDP. If anything, I would describe it as a user space replacement for TCP. It uses UDP, yes, but that's because UDP is about as close as you can get to IP at the user level.
It's about time, too. Certainly user-level substitutes for TCP are not a new idea (I've worked a good deal with UDT), but this the first one to my knowledge that has the weight of something like Google behind it.
There are a lot of advantages to doing this as the user level with UDP as well. Not needing to install a kernel module, for one. But also P2P rendezvous connections through NAT with UDP (we wouldn't even need to wait for router manufacturers to adopt anything), better control over protocol options without going through the kernel, custom congestion control algorithms, finer control over buffers (mmap'd files as buffer for file transfers anyone?), etc.
I'd much rather see this at the kernel level, but TCP as it's implemented on most systems has too many issues on modern networks, and most acceptable user-level solutions (ugh, parallel streams) are really terrible hacks. If Google does it right, maybe it'll force OS developers to wake up and rethink the increasingly clunky and antiquated BSD sockets API that is the source of many of TCP's woes and finally modernize the network stack for the fiber era.
What exactly about the BSD sockets API makes it a source of TCP's many woes? What woes are you talking about?
Also, why do we want to re-implement TCP in user space, when it still needs to traverse the kernel as a UDP packet, this doesn't have any of the benefits that netmap or PF_RING for example bring to the table where the software is doing the full stack and thus there is less latency involved.
QUIC as it currently stands will be sitting on top of UDP, so all packets traverse the kernel, get dumped to user space, user space parses it, and sends packets back, as UDP. All that extra copying, when with TCP in the kernel the kernel would be responsible for reassembling packets, would be responsible for ACK'ing packets, and stuff like that.
Also, mmap'd files as buffer for file transfers ... you are going to have to explain that one. There are already various zero copy mechanisms for sending files, such as sendfile().
As for new congestion control algorithms, great, sounds fantastic, but how much more of a nightmare is it going to be to get the parties to agree on a congestion control mechanism? We already have that problem now ...
Sorry, but I see a whole lot of complaining without any real data or information to back it up. [[citation needed]].
Maybe TCP's issues aren't apparent when you're using it to download page assets from AWS over your home Internet connection, but they become apparent when you're doing large file transfers between systems whose bandwidth-delay products (BDPs) greatly exceed the upper limit of the TCP buffers on the end systems.
This may not be an issue for users of consumer grade Internet service, but it is an issue to organizations who have private, dedicated, high-bandwidth links and need to move a lot of data over large distances (equating to high latency) very quickly and often; CDNs, data centers, research institutions, or, I dunno, maybe someone like Google.
The BDP and the TCP send buffer size impose an upper limit on the window size for the connection. Ideally, in a file transfer scenario, the BDP and the socket's send buffer size should be equal. If your send buffer size is lower than the BDP, you cannot ever transfer at a greater throughput than buffer_size / link_latency, and thus you cannot ever attain maximum bandwidth. I can explain in more detail why that's true if you want, but otherwise here's this: http://www.psc.edu/index.php/networking/641-tcp-tune
Unfortunately for end systems with a high BDP between them, most of the time the maximum send buffer size for a socket is capped by the system to something much lower than the BDP. This is a result of the socket implementation of these systems, not an inherent limitation of TCP.
An accepted user-level solution to this issue is to use multiple sockets in parallel, but that has its own issues, such as breaking fairness and not working well with the stream model. I can explain this more if you want, too, just let me know.
As for zero-copy with sendfile, well, even when you do zero-copy, the above issues still apply, because the socket buffer is still used. Admittedly my research into zero-copy is very cursory, but from what I understand, even when you use sendfile, it still copies from files into the TCP send buffer, so zero-copy is actually less zero than it seems. It just doesn't require the whole user space read-buffer-write-buffer loop, which does yield a noticable performance increase, but that doesn't mean the buffer magically goes away and the BDP issue is solved.
What I was suggesting, by using an mmap'd file as the send buffer, would allow the TCP send buffer issue to be circumvented completely. In a user space protocol specialized for file transfers (not saying QUIC is, but I'm not talking about QUIC), you wouldn't need to use a send buffer, and the window size would become limited only by the BDP; the file contents are always available without needing to buffer them like the socket API does, and the network will never have to be idle just because the send buffer is capped at 10 MB or something.
You mentioned QUIC needing to do the whole userland dance to let the application parse packets and all, and that is a valid point, but, in the case of reading, it's entirely possible to only read in the header data you need from the datagram and let the kernel take care of as much of the actual copying as it can for the rest of the packet. In the case of writing, you can "plug" the socket, write headers in user space, then zero-copy the file segment for added performance.
Certainly there is a performance hit, since it's still going into user space, but there wouldn't be if this was all part of the kernel. Like I was saying, I'm not so much excited over the fact that this is user space or anything, but more so that it has the potential to exert pressure on the Linux community and such to make finally make some broad and well-needed changes to the network stack. Even if QUIC isn't being designed to address the issues I'm mentioning, maybe a user space-configurable socket library with the backing of Google will make experimentation with internals normally obscured by the socket API more accessible to people, and something good will come out of that eventually.
Anyway, all of these issues stem from the fact that BSD sockets are meant to be a general purpose communications protocol. However, some applications (such as file transfers) don't need all of the amenities general purpose sockets offer, such as separate kernel-allocated buffers so applications don't need to keep buffers after calling write(), etc.
There are other problems with TCP, such as slow start being, well, slow to converge on high-BDP networks, bad performance in the face of random packet loss (e.g., TCP over Wi-Fi), congestion control algorithms being too conservative (IMO, not everyone needs to agree on the same congestion control protocol for it to work well, it just needs to converge to network conditions faster, better differentiate types of loss, and yield to fairness more quickly), TCP features such as selective ACKs not being widely used, default TCP socket settings sucking and requiring a lot of tuning to get right, crap with NAT that can't be circumvented at the user level (UDP-based stream protocols can do rendezvous connections to get around NAT), and more. People write whole papers on all these things, though, and I don't want to make this even more of a wall of stupid text than it already is.
Okay, you can call me out for not having citations or backing up my claims again now. Problem is most of the public research exists as a shitty academic papers you wouldn't probably bother reading anyway, and most of the people actually studying this stuff in-depth and coming up with solutions are private researchers and engineers working for companies like Google.
how would you differentiate types of loss? The main issue with TCP over large BDP at the moment is not the limited buffer size IMHO. You can increase is dramatically on modern Linux. You can maybe attempt to work around congestion mechanisms and slow start by using very aggressive algorithms. But I wonder how researchers work with packet loss?
>If Google does it right, maybe it'll force OS developers to wake up and rethink the increasingly clunky and antiquated BSD sockets API that is the source of many of TCP's woes and finally modernize the network stack for the fiber era.
The title seems wrong, at least based on a quick glance this doesn't look like a UDP replacement. It could be a TCP replacement that's encapsulated in UDP.
There would potentially be huge benefits to be had from replacing TCP, but of course that's something the people have been trying to do for a long time and failed. Google has a huge advantage in achieving this thanks to controlling a significant proportion of clients and servers.
These use cases spring to my mind: real-time data and multi-player action games, perhaps even streaming media.
TCP and HTTP are less than ideal transports when it comes to real time data. In case of a network error, TCP retransmits the lost data but in a real-time environment, the data is already old and therefore next to useless. Real time action games (like quake) favor udp over tcp.
Some questions: Why replace UDP? Why not work on top of UDP? There must be a reason for this. What about multicast?
UDP doesn't have any guarantee that packet is received. If you stream video that can handle few lost packet it is ok, but if you need reliability than you need TCP. So, it's kind of programmer decision what to use. In reality there should be protocol that is lightweight and somewhere in between TCP and UDP (from personal experience). I haven't read anything about this new google protocol, but i'm not sure if they are trying to do that. Well, maybe i should go and read it >:-)
I'm all for innovation, but I become wary when a company that controls a sizeable proportion of the browser market-share and also serves a not insignificant proportion of all web traffic makes extensions like this.
Google is the least scary company in this regard. They're putting their extensions out into public hands instead of using an "embrace/extend" model. SPDY is open, soon to become an official web standard, and implemented by popular web server software and browsers (nginx, apache, firefox). The same is true of Chromium as well, the core of Chrome.
I'd go so far as to say that in these areas Google is serving as a model corporate citizen, and I wish others would imitate them.
SPDY was heavily used by Google back when it was basically "do what Google do", putting other browser developers under pressure to implement it somehow or be seen as slower. (There was nominally documentation of how it worked. It was wrong.)
Having open-source implementation do not mean you're not going to "embrass and extend". It's a common misconception.
If you're the only big player, that your stuff is open just means others can also implement your protocol. But since you're the only big player, they can't have their protocol.
So you can make your protocol work for you, do things that will bring you more money, and have it open-source, AND force it upon others.
Forking it doesn't mean shit if you're forced to use it to access _ANYTHING_ anyways.
Chrome isn't the only major web browser. Google doesn't have the power to unilaterally decide what all the other browsers will do. Nor does it look like they're trying to get that power; they give hundreds of millions of dollars to Mozilla, one of Chrome's major competitors, and the Mozilla people don't always go along with what Google wants.
>I'd go so far as to say that in these areas Google is serving as a model corporate citizen.
Well that is a curious way to look at it.
Instead of working with other companies to define a common standard they force it through attrition. That's the exact opposite of how this sort of thing should work.
And given Google's disgraceful behaviour in abusing the FRAND process they are hardly a model citizen by any measure.
Sorry, but standards aren't formed in committees, they are standardized there.
The right way to do this is: Put out your stuff, get others to implement it, everyone has experiences, get together and standardize based on those experiences.
This is why folks like IETF rely on "rough consensus and running code" when standardizing.
I always see this idea that companies should get together in a room and define protocols, and having seen standards orgs that do this, it scares the hell out of me.
Exactly, ask the ISO how well the standards specified by their OSI reference model [1] competed with TCP/IP et al. The only substantial remnant of the OSI network standardization process today is the generic concept of protocol layers.
Or see the HTML standardization process. Or a hundred other examples.
Standardization almost always follows implementation, not the other way around.
Okay, can you name some standards which were created by multiple companies working together? (I'm not talking about maintenance -- HTML was invented by Tim Berners-Lee, SGML came out of IBM, JavaScript was Netscape, etc.)
The Java Community Process back in the day was a good example of how lots of large companies working together could define a standard and yet still leave room for innovation.
H.264 and WiFi standards are recent examples of a lot of companies working together.
Yeah, but 90% of those standards were TERRIBLE. Like, GOD AWFUL. EJB? SOAP?
It's the hangover from those standards that leaves kids today thinking that Python or Ruby are simpler than Java.
Eventually the JCP turned into "take popular open source ideas like hibernate and spring, and bless them with a JCP number". Which worked a lot better, and each of those individual projects was usually launched by an individual or small group initially.
The language Java is simpler than the language Python or Ruby. Fewer keywords, those keywords do fewer things.
Ruby and Python seem simpler for most web work due to better frameworks and dynamic typing, which is less simple than static typing, but is easier to work with in most web situations.
I guess I'm drawing a distinction between 'simple' and 'convenient'. Monkey patching in Ruby is not simple, but can be convenient and make rails possible.
I'm undecided whether Python/Ruby or Java are simpler.
At least to me dynamic typing seems much simpler than static typing: from a very very pragmatic view that just means no compile-time checks (?), and I'm not that good with Ruby, but I'd consider Pythons type system much simpler than Javas (Python has none of: primitive types, null, arrays, interfaces, visibility (well, there is name mangling for __...), enums, final/abstract, the checked/unchecked mess (look up what the correct way to catch exactly: all checked exceptions is!), casting, generics and everything that comes with it: have a look at the index at the left side of http://docs.oracle.com/javase/tutorial/java/generics/index.h... -- but to be fair Pythons type system has a couple of nontrivial features (metaclasses, multiple inheritance, ...) too)
Python/Ruby have a lot of very useful language features which make the languages much less of a pain to use than Java (the language), but for any meaningful comparison of language complexity you'd surely also have to consider the intricacies of the underlying VMs. The Java VM is quite complex and also quite powerful. For simple programs that don't require high performance or efficient memory management or whatnot that might not matter much though.
[Now, after having written this, I don't think its possible that any of the three languages is strictly simpler than any of the other ones.]
BrainF##k has even fewer keywords than Java, that doesn't make it simple.
I understand your point. Ruby and Python both have dynamic behaviour which makes it difficult to predict ahead of time what will happen when a program is run. In practice, however, the full range of possibilities is never exercised. It's not enough to look only at the language specification when you make these generalizations.
EDIT: TL;DR, essentially what matters is not the complexity of the language, but the complexity of programs written in the language.
The standards weren't terrible it was the entire concepts themselves and reflective of that era of computing. Most people thought XML was the future of the world and component architectures would be how all business software would be written. Little did we know.
But I still use SOAP everyday with Java, .Net, iOS and JS clients. So the benefits of companies working on commons standards is still pretty clear.
It's not just XML. Why would you ever choose SOAP over a simple REST endpoint? If you use SOAP frequently, have you used jaxb or jaxrpc? Those things were awful.
The systemic problem, which isn't just hindsight, is that you had 6 corporate vendors, each with their own internal politics, in a room with cross-corporate politics, creating a standard. Of course it will be a bloated monstrosity. It's not just hindsight to say that, that arrangement could turn all kinds of great ideas into disasters.
>Why would you ever choose SOAP over a simple REST endpoint?
I thought the obvious answer was always tooling. Point Visual Studio at WSDL, and immediately get IntelliSense and go to town. Take your easy "object oriented" designs, and pop them on the web, with less of the trouble of DCOM or CORBA, and more security. But of course it only works if the tooling is in place, and there was probably only a couple of interoperable stacks.
But at least there was a real reason, and serious value for a lot of developers. To compare, JSON doesn't even have a standard date format defined.
SOAP/WSDL has IDE support and various tooling because it would be almost unusable without it.
Imagine doing SOAP entirely by hand, with a semi-complicated WSDL. It's a nightmare. Doing a RESTful service based on JSON data is not just not a nightmare, it's a breeze, with no need for any of the tooling you describe and with, in my opinion, absolutely no difference in API richness, interoperability, security, quality etc.
If they'd announced this before they had a working implementation then Microsoft would have either created their own competing standard or wanted changes that only benefit themselves and slowed down development.
AFTER EDIT:
Why the downvote?
Ok, maybe I shouldn't have singled out Microsoft, but I think their past behaviour explains a lot the way Google are behaving now. I'm thinking in particular of what happened with XForms and WebGL.
Seriously, why the downvotes? I don't think developing in secret is how things should be done (in a perfect world). I'm just theorising why Google might have chosen to work that way. Is this not a reasonable motive, or is there a better explanation for their behaviour?
They're still right, it is a security risk.
Now, it doesn't mean we shouldn't have it. But it is one. Heck, the well known chrome vulns of the past alone should show that.
Your kind of skepticism is ultimately one of the few things that holds corporations in check, so I am not discouraging you from persistently aiming it at Google, Facebook, et al.
But that being said, I was a full-time dev there for 5 years, and I can say with a (however misguided) degree of certainty: all of those projects have one goal in mind - to make the experience of using the web better. Whatever else is true about the dangers of monoculture, Google's intentions are benign.
I think it's entirely sensible to think about the influence that companies like Google are gaining.
Each individual component is a sensible step forwards, but there's no doubt that Google could choose to do some pretty anti-competitive things if they wanted, and in the future there will always be different, more short-term, management in charge
A big company will always have the potential to do harm. If it's not about what they're doing, but what they could do, then any company that has influence in a market needs to be taken down.
I'm not against companies having influence; the first thing I said was that I was all for innovation and what's the point of innovating if you won't be heard?
The point is that Google are in, not just a position of influence, but a position of control. Google don't need to seek the acceptance of the community at large for SPDY to be useful to them. This is a very different standpoint from, say, Mozilla.
arguably, google are well positioned to see where improvements to traffic protocols can be made due to both chrome's large market share and their massive traffic requirements for google search et al.
Later on if/when their protocols are finalised and released, I'm sure one standards body or another will work to integrate them.
As other's have pointed out, it's not a replacement for UDP or "improved version of UDP with some extra features such as encryption" as mentioned in the original article but a reliable, encrypted protocol using UDP as transport layer. That makes sense because most NAT boxes etc. have to let UDP packets get through because they have to support DNS, VOIP protocols et el. I suspect the encryption part has more (or at least just as much) to do with keeping the middle-boxen's dirty hands off of the the protocol innards than MITM security.
That's how one finds out about the real-world benefits. I'm not an ops person, but once it's already deployed, how hard is it to just leave it there for the limited benefit it does give?
The guy seems to know what he's talking about, and has actually run some tests but the test setup seems kind of weird.
Effectively he's saying "if you only change HTTP to SPDY and make no other changes, then it's 3% slower (though more secure)" or "if you only change HTTPS to SPDY and make no other changes, then it's only 4% faster" assuming your site looks like some kind of average of the top 500 sites.
And his conclusion is much more positive than most of the 3rd party coverage of it:
"I believe SPDY is still awesome, and is a step in the right direction. This study and its conclusions do not mean we should stop working on SPDY, but rather that we should work on it more, and make it even faster."
Guy does know what's talking about but there are some limitations to his tests - which to be fair he's pretty open about in the post.
The biggest challenge is optimising for SPDY is different to optimising for HTTP and so many of the test sites may have behaviours that aren't optimal for SPDY.
In addition to Google, Facebook and Twitter are/have deploying SPDY so I think we can be sure it has benefits.
Yeah, another way of framing it is: even if you do not put the least effort into SPDY, you will benefit a little from security, if nothing else. Tuning incrementally will buy you further improvements.
I believe there was a flaw in his argument, and that was Facebook was using SPDY on the front end, but not on all the back end services, or external services Facebook uses...
I really, really wish this particular link would disappear from the net. It is a TERRIBLE test. It has been soundly refuted over, and over, and over again. Yet it recurs.
He used a non-caching proxy with SPDY, against distributed non-SPDY back-end resources. This is not how anyone in the world will actually use it, and his test is effectively akin to testing the speed of a sports car by having it drive behind a transport truck: The whole foundation of the test is nonsense to begin with.
why does google need to come up with protocols? does the IETF and other processes in place for over a decade not work for them? I know nothing of the protocol (the link only mentions the 'cheeky' naming scheme and the source in the chrome tree); I'm merely commenting on googles motives...
How do you think things get proposed to organizations like the IETF? Someone comes up with a working implementation, proves it to work & to be better than the existing stuff, and then they propose it for inclusion.
That's what Google is doing, and they're using Google Chrome as their test bed (along presumably, with a bunch of non-Chrome stuff internally).
If what you're hinting at is that they should first start some wider discussion about what a next-gen UDP should look like before they start doing anything, then I suggest you look up "bikeshedding". Such a process would never get anywhere.
I appreciate their initiative and their wealth of engineering talent. I completely get what you're saying -- someone has to propose it; and a usable reference implementation is all the better. Unfortunately it seems, given Googles mammoth size, that many folks who might otherwise chime in simply shrug and go "oh, Google has already solved/implemented/popularized a solution" and move on.
That's not happening with SPDY, WebRTC, Dart, or WebP. Google's competitors are going over these proposals with the proverbial fine-toothed comb. I get the impression that people are extra skeptical of Google standards because of the potential to get steamrolled.
Disclaimer: I don't work anywhere near the QUIC, SPDY, or other protocol guys.
I heard someone here in a express it this way at a techtalk when asked about the motive for doing things like SPDY (paraphrased):
"We want the web to be as fast as possible for EVERYBODY. So some of us are building things like SPDY since we can implement it both the server and browser for Chrome users. Meanwhile, other folks are working to make the web faster for non-Chrome and mobile users too (PageSpeed, our public DNS, paying people like Steve Souders to do performance research, etc.)"
Good news! Their crypto stuff looks pretty reasonable. Key exchange uses ECDH with either the P-256 or curve25519 polynomials. Once the session key is established, it's encrypted with AES-128 and authenticated with either GCM or HMAC-SHA256. None of this is implemented yet, but it's at least cause for hope.
So like others have pointed out, this is not aiming to replace UDP any more than HTTP is aiming to replace TCP. So it's more like a feature rich TCP replacement, a transport protocol that happens to sit on top of UDP (which is what you have to do unless you want to write a OS driver).
As for impact, depends entirely what applications they plan to deploy it on. WebSockets, peer-to-peer WebRTC and media streaming could benefit from real-time friendly properties (fixing head-of-line blocking and enabling selectively unreliable packets). Hard to see anything significant for traditional web content delivery.
There are many things that you might want from a post-TCP transport protocol. Look at some of the SCTP features for a taste. It has stream multiplexing over the single connection, multipath and dynamic address changes, datagram mode, per-message reliable/unreliable option, to name a few.
SCTP seems to have bombed partly because they were "doing it right", ie getting their own IP protocol number and trying to get it deployed inside OS IP stacks instead of of sitting at application level on top of UDP.
The FEC feature in QUIC sounds a little unorthodox unless it's for multicasting or some control plane function, since at this protocol layer it's considered prudent to be sensitive to congestion when coping with packet loss...
Low uptake as routers will not have built-in support, and admins will be slow to enable it when and if it becomes available (unless its tunneled over another protocol).
IPV6 vs IPV4 question is orthogonal to those addressed by the QUIC protocol. IP layer is for addressing nodes on the internet and routing packets to them in a stateless manner (more or less... I'm not counting routing table caches and such as "state"). TCP (and this QUIC protocol) are built on top of the IP layer to provide reliable end-to-end data transfers in some sort of session based mechanism.
I was thinking primarily of the security aspects, which is one of the few things called out. I'm all for secure and discrete transfers of data between endpoints, since TCP programming APIs often require maintaining process state for each connection and can impact service scaling.
If this is really more or a session layer protocol on top of UDP instead of its own transport protocol, then all the more power to them. It will be more of a programming model than a network configuration quagmire, and I'm sure that eventually intelligent optimizations can be performed via packet inspection if the protocol state details are not encapsulated deep within the security (i.e., trust/privacy) bubble of the protocol.
UDP is connectionless. How can we have encryption without a connection? Using a block cypher would be less secure than stream cyphers and the controversy could halt adoption.
come to think of it OpenVPN uses UDP also, and that's considered quite secure. So what kind of encryption are they using? Rolling their own sounds scary.
Say, you want to design a protocol with reliable delivery and/or loss detection. You will then have ACKs, send window and retransmissions. Guess what? If you don't follow windowing semantics of TCP, then one of two things will happen on saturated links - either TCP will end up with all the bandwidth or you will.
So - surprise! - you have no choice but to design a TCP clone.
[ EDIT ]
That said, there is a fundamental problem with TCP, when it's used for carrying secure connections. Since TCP acts as a pure transport protocol, it has no per-packet authentication and so any connection can be trivially DoS'd with a single fake FIN or RST packet. There are ways to solve this, e.g. by reversing the TCP and security layers and running TCP over ESP-over-UDP or TLS-over-UDP (OpenVPN protocol). This requires either writing a user-space TCP library or doing some nasty tunneling at the kernel level, but even as cumbersome as this is, it's still not a reason enough to re-invent the wheel. Also, if you want compression, it's readily available in TLS or as a part of IPsec stack (IPcomp). If you want FEC, same thing - just add a custom transform to TLS and let the client and server negotiate if to use it or not.
I mean, every network programmer invents a protocol or two in his lifetime. It's like a right of passage and it's really not a big deal. Unless it ends up attached to a brand name, in which case it starts attracting snarky remarks like this one :)