TCP is very old. You would not do that today, and indeed they didn't, QUIC's cryptography is built in.
Two reasons to want cryptography built-in. One in some sense political, "Pervasive Monitoring is an Attack" is BCP #188. We must defeat such attacks on the Network by encrypting everything possible. The other is pragmatic, middleboxes induce Protocol Ossification, making innovation difficult or impossible, by encrypting everything we prevent a key source of ossification, if the middleboxes can't understand the protocol they also can't rust shut extension points.
Ossification isn't really a problem for Homa's intended use case IMO; it's meant to be used on intra-DC networks with latencies already as low as 1us-2us, and is optimized exclusively for this design space (lots of short messages, high load, etc). Everything on the path is likely going to be controlled and on your network already and won't be going near the WAN, probably only TOR/same isle at worst, if I had to guess (I don't have any measurements on intra-rack tail latencies in DCs or anything, I'm just spitballing here.) Third party gear is a huge problem in the last mile but not in tightly controlled networks that something like Homa targets; you not only have to adopt a new socket type but also the RPC design to go with it, and reap the benefits. I suspect most people aren't considering this unless they already have strong network control with their own hardware/SDN.
Similarly, you could make an argument that because this interconnect tolerance is so low, pervasive monitoring has a different risk profile. Two servers communicating via TOR thru a 40G NIC using Homa don't even have an opportunity to pass the packets elsewhere in the hierarchy to be snooped by someone else, unless it's just streaming packets directly into the three-letter-agency van in the parking lot I guess. For all practical purposes in a system like this the requirements are so tight you can only afford direct links between systems (or something close to that) if you want to maintain the desired operational behavior, so third parties appearing between two points might not be possible, much less likely.
That said I think including encryption would be good. But the reasons would be very pedestrian IMO: for one, it ticks off the annoying compliance checkboxes (both political and social) and means there's a single design for anyone who implements Homa to follow, so people don't have to painstakingly re-create it. And a corollary of that checkbox tick is that it nips threads like this one in the bud regardless of all the above. :P
> Two servers communicating via TOR thru a 40G NIC using Homa don't even have an opportunity to pass the packets elsewhere in the hierarchy to be snooped by someone else, unless it's just streaming packets directly into the three-letter-agency van in the parking lot I guess.
Something like FASHIONCLEFT. Your smart managed switch's firmware squirrels away summaries (e.g. it sees 400GB of data about Project Smith and it notes [400GB, Project Smith]) and then later squirts such summaries over legitimate links to distant nodes, but passing through another device with compromised firmware, and the other compromised device removes the extra data and gives it to the NSA.
The NSA values this because plausible deniability is essential to their work, if you realise you were compromised you are likely to blame the destination not them.
TCP is very old. You would not do that today, and indeed they didn't, QUIC's cryptography is built in.
Two reasons to want cryptography built-in. One in some sense political, "Pervasive Monitoring is an Attack" is BCP #188. We must defeat such attacks on the Network by encrypting everything possible. The other is pragmatic, middleboxes induce Protocol Ossification, making innovation difficult or impossible, by encrypting everything we prevent a key source of ossification, if the middleboxes can't understand the protocol they also can't rust shut extension points.