Things like that makes me wish NDN[1] would succeed.
Essentially in this network instead using addressing to address hosts, they address the data. This brings interesting properties, since routers are aware what data is being transferred they can start caching it.
In the end no CDNs are needed (network takes care of it) so people can host data themselves without worrying of it getting too popular.
Except the end-user ISPs are quite content with requesting traffic from the providers; they like the imbalance as it gives them something to negotiate with (that's why inbound is free on cloud - they're happy to fight the imbalance). Why would they start caching your images for free?
Also doesn't that mean everyone in between know you're requesting degeneration.jpg? Vs now where they'd only know the domain and maybe could do transfer size analysis? Sort of the opposite direction of TLS everywhere (CloudFlare exempted)?
Also how do you bootstrap it? If I upload some image and it's instantly popular on reddit and generating many Mbps of traffic, don't I have to stick around long enough to get it up and cached? Or would there be free seed services that will get the first several GB out?
> Except the end-user ISPs are quite content with requesting traffic from the providers; they like the imbalance as it gives them something to negotiate with (that's why inbound is free on cloud - they're happy to fight the imbalance). Why would they start caching your images for free?
Normally that would benefit ISPs (they do have a network of routers so it would naturally help them, since they would not need to send the same thing over and over again, they do pay for traffic to their uplinks).
But, on the other hand ISPs artificially reduce their bandwidth and expect companies like Netflix to pay and do peering with them. Well, this won't help against it, but while it can replace TCP/IP it can also work as an overlay network and you can skip the ISP.
> Also doesn't that mean everyone in between know you're requesting degeneration.jpg? Vs now where they'd only know the domain and maybe could do transfer size analysis? Sort of the opposite direction of TLS everywhere (CloudFlare exempted)?
The protocol itself doesn't leak that information. Remember, there are no addresses to individual machines to people, only addresses for the data. When you request something the routing protocol will route the request to the source of data. Each router remembers the previous hop the request came from. On each hop a router checks if it already caches the data and responds, otherwise forwards request further. So even the source website doesn't know who requested it. I suppose the issue might be that router immediately would know what you requested.
You can mitigate it in two ways:
- encrypt the data and only give key to decrypt it to subset of people who supposed to be able to access it
- encrypt data individually per user, in that case you no longer can rely on caching properties of the network and the source server needs to be aware who requested what (since it is encrypting the data). i.e. you falling back to the way how TCP/IP now operates.
> Also how do you bootstrap it? If I upload some image and it's instantly popular on reddit and generating many Mbps of traffic, don't I have to stick around long enough to get it up and cached? Or would there be free seed services that will get the first several GB out?
Well you're expected to host it all the time. The network won't be caching it forever individual packets have TTL and also routers can purge data if they need to make space for other data that is now popular.
I suggested NDN as a response to that S3 bucket doesn't handle highly popular content well. The NDN properties allow highly popular content being hosted even from places with low bandwidth, because the network will handle the load.
Not really. IPFS from what I understand is a P2P application, with filesystem like functionality.
NDN on the other hand is a network protocol that is capable to run without TCP/IP (it's actually designed to be able to replace it). They actually built a testbed: https://named-data.net/ndn-testbed/
Similarly to TCP/IP you can build application that utilize the protocol. In fact I suspect implementing IPFS in NDN would be much easier than in TCP/IP.
Essentially the big change is that instead of using addresses for servers what if you put addresses on the data itself. That approach gives some properties:
- network can cache the data - no CDNs needed, no slash dot effects etc
- free multicasting (a hard problem at scale with TCP/IP)
- better handling of slow/lossy network
- easy multipath routing
etc.
It's very efficient for things that can potentially have many people interested in. Like streaming popular videos, hosting images (like discussed here imgur). The benefit for these sites is that they would have lower costs to operate because they would require less bandwidth, because anything that's popular will be cached by the network.
In a way, it also was created by people who contributed to developing Internet and is NSF founded. The head behind it is Van Jacobson[1] who was responsible for implementing congestion control in TCP/IP also creator of popular tools such as traceroute.
Unlike FreeNet it was designated that while it can work as an overlay network it is actually capable to run without TCP/IP so it could even replace it.
Also unlike FreeNet it's not trying to obscure who is hosting the content. Basically you get a prefix (which looks like unix file path) which you start advertising to neighboring routers (like IP is advertised in TCP/IP) so then all requests under it (as long as no cached version is available anywhere) will be forwarded to you.
I imagine being an image hosting service it would be cheaper in the long run, although I have no idea by how much and if it would make sense all things considered.
