It should be noted that this is a map at what we would call layer 3 in the OSI model. ipv4/ipv6 space and BGP relationships between ASes.
Giant transit ISPs like Cogent own little if any underlying fiber infrastructure, which is much harder to map and have 100% topology discovery of by researchers sitting at desks somewhere. Lots of transit ISPs like HE, Telia or Cogent buy inter-city transport from "carrier of carrier" specialists who own the underlying assets, run DWDM systems on them, and so forth.
Without proprietary information from carriers' GIS systems (at Level3/Lumen/Centurylink, Zayo or similar) it's much harder to do that.
What are they providing if they don't own the cables? Are they shared with other non-ISP issues, or is it more like when a company builds an office building and then leases it from themselves?
what I described above would be at layers 1 and 2 in the OSI model (the cable itself and underlying physical assets)
a company like Cogent, HE, they're providing at layer 3 the routers, network engineering, IP space (in many cases, not all, mostly for enterprise customers), and what is known as "IP transit" - transiting a customer's traffic through their network to all of their peers. And maintaining the cost of buying all those inter-city transport circuits to link all of their routers within their AS together.
> this project aims to concisely, but still comprehensively visualize the current state of the World Wide Web, and document the largest and most popular websites
You don't want to call it the map of the Web instead?
Speaking of technicalities someone could argue that when measured by protocol usage statistics, the web is (almost) equivalent to the internet.
When following where DNS is heading and why DNS over HTTPS is preferred over DNS over TLS to avoid censorship and manipulation, this kind of reinforces the point.
Most areas of the internet, and most clients on the internet these days use HTTP/S as a network protocol. Something like an IRC client on mobile is not really an IRC client.
(I'm purposefully ignoring sstp, snmp and dgram sockets in the gaming sector here, because they are actually marginally small compared to "all of the internet". I also think routing layers can be ignored, because a consumer's layer 1 is their layer 6 anyways; and someone could argue that it's infrastructure and not a usable internet by itself.)
Even when arguing that smtp and imap play a huge role...lately even Microsoft pushes cloud based office suites, so I'd argue that is also not the case anymore.
Bittorrent alone probably accounts for a huge chunk of internet traffic, and it's not http. SMTP and NNTP probably also account for large fractions because the MTAs still speak SMTP, not HTTPS and Usenet itself is something like 100 TB per day.
Either way, arguing that the web == the internet based on protocol usage is to me a bit like arguing TCP == HTTP/S. They're different things at a conceptual level and the semantics are an important distinction to a lot of applications.
These kind of semantics are not really important for the masses.
As I said, the end user's OSI layer 1 is the infrastructure's OSI layer 6 anyways. How the routing layer is achieved doesn't matter, as the masses use HTTP as a protocol (Android, iOS, Browser).
Sidenote: The torrent protocol and BENCODE is usually encapsulated in its transport layer (and yes, it's HTTP/S) as raw UDP is blocked and can't be used to discover peers in most settings due to carrier-grade NATs not offering a way to have a public IP anymore.
I never denied that other protocols exist, hence the reason for my previous comment.
But when people actively shame others for not knowing how routing infrastructure is solved, I think the argumentation baseline is wrong on the technical side that should know better and therefore argue better.
Users use HTTP. Mobile E-Mail clients that use SMTP/IMAP are dying out and moving over to HTTPS infrastructure and REST/JSON based APIs.
Look at G-Mail, look at Outlook, look at any large-scale E-Mail provider's Apps in the Play Store or App Store. The AOSP E-Mail client is the rare exception, and their network traffic gets blocked in most countries I've visited when not using a VPN.
My comment was about the argument that other protocols exist for legacy reasons, and I was challenging the general assumption in the sense that mobile took over most portions of the "internet". And mobile only uses web-based network protocols.
My argument was that in the future, it's very likely to see other protocols for our infrastructure to move over to HTTPS as a transport layer. SMTP/IMAP via HTTPS or something similar is very likely to be once DNS via HTTPS is rolled out and the default.
> Either way, arguing that the web == the internet based on protocol usage is to me a bit like arguing TCP == HTTP/S
Coming back to the argument that someone might also argue that TCP==HTTP/S:
As all internet traffic is encapsulated by SONET on the ISP side, the very same argument would be that UDP and TCP aren't used (so VC == TCP/UDP?) - as the real transport layer is VC4 or similar containerized point to point protocols.
We can also play this a little longer up to the point where cables are not really cables anymore because of fibre infrastructure and how wavelengths are multiplexed, but that wasn't my point.
My point was that given the "internet infrastructure" of users having a router box thingie at home and they connect to their view of the internet, they'll use HTTP most of the time.
ISP-provided routers usually don't even have a configuration capability to unblock other protocols anymore, so other network protocols will very likely die out soon because UDP 53 and HTTP are always allowed (hence the reason for the DNS via HTTPS movement).
And having worked at an ISP to know these things is a very rare conditional state, and therefore that specialized knowledge cannot be extrapolated to the general society.
>We can also play this a little longer up to the point where cables are not really cables anymore because of fibre infrastructure and how wavelengths are multiplexed.
Indeed we can, and we'd soon arrive at my original point again which is that conflating things like "the web" and "the internet" or TCP and SDH or $thing and $related_thing might win you some imaginary internet points on HN but these things exist as distinct concepts for a reason, and that reason is because the distinction is relevant to certain groups of people.
> chances are that they use an app instead of a browser or ask Alexa
which in return is either a WebSocket, WebRTC or HTTP/S stream and kinda reinforces my point that the Web and Internet are equivalent metaphors for most users.
What percentage of the properties in the map are primarily accessed through a proprietary app? Facebook, YouTube?
I will concede that deep linking makes apps more web like, but the hybridization of accessing places through web sites AND apps, and the expectation of both options, makes this MORE than just the web to me.
Since these continents provide more than just websites; they provide mobile (iOS and Android) and other services, I think the original title is apropos ...
Link to PNG file: https://www.caida.org/projects/cartography/as-core/pics/2020...