> But if I had to place a bet, it'd be that you aren't using IPv6. And you're nerdy enough to be reading a blog post on how much IPv4 addresses cost!
$ dig fly.io AAAA
fly.io. 590 IN AAAA 2a09:8280:1::a:791
And looking at the Chrome network inspector, I loaded the page over IPv6.
The reality is, pretty much anyone who uses a major ISP in the US has IPv6 (Comcast, Spectrum, etc. hand them out without customers even asking), and it's even more likely in other countries.
The big blocker right now is cloud providers. I lost IPv6 on my personal website when I switched from Linode to Digital Ocean (with a load balancer instead of putting them VM directly on the network). At this point, it sounds a lot like "we paid a lot for 3.0.0.0/9 so we're going to drag our feet on IPv6 to hurt competing cloud providers".
Cloud customers are somewhat to blame as well -- enabling IPv6 doesn't get you new users today, so nobody does the work to configure things twice. But the ISPs and the users are ready. It's just us, the tech elite, that's slowing everyone down. And it makes sense, because we burned a lot of money on IPv4 addresses and we don't want our imaginary money to become worthless.
I'm not sure it's the sunk cost of IPv4 addresses that's causing this. My guess would be that the "support IPv6" JIRA ticket gets perpetually thrown into the backlog because it won't get any new customers to sign up, so it's clearly not important from a "ship MVP" perspective.
Stack overflow's reply to "why do you not support ipv4" is more or less what you describe:
> It's just a lot of work for honestly not a lot of gain just yet. Things like HTTPS and HTTP/2 are far more impactful for users. We'll get to IPv6 eventually, but it's unlikely to happen any time soon. There are simply far more impactful things we can do with our time on the sysadmin front for the foreseeable future.
> This is something everyone on our SRE team would like to do. We just don't have the time yet, even more important things are in queue.
Other services sit on giant heaps of ipv4 space and see it appreciate. For them, price increases are wonderful, because it means they can charge higher rents for the ipv4 space.
Stackoverflow's quote you pasted makes me feel like an old fart on the Internet. I imagine there are a lot of rose-tinted glasses at play here, but I kind of remember an Internet when we would do things for the good of the network as well as for the good of the user.
Taking action as a group to speed adoption of a technology that makes the kind of innovation and autonomy we had in the first thirty years of the Internet accessible to the next thirty years of the Internet is impactful for users. The gigantic companies who have bought /8s and /9s of IPv4 space are not going to undermine their investments. It will take companies and network operators who are big enough to decide but small enough to still be idealistic who say "yes, IPv6 is worth it and we are going to spend the time and people and money and resources on doing it."
This is the same problem we have with housing; it can't be an investment and a thing everyone needs. At least the difference here is probably no one winds up hungry under a bridge taking ill-advised substances to handle daily life simply because they don't have a /24 to call their own.
Yeah, this development had advantages as well as disadvantages. I feel that websites of today have become easier to use for average non technical people for example than they were back in the day. This is what the greater user focus motivated by commercial interests brings you. On the other hand you see technical improvements being neglected like in this instance.
On the bright side, IPv6 adoption is increasing constantly, and if you fit the curves we'll have 100% adoption (or close) in the 2030s.
Apparently here in Australia (like much of the rest of the world) we still only have ~25% of our population on ipv6 networks. Africa is almost entirely dark.
Weirdly India is the world leader here, with 73% of devices being ipv6 capable.
OrgPad.com went to Hetzner from DigitalOcean because of this. We saved some money and actually get a comparable quality with IPv6. That is not bad at all. Of course, Hetzner don't have such nice graphs for the VPS, there aren't many features like managed databases etc. that DO has, but we don't need that since we manage on our own (which is also one of the reasons we get better performance for less money).
IPv6 at DO is really more for ticking a box instead of real production use. E.g. DO Spaces don't support IPv6. You can't run SMTP on IPv6, because DO uses somewhat non-standard /124 for a droplet and you can't have a IPv6 floating prefix/ IP.
> The reality is, pretty much anyone who uses a major ISP in the US has IPv6 (Comcast, Spectrum, etc. hand them out without customers even asking), and it's even more likely in other countries.
But Verizon FIOS does not, long after they should have.
> The reality is, pretty much anyone who uses a major ISP in the US has IPv6 (Comcast, Spectrum, etc. hand them out without customers even asking), and it's even more likely in other countries.
Unfortunately, there are still major ISPs in the US that don’t support IPv6. For example, Verizon (not Verizon Wireless) has been promising IPv6 for about a decade and has so far failed to deliver.
I'm from Quebec in Canada, the 2 majors ISP (Vidéotron and Bell), don't support IPV6. I don't know for others provinces, as they all have their own IPS (Bell is in a few of them to my knowledge, Vidéotron doesn't).
The reason simply is that when no type is specified, `dig` performs a lookup for an A record (i.e. IPv4 address). When you manually specify AAAA, it does AAAA lookup, which returns IPv6 address. (details in `man dig`)
One possibility: You might be getting that AAAA from a v4 DNS server (or resolver/cache), whereas `dig -6 fly.io` attempts the query over v6. For example, try `dig -4 fly.io AAAA` ("use a v4 DNS server address from /etc/resolv.conf to get the v6 (AAAA) record") versus `dig -6 fly.io` ("use a v6 DNS server from /etc/resolv.conf to get the address") - the latter may not work if you don't have IPv6 set up on your local network. Basically, your /etc/resolv.conf on your computer (or home router) is pointing at an IPv4 address for a DNS server, so that DNS server/resolver is probably defaulting to giving you a v4 record back.
Your browser, however, may also be circumventing all of this and may be using DoH to a public DNS server, and therefore may have a completely different DNS querying experience than what you're seeing with dig...so you should also take a look at what your browser's network inspector tells you about the host you're getting the fly.io content from.
I wonder if there's a good service for testing the IPv6 readiness of a website. Something like SSL Labs or Mozilla Observatory, but for IPv6 on the server side.
> One last thing: a new 5-digit ASN will cost you about $500 from ARIN, but there are auctions for 4-digit ASNs, and they run into mid-five-figures. If any of you can explain this to us, we'd be grateful.
To answer the authors question: lower number ASNs are seen as older more established networks when negotiating peering. It's the equivalent of doing a WHOIS on someone's personal domain and seeing when they started seriously internetting.
Obviously shorter numbers are more memorable and easier to type frequently, which is a desirable trait for network operators.
You probably spent a few bucks on "fly.io" and applied a mental value to the outward perception of a short memorable domain.
> It's the equivalent of doing a WHOIS on someone's personal domain and seeing when they started seriously internetting.
We have a moderately-sized regional network at my employer (that's not an ISP) and I've seen a snide version of this. Our previous Director of IT who also made day-to-day decisions on things like peering would refer to 32-bit ASN holders as "six-digit trash." In his view, anyone who didn't have a 16-bit ASN was virtually not worth talking to. Not for technical reasons like our equipment didn't support the newer ASNs, just that they were Johnny-come-latelys.
He was quite proud of the fact that our employer had and still held a sizeable quantity of legacy, as in pre-RIR, IPv4 address space. I'm fairly sure one of the reasons he quit was because of the decision the board of trustees made to transfer chunks of this space to other entities in our same line of work.
No need to apologize, we all didn't like him. If you'd told us that the easiest way to get him to depart in a huff would be to donate some unused IP addresses to other groups that needed them, we'd have deployed NAT64 and heaved the legacy allocations overboard in a long weekend.
This is why people have been trying to promote IPv6 for the last couple decades. It is silly that new internet companies should have to pay money to those who got here first.
Still waiting for AAAA records on news.ycombinator.com. IPv6-only SIP trunking would also be nice, since an IPv4 address is the main cost of hosting a small PBX.
Yeah, I truly do not understand what's causing Hacker News or Reddit to drag their feet. My guess is they've simply done the calculus and decided the number of impacted users doesn't justify the cost, and meanwhile, they already have their v4 allocation so there's no financial pressure either.
I'm starting to think the RIR's should just start charging increasing YoY maintenance fees on v4 address space for companies that haven't rolled out v6. Those companies, who already have an allocation, are really benefiting from a negative externality. Maybe it's time they start paying for the privilege.
Its almost an incentive to not do IPv6, then. ISPs have v4 working fine, its only the new guys that get screwed, so the incumbents can actually make life harder for competition by making sure everybody still need v4 addresses. Or, they dont want their "investments" to depreciate, kind of like why some NFA collectors oppose re-opening the machinegun registry.
IPv6 should have been designed to be backwards compatible and smoothly transition people over. Why not design it so you can just quietly migrate to ipv6 on the infrastructure side by introducing automatic translation of ipv4 packets into ipv6 and back by padding and removing zeros, and automatically reserving all the resulting ipv6 addresses to their ipv4 equivalent owners.
This situation is the result of poor design and roll-out decisions, not the fault of everyone else.
> IPv6 should have been designed to be backwards compatible
Impossible. You can't communicate between two hosts without both hosts being able to address the other -- and there's no way for an IPv4-only host to pack 128 bits of address into a 32-bit host field.
I don't think it's impossible per se. You could probably get around that limitation via a beefed up version of NAT, along with a couple DNS hacks. Obviously, that's some ugly hacky BS though.
I think better design could have avoided that issue to begin with by facilitating smooth roll over.
Still, I'd start with translating ipv4 addresses into a reserved range of ipv6 addresses. ...and if ipv4-only clients can't initiate new connections to ipv6 addresses outside that range, OH WELL. (There's probably wiggle room to NAT your way through anyway) More importantly, if there's a default in-built way for ipv6-only machines to initiate connections to ipv4-only machines, and the ipv4 machines can return traffic via some type of NAT, there's a lot more incentive to switch to support ipv6 since more machines would be using it.
What you describe there is basically NAT64 [1]. Not a ton of reason to actually deploy NAT64, though, when practically everything's going to be dual-stack anyways and you're not saving any IPv4 addresses versus that dual-stack arrangement with a typical IPv4 NAT. (There are also compatibility issues to consider; not everything can work with IPv6 addresses at the application level, especially early on in the transition.)
IPv4 addresses can be converted to an IPv6 equivalent [0] and that can be used to keep the existing owned v4 blocks intact for protocol translation. What would have been nice is if standards would have forced migration of owned blocks and pressed Tier0 / Tier1 providers to handle "legacy" v4 traffic on ingress into their ASNs. Carriers are the driftpins in the equation here, they have to start the ball rolling and there needs to be an interop transition phase. All the pieces are there, there's just no incentive.
Consider this: The OS reserves 10.0.0.0/8 for this purpose, possibly with a few common exceptions (10.0.0.0/16 maybe) being allowed through unmodified. If an ipv4 server has an incoming connection from an ipv6 address. That ipv6 address is given a corresponding 10.x.x.x address. Likewise, if an ipv4 client does a gethostbyname a 10.x.x.x address is allocated and returned. This simple system already supports 16m concurrent connections.
Sounds like you're describing NAT-PT, originally proposed in RFC 2766 (published in 2000).
It suffers from a lot of practical problems, which are described at length in RFC 4966. For example, without careful special handling it breaks protocols like FTP/SNMP/H.323 that embed IP addresses in data packets; it breaks IPsec; it can't handle fragmentation; and it suffers from scalability issues.
> it breaks protocols like FTP/SNMP/H.323 that embed IP addresses in data packets
Well it doesn't break them exactly. An ipv6 capable application would connect to the embedded ipv6 address and not be natted. But it fails to make a legacy application work, so you are no worse off at least.
I wonder if you could use something like IP in IP (https://en.wikipedia.org/wiki/IP_in_IP), basically just slap another IP header on your packet so your IP becomes two 32 bit IPs, one for your ISP and one for you.
That sounds really painful to work with, and it would still largely lack backwards compatibility -- hosts on both ends would need to be patched to support the "double address".
patching hosts would be easy, but the routers in between would probably not need any changes and that's big since it was one of the holdups of v6 deployment.
That exists for IPv6 in various variants: 6in4, teredo, 6to4, etc. (and the inverse variants of tunneling IPv4 over IPv6 also exist in many different forms).
Many of those can be used even if your own ISP has no support for IPv6 at all.
Many of the require manual configuration (as in the case of tunnel brokers) however, which makes it less likely for non-technical users to use them.
The idea would be to add first class support for it in the OS. Your ISP DHCP could add an addtional option specifying your upstream ISP address as well as your local address for example. Then your hosts interface will get configured with both addresses and automatically add wrap outgoing ip packets in the outer address.
This would also be possible with IPv6 (not with DHCP involvement, but with other mechanisms).
But the issue is that some ISPs (and cloud providers) simply don't care or even actively drag their feet (since it may be in their business interest to do so for now).
As I'm (slowly, begrudgingly) learning about IPv6 I can't help but agree. Astounding that backwards compatibility was abandoned in favor of ... Being able to hand out a /64 or /48 to anyone with a pulse? No residential customer needs that many addresses, yet I can call my ISP today and block that off for eternity. To what end?
IPv6 should just have been a superset and 0..1.xxx.yyy.zzz.ttt should have been the legacy IPv4 space. I really don't get the point of all the bizarre block assignments, hexadecimal fetish, and general "why is this seemingly purposefully such a pain in the ass" feeling I get whenever I look at it.
It is something like one /48 for each second square meter of land on Earth or something in that ballpark. It definitely can be exhausted, but that is only because we more or less use the last 64 bit for grouping hosts, autoconfiguration etc. which doesn't have to be the case everywhere (e.g. container hosts, where giving each container or even a container host a /80 can be more than enough).
The IPv4 packet structure looks like it has a lot of leeway to evolve/handle/crossmap to ipv6. especially with the version and "option" sections.
So ipv6 came out almost 24 years ago. And its adoption is a disaster. It's adoption rate was a disaster at the decade mark.
I don't understand why ipv4 wasn't evolved to interoperate with ipv6 at whatever level was necessary (servers, OS, routers).
I don't understand why there isn't an evolved ipv4 that can route ipv6 and ipv4 traffic to the same application endpoints.
I don't understand why there isn't a section of ipv6 address space that is "special" that contains all of ipv4 + possible ports for NAT.
I'm sure there are reasons, but I've also seen a lot of dogma from ipv6 people over the years that comes off as my way or the highway.
But why you can't get the big boys in the room in the modern corporate-consolidated internet (AWS, MS, FAANG, US government, europeans, mobile carriers etc) and work out how to get ipv6 transitioned.
To me, ipv6 was always a lot like the security group at large companies: they dictated a policy, tried to enforce compliance, but never offered solutions. And solutions is what gets your policy or protocol adopted.
IPv6 solves a problem that the big players want unsolved. IPv6 adoption would be a de-centralizing factor, and central powers are therefore against it. I have a hunch that the only reason that IPv6 has gotten any traction is because the absolute largest players needed it internally.
> we assign distinct public IPv4 addresses to each app running on Fly.io.
Why not charge a premium for even one distinct public IPv4 address? Most applications are HTTP(S)-based and could share a reverse proxy with a thousand other apps, right? You could even spin the lack of distinct public IPs as a positive: zero IPv4 footprint.
The post talks about this (I take your point that we didn't write about it clearly enough). There are two reasons:
1. We don't want to do anything to discourage people from building non-HTTP applications on Fly.io, because we're fans of weird applications.
2. Just giving every app a routable address simplifies our own deployment logic (at the expense of acquiring blocks of IP addresses).
Really, the reason we were moved to write about this is (2); specifically: so long as IPv4 addresses are appreciating assets (as current buyers, we can report: they remain appreciating assets), then there is a sense in which holding IPv4 blocks is really just holding money in a different, somewhat less liquid (but potentially profitable) form.
We're not saying that's a good thing, just that it's interesting that you can essentially take out a mortgage on a large block of addresses and live in it while it appreciates.
> Most applications are HTTP(S)-based and could share a reverse proxy with a thousand other apps, right?
Try running your SaaS app on an IP address shared with a porn site and you'll quickly discover all the strange ways older middle-boxes try to police network traffic.
We recently received a /23 block from ARIN by sitting on the waiting list for about 2 months. If you're not in a hurry and you're not looking for a large block of IPv4, wait. Small companies still have a chance at IPv4 without paying a fortune.
Oh, this is very cool. What is your company doing with the addresses? I wonder if there are applications where it's easier to get addresses than others.
This is a hard market to time. Nobody
believes the Internet will be IPv6-only
within the next few years. There are
credible people who believe IPv4 addresses
will be scarce and useful indefinitely.
We might put money on you getting away
with an IPv6-only app 20 years from now.
But we'd have said that 20 years ago, too.
This was interesting to me, so I looked up adoption metrics.
"An ISP or site administrator can easily run a huge number of CurveCP servers on a single global IPv4 address, even if the servers are independently operated with separate long-term public keys. This feature is provided by a simple extension mechanism in CurveCP addresses.
CurveCP servers are inherently anti-aliased, providing automatic virtual hosting and fixing some of the deficiencies in the "same-origin" policy in web browsers. This feature is provided by a simple domain-name mechanism in CurveCP addresses.
If a site has two server addresses, and one server is down, a CurveCP client will quickly connect to the other address.
A CurveCP connection remains fully functional even if the client changes IP address.
CurveCP is fully compatible with existing NAT (network address translation) mechanisms; none of the above features require clients or servers to know the global addresses of their gateways."
I will keep using CurveCP forwarders on the home LAN because I like it and just in case it ever sees interest from a wider audience, like Curve25519 eventually did. No expectations.
If a site doesn't use SNI, therefore serves no more than one HTTPS site from any single IP, don't you still leak the same amount of information simply from connecting to that IP on port 443?
No. SNI sends more information. It sends a "servername" in the ClientHello. With earlier TLS versions the host could also be leaked in the server certificate that is sent to the client to be verified. With TLS1.3 that is fixed. All that remains is to plug the SNI leak. Experimental encrypted ClientHello is available, but not in popular graphical browsers.
Popular graphical browsers send a servername in the ClienHello even when SNI is not required! (Most HTTPS site submitted to HN do not require SNI.)
But since there’s no ability to share IPs between different sites without SNI, haven’t you just made things easier for censors/snoops? Instead of having to sessionize TCP and decode TLS, they can get everything they need with stateless L3 monitoring.
The reality is, pretty much anyone who uses a major ISP in the US has IPv6 (Comcast, Spectrum, etc. hand them out without customers even asking), and it's even more likely in other countries.
The big blocker right now is cloud providers. I lost IPv6 on my personal website when I switched from Linode to Digital Ocean (with a load balancer instead of putting them VM directly on the network). At this point, it sounds a lot like "we paid a lot for 3.0.0.0/9 so we're going to drag our feet on IPv6 to hurt competing cloud providers".
Cloud customers are somewhat to blame as well -- enabling IPv6 doesn't get you new users today, so nobody does the work to configure things twice. But the ISPs and the users are ready. It's just us, the tech elite, that's slowing everyone down. And it makes sense, because we burned a lot of money on IPv4 addresses and we don't want our imaginary money to become worthless.