The lack of usage in static blocks isn't entirely surprising. A lot of universities got their IP space early and utilize it poorly. I know this because it is in part my fault.
When we were assigning IP addresses for the dorms at Berkeley, we gave every dorm a /24, and then reserved the first 50 IPs for "future use" and "internal use". Most of those were never used. And unless 200 people signed up per building, some of the top end was missed too.
My understanding is that they have since fixed this on the wifi since many people bring three or four devices now, but the hard wired connections are still poorly utilized.
My fraternity house had a housing license for up to 22 people. We had a /16. Though, my understanding is that only the first /24 is currently routed to the house, and we moved to a larger house that's licensed for 50-something residents.
Recall AllAdvantage.com, which had a pyramid-scheme style payout for installing spyware on your computer and browsing the web. Payout capped after browsing 48 hours per month with the spyware on. One of the guys trimmed down Win95 and bought a student VMWare license. A gig of RAM was about $4k at that time. He scripted everything up so that at boot time, the Win95 guest would mount an SMB share from the host, remove the first line from a shared CSV file containing fake account details, and proceed to register a fake account at AllAdvantage with him as the referrer, and randomly browse for 48 strait hours (using a local caching web proxy) like a meth head on a bender, and then shut down. Monitoring software on the host would see the shut down VM and spin up a new one with a new IP. The VM IP assignment code hopped all over the /16 to reduce suspicion. AllAdvantage went out of business before it failed to ban him. 30 VMs ran simultaneously. In a 30-day month, he had 450 VMs run, and got just under $2 in referral fees per VM. The fake accounts never made enough money to get cheques mailed to their fake addresses, so AllAdvantage didn't get returned mail or un-cached cheques, but he was still very surprised that he never got caught. He figured he had less tan 50% chance of making it 4 months and paying off his 1 GB of RAM. He wasn't really doing it for the money, but saw it more as a kind of cat-and-mouse game.
He was always doing stuff like that, enjoying being the mouse in a cat-and-mouse game, with almost none of it for money. He just got a kick out of seeing people doing silly things and demonstrating the silliness of their ideas. You can get into a fair amount of mischief with a /16, which is presumably why now they only route the first /24 to the house.
Yes, he made a few thousand dollars net profit over 7 to 9 months.
He also had a poker bot running for a while (on sites that didn't mention bots in their ToS) and kept complete logs of cards seen and player actions. There were plenty of players who could beat his bot, but his bot would refuse to sit down at tables with players with too good records against it. With three credit cards running three accounts, that poker bot was paying his rent for the first year or two after he moved out of the fraternity house.
He told me a couple of times that he'd give me FTP access to his logs, but never got around to it. When I heard that RC4 had a bias in its key schedule, I thought a bit about how to make an unbiased shuffle, and figured that a lot of people would use a naive biased shuffle. I was curious if my friends poker hands would show such a bias. I also wanted to check against the Perl, GNU C, MS C, and Visual Basic built-in rand() implementations, checking for srand(time()) and srand(time()^getpid()). (Perl still ruled the web in those days.) Later, an academic paper came out explaining that the most common poker site software was written in Pascal, and had both flaws I wanted to look for (plus another flaw due to an off-by-one error in the author's understanding of the Pascal Random() API.) I didn't think to look at popular Pascal implementations' pseudorandom number generator implementations, so I doubt I would have found the predictable prng seeding flaw via my friend's logs, but there's a chance I would have. I'm not sure how much an edge in poker the shuffle bias flaw gives, but I imagine it's tiny.
To me (a non-expert) it feels that we have the same problem as land use. Vast unused lands are just sitting there fenced, while some other parts of the town is crazy crowded.
Of course, IP address might not have the same physical boundaries, but instead they have organizational barriers, which can be equally tricky ones, if not trickier.
The bottom line is that we suck at resource management and distribution at nearly every field.
The answer for land was property and inheritance taxes so families couldn't just sit on valuable land non-productively forever.
If it cost $1/year*address most entities with a /24 or /22 would barely notice, but HP might decide that they don't need 33.5 million IP addresses after all.
The problem is that the growth of the internet is up to now exponential. Even if we can get a few more years by spending inordinate amounts of money on reshuffling IPv4 addresses, it won't solve the underlying problem. It'll just delay the inevitable IPv6 deployment.
I've been reading about imminent address space exhaustion and the pending switch-over to IPv6 since I was proud of my five digit slashdot ID. It's easy to get cynical about it.
Oh, for sure, people have been warning about it for almost 15 years now (got my first tunnel in the early 2000's). But the difference now is that we've actually reached exhaustion, so I don't think it's fair to say nothing has changed.
It was the same thing when I was at University of Delaware. Each hard wire got a IPv4 public address. I lived in one of two apartment towers. From what I recall there were three /24s split between the two.
Much like your setup the first 25 were reserved. In addition anything past that was rate limited. I recall anything past that was ~10->25Mbs. Going into the lower range though got around 500Mbs. I know similar stories from other state employees. The times when IPs were more prevalent.
Same thing at SUNY Polytechnic (previously SUNYIT). Its been a few years since I left but when I was there all hardwired devices had a public IP address.
I have several clients that have a class B or class C and use it mostly for their internal address space instead of using private address space. What a waste.
It's a very good way to make sure that if those clients need to route their internal networks to each other, they don't have to renumber. If two companies using 10.x.y.z networks merge, you are likely to end up with a mess. Heck, if you need to VPN to a 10.x.y.z network and your local network is also 10.x.y.z, you'll end up with a mess.
(The IPv6 solution here is Unique Local Addresses, where fdXX:XXXX:XXXX::/48 are all permissible local networks, and if you use a decent RNG to generate the 40-bit number XXXXXXXXXX, you're unlikely to hit a collision with any other actual, active site, let alone one you might want to route to.)
Put in your MAC address, get back a ULA. Then register it so that in the future if someone happens to have the same mac address you don't accidentally use the ULA.
I'm following the RFC: "Locally assigned Global IDs MUST be generated with a pseudo-random algorithm consistent with [RFC 4086]. Section 3.2.2 describes a suggested algorithm."
I happen to be of the opinion that /dev/urandom is more likely to comply with RFC 4086 than the suggested algorithm, but I may have an unfairly low opinion of the distribution of timestamps and MAC addresses.
Edit: If you have some time and like high quality networking commentary (aka 'rants'), I strongly recommend to listen to this packet pusher episode with Geoff Huston, where he plays devil's advocate for IPv4.
(I know this is drifting off.) Geoff Huston is a phenomenal speaker (and researcher of course). If you have even more time and want to learn more about how the address spaces and routing tables are evolving, I definitely recommend his talk on BGP at RIPE 68 (2014) which I was lucky enough to attend.
A note -- if you're linking to arXiv, it's better to link to the abstract (http://arxiv.org/abs/1606.00360) rather than directly to the PDF. From the abstract, one can easily click through to the PDF; not so the reverse. And the abstract allows one to do things like see different versions of the paper, search for other things by the same authors, etc. Thank you!
I'm happy that my ISP gives me a /29 ipv4 block :).
An example of a real world effect is one of the largest fibre ISPs in the U.K. They are the only provider afaik to assign users internal IPs, which is then NAT'd. This then gets NAT'd on the router again - double NAT, woohoo! :) I know they desperately tried to get enough public space for all their customers, but were unable to buy a large enough space for it, so were forced into this position.
Interestingly, the U.K. has at least two /8 ranges that aren't even advertised on the public internet, owned by MoD (25/8) and Department of Work & Pensions (51/8). That's 32 million addresses unused as far as anyone can tell from the outside.
Well it's not like they could have deployed IPv6 (with NAT64) instead (/rant). Most ISPs around here (France) provide IPv6, some of them since like 10 years ago.
Not supporting IPv6 today is just ridiculous, and I've jumped ship† regularly during those 10 years, explicitly mentioning IPv6 (lack thereof or unsatisfying support) as the reason when cancelling. I understand that not everyone has this chance (due to choice availability) to pressure their ISPs, but seriously they weren't forced: they dug themselves into this hole.
† One of them even dropped a previously perfectly functional IPv6 support for me purely due to a contract update and argued it wasn't possible to bring it back. Seriously.
Question - given that they've condemned you to an awful carrier-grade-NAT, do they at least support IPv6?
I'm of the opinion that this is probably the future for all residential internet users - ISPs that put everyone on an IPv4 NAT so that grandma can still get her hotmail and yahoo, while hopefully providing unencumbered IPv6 access for everything else that requires a real network connection.
But then a lot of software still only supports IPv4, e.g. Dropbox or Resilio Sync. In the latter case that gives me some serious headaches, it has the tendency to open a lot of connections at once. I think my ISPs DS-Lite uses some quota, because when I have sync open on more than a certain number of devices, IPv4 connections start getting dropped and IPv4 is basically unusable for 5-10 minutes. I am pretty sure it's related to DS-Lite, because it doesn't happen to devices that are connected to a VPN.
It would help me a lot if all (mostly proprietary) software vendors would finally start supporting IPv6.
AAISP and Sky are the only two ISPs to provide IPv6 in the UK. BT Consumer have been promising it "real soon now" for the last few years, and the rest of the ISPs are sitting on their hands.
I was pleasantly surprised when I was given a global IPv6 address at my parents house (where they use Sky). I had no idea they'd already managed to do it, and was expecting them to drag their feet for as long as possible.
AFAIK, Sky has rolled out IPv6 to all staff now. Except, well, it's enabled at a cabinet level (AIUI) so everyone served by the same cabinet has IPv6 now.
Sky gave an interesting technical presentation to the UK IPv6 Council in June about the roll out[1]. There is another presentation scheduled for October if you want to know more[2].
Yep Zen is great! That's where I got my /29 from. They also fix issues very quickly - when netalyzr complained that my ISP couldn't handle fragmented udp packets, I let them know and they fixed it within a few days.
That's fairly normal these days. The combination of ipv6 + cgnat for ipv4 (carried over the v6) is called DS-Lite[0]. ISPs in asian countries have been doing that for years now european and american ISPs have been ramping it up recently too.
ISPs in asian countries have been doing that for years
And they usually call it "web access" or something similar, never speaking of an "Internet connection". Which is what it is, a way to access websites and usually not much else.
Is it really? Apart from the missing port-forwarding, DS-Lite seems to be one of the better solutions to the IPv4 shortage to me. There's only a single NAT at the ISP, the local router tunnels IPv4 packages directly to the AFTR. For peer-to-peer applications, both UDP and TCP hole punching work fine.
Maybe some ISPs will implement the Port Control Protocol [1] at some point, which would allow port forwarding with the DS-Lite NATs.
> For peer-to-peer applications, both UDP and TCP hole punching work fine.
Sadly, that's only half correct. Yes, nat traversal usually works to establish connections. But in practice port mappings are not necessarily the same thing as NAT table entries. If your p2p application contacts a lot of endpoints, even from the same source port, this can eventually lead to saturation. At that point you'll get packet drops and ICMP errors.
In other words, on some aftr implementations p2p can lead to resource exhaustion, leading to a pretty bad ipv4 experience.
> Maybe some ISPs will implement the Port Control Protocol [1] at some point, which would allow port forwarding with the DS-Lite NATs.
Some already do. The CPE can forward local mappings to the AFTR.
They're only looking at IP addresses that access a big CDN as clients. They won't see servers that way.
As mobile devices migrate to IPv6, the address space problem should be less. It's too bad that most mobile devices don't have permanent IPv6 addresses - more peer to peer applications would be possible.
I'd say it's more like most mobile devices don't have IPv6 period, at least not here in the US. Every time I've done or seen network tests on a phone on the major cell carrier networks here in the US, there's been no IPv6 to be found.
Public IPs are about $10/each on the market. If you're running a serious service, $10 is a fairly small cost, so you just buy whatever you need. IPv4 exhaustion has mostly affected people that were getting addresses for "free".
A couple of months ago I sold two networks (a /22 and a /24 ) that I had accumulated over the years. I got closer to $5/ip but a fairly small volume. A little bit of paperwork but not too bad.
I thought it was interested that only a couple of years ago paying for IPs was very much against policy and now they even have pointers to brokers on the APNIC website.
> I thought it was interested that only a couple of years ago paying for IPs was very much against policy and now they even have pointers to brokers on the APNIC website.
Selling things for what the market will bear is the only way we've managed to figure out to fairly provide resources to those who want them the most.
At least in the US/EU there is no such thing as a "carrier license". You simply register with an RIR, Pay to "lease" an AS number, request (or buy/transfer) address space.
Once you've done that, buy transit from a provider in your data center or office building, set up a BGP speaker (Quagga, BIRD, GoBGP), and announce your IP space (has to be at least a /24 for IPv4, and a /48 for IPv6).
Sorry, I should have specified I was talking about AU.
To sell internet connectivity here you must have a carrier license from ACMA. I've heard that it costs ~3k to get one. Which makes setting up an neighbourhood ISP just expensive enough that no one bothers. (this is on top of all the normal costs you mentioned)
mostly affected people that were getting addresses for "free".
There are still a lot of "free" IPv4 addresses out there.
The problem is the cost of IPv4 addresses is not evenly applied. There's hoarding by some early adopters. E.g. I see that Stanford gave back its original /8 allocation, but MIT did not.[1] And why is Prudential Securities still sitting on a Class A block?
If MIT or Prudential had to pay 16,777,216 * $10 per month for their IP addresses, you can bet that most would be returned within 48 hours! Even if they had to pay only $16 million per year, you can bet they'd return most of those addresses.
A true capitalist solution to the IPv4 "shortage" never happened. That's why you have the current situation. Some organizations are sitting on huge swaths of unused addresses, others (the free market) are forced to pay $10 per month.
There are about 4 billion possible IPv4 addresses (sure, some are reserved for e.g. multicast, but those could have been reclaimed). If each and every IPv4 address cost $10 per month, there would never have been a need for IPv6. Even if each and every IPv4 address cost only $1 per month (about $50 billion per year in aggregate), there would never have been a need for IPv6. Or at least IPv6 could have been postponed for a while and "done right".
Note that a secondhand IP address costs $10 to buy, not $10/month.
IMO Postel's allocation of IP addresses can be rationalized from a capitalist viewpoint as a form of homesteading. Some people end up with windfall profits three decades later, but it's a small price to pay for a peacefully functioning market. Owning a $100M asset that's underutilized is already an economic incentive to free up addresses (just don't tell my company); a Georgist property tax on IP addresses would provide even more incentive but it would also likely cause a revolt.
Since there are going to be 4B devices on the Internet soon if not already, talking about "there would never have been a need for IPv6" seems to imply the existence of an address-less underclass.
Note that a secondhand IP address costs $10 to buy, not $10/month.
Yeah I really screwed up on that one!!!
I got confused because often the retail price is in that range. E.g. Comcast Business will give you a static IP for $20/month.[1] But reading further, they will give you 13 extra IPs for $40/month, so clearly I didn't think it through.
The market for IPv4 addresses is ultimately a band-aid solution because it affects routability. The more efficient such a market becomes at distributing addresses topologically, the less efficient packet routing becomes.
IPv4 markets are a dead-end and are no substitute for a real solution.
The real problem is the old big networks, and it would be free to chop those up into /16 blocks, and cheap to chop them into /20s. There's not really an efficiency worry. It's a matter of putting in more megabytes of memory for the BGP tables. We could shatter the network into 15 million /24s and still be able to route everything efficiently.
There are more than a few old routers still in use, RAM maxed out, that are bumping into their limit. Whenever an aggregated net gets accidentally split, they start failing.
On the other hand a lot of those failed at the same time two years ago when active BGP entries went over half a million, so now's a better time than usual to add a few entries.
For reference, there's about 630k active BGP entries, with 50k more per year. It would only accelerate the death of those routers by six months, 24k entries, to split up everything larger than /16.
Feels like real estate. The move to IPv6 could be seen as a socialist land redistribution scheme, since it imposes a cost upon the entire system (by requiring network upgrades) in order to make cheap "real estate" continually available to the masses.
There is no redistribution going on. Everybody who has IPv4 addresses now is free to keep them.
If you want to compare to real estate, it is more like building a completely new city in the middle of nowhere. Initially the costs are high because you have to build new infrastructure. Initially, there is nobody there so there is no incentive to move to the new city.
As the new city gets popular, everybody who has real estate in other cities may feel compelled to get real estate in the new city as well, but they don't have to give up their old property.
Because DigitalOcean fought, and possibly paid, for their blocks for you. I don't do it any more but even in 2010-2011 getting blocks from ARIN took real work (ask for /18, get /19, lots of justification and explanation, etc).
Now everybody's pretty much out. So I don't know DO's blocks, but very likely if they're running out they're having to buy more. Depending on how they cut it up a /16 -- I know they have at least one /16, because it came up here yesterday -- can serve at least 60,000 droplets or so.
The main justification for getting more space is to show projected usage based on past usage. So if I get a /19 and use it in one month instead of three, I'll get an /18 next time, because that's about how much I'll need to sustain my growth. Rinse and repeat. Speaking on DO, it's likely that they grew so fast in the beginning they were able to acquire an awful lot of space.
I've pondered the IPv4 issue a lot, having recently acquired a /24 block for running an Anycast network (blog post coming soon).
People have asked me my thoughts on IPv6 adoption, and I have to honestly say I'm pretty bearish on IPv6 being quickly adopted. I try to enable IPv6 on all my friends' routers, but even today, many routers, though they support 6 (notable exception being DD-WRT.. WTF), don't enable it by default, which means they don't enable the dual-stack configuration needed for transition. People then place these routers in a dusty area under their computers or behind their couches, where they sit, basically untouched, for 10+ years until they blow up and require replacement.
The people I've talked to that happen to control a lot of IPv4 addresses tell me that even at the current ~$10-12/ip strike price, nobody is interested in selling because the IPs are more valuable to them for use with datacenters and leasing than for selling them at auctions. And nobody's putting pressure on the people that own huge IPv4 subnets (and I genuinely doubt use all of it) to start splitting them up and releasing them. Not to name names, but the original developers of the Internet come to mind (huge research universities like MIT).
As such, I'm expecting the price of IPv4 addresses to increase substantially over the next 10 years, and I'm really not anticipating that price to drop for longer than that even. Because at the end of the day, if you want to support everybody, you need 4. And even today, dual stack is not the default option (despite ISPs like Comcast being ready for it). IMHO, If you need IPv4s for something like an Anycast network or a hosting service, the time to get them is now.
Most people run the modem/router combo that their ISP provides them. The ISP can flash that modem remotely to upgrade it to IPv6. As far as I know, that's how Comcast in the US managed to deploy ipv6 massively. It often surprised me that in some small US towns, I'd be in AirBNBs or B&B, and they had IPv6.
In my neighbourhood in Canada, looking at the wifi routers, 80% of my neighbours are running the default configuration of their ISP.
Also, people need to replace their modems more often than we think. In the past 5 years we had the switch from 5mbps DSL to FTTO (25-30mbps) and now the switch to FTTH. A neighbour switched from 5mbps DSL to FTTH, for basically the same price (but the same ridiculously low monthly quota). He's using FTTH for an empty house to connect his alarm system to the monitoring company. In 6-12 months (rumours were 2017-Q2 unless they * again), when Bell finally decides to enable IPv6, that network will be IPv6-enabled.
Let's count the ways this is broken: IPv6 is off by default (and it sounds like some builds don't have it at all). To enable it, you have to enable two settings (presumably if you only enable one you'll end up with half-broken IPv6). Firewalling is not included. There's no mention of prefix delegation, but there's plenty of instructions for configuring deprecated 6to4 or deprecated tunnel brokers.
Yeah, this is beyond broken. On my DLink router at home, I had to literally click one "setup" button to get IPv6 working. That's how it should work: either be very simple, or work by default out of the box (preferred).
Which means it won't be available for people that want to run their own IP subnets. I doubt they'll be splitting it into /24-/16s and reselling it anytime soon.
Assuming consumer-grade router manufacturers started fully enabling IPv6 by default (and ISPs did too), how does Grandma get the automatic firewall-ish security benefits NAT provides? Does the end of NAT mean Grandma has to start caring about firewall settings?
By using a stateful firewall with a default-deny policy on the external interface?
One would hope that SoHo routers that ship with IPv6 support are configured that way by default so they mimic the apparent behavior of NAT (though NAT is not, and is less effective than a firewall).
Less memory? A complete inside-outside-remote address triple for IPv4 NAT uses less memory than even a single IPv6 address. And keeping state requires two.
It does provide somewhat of an isolation on the WAN side.
If your grandma installs ElasticSearch on her box, she won't get pwned because it listens on 0.0.0.0 on her laptop and she didn't get the latest updates and is vulnerable to RCE.
Consumer-grade routers that support IPv6 out of the box (in the official firmware) normally block incoming IPv6 connections from WAN side by default, so it won't really be different from NAT security-wise.
NAT doesn't isolate your router, but it does isolate devices behind your router from unsolicited WAN-side connections (modulo hacking the router itself and hijacking sessions, of course). How is this not a security benefit?
On this topic, does anyone have any advice for pressuring ISPs to move on IPv6?
I've tried calling and public shaming Wide Open West to no avail. They don't seem to care at all. Time Warner Cable is the next fastest ISP (and they do support IPv6), but they want almost the same price for 1/12th the speed (600mbit/s WOW vs 50mbit/s TWC); so I begrudgingly stick with WOW anyway =(
One thing I cannot understand about the transition to ipv6 is why weren't all ipv4 addresses converted via encoding to ipv6. This way ipv4 and ipv6 could talk to each other.
And all service websites could jump on the ipv6 bandwagon without hiccups.
By encoding, I mean every ipv4 segment would be encoded as a hexadecimal set and that would be merged into an ipv6 category under a special prefix.
When we were assigning IP addresses for the dorms at Berkeley, we gave every dorm a /24, and then reserved the first 50 IPs for "future use" and "internal use". Most of those were never used. And unless 200 people signed up per building, some of the top end was missed too.
My understanding is that they have since fixed this on the wifi since many people bring three or four devices now, but the hard wired connections are still poorly utilized.