ISPs are dying to find some sort of "value add" to get more revenue, but it's just not there. ISPs need to realize that they are strictly a utility now. All the value that they could possibly add can be obtained from any other company now. They used to provide TV service, but nobody wants linear TV and can just send their money directly to the company that produces the TV content and have it delivered over IP. They used to provide phone service, but nobody wants phone service; they have a cell phone and can get a fake landline from any number of companies for almost no money. They used to provide a DNS resolver, but they used the DNS resolver to inject fake addresses, so the power has been moved away from them. They used to provide Usenet, but their "content partners" didn't like that, so now you get Usenet from a third-party provider.
IP is just too versatile. All customers want from their ISP is the dumbest possible pipe, because they can get all other services over that pipe and pick the best deals. If you aren't interested in infrastructure, then you shouldn't be an ISP.
(That said, I don't see why companies like Comcast aren't pursuing things like becoming Cloudflare or AWS. "Click here to host your docker container 1ms away from 87% of the US population," seems like a service that could make a lot of money.)
ISPs providing DNS and caching was a key part of the Pai FCC's reclassification of ISPs as an "information service" instead of a "communication service", as part of their repeal of net neutrality.
The FCC has a lot of leeway in deciding whether something is an "information service" or a "communication service", but they do have to be able to justify the decision. Pai went with the ISP's providing DNS and caching.
I wonder, if most people end up using DNS from third parties, and caching from third party CDNs, it will make it easier for a future FCC to reclassify ISPs as communication services, and put back net neutrality?
PS: Pai's approach partially backfired. He also tried to use the FCCs power to preempt state regulation to prohibit states from imposing net neutrality. But reclassification of ISPs as information services meant that the FCC no longer had the power to regulate net neutrality (indeed, that was the point of Pai reclassifing them), and the FCC's power to preempt state regulation only applies to things that the FCC could regulate.
"ISPs providing DNS and caching was a key part of the Pai FCC's reclassification of ISPs as an "information service"..."
I thought the same thing when I read about that decision. Many try to label DNS as "intrastructure", with the implication that some third party and not the user should have control over it. As an ordinary user, I do not use third party DNS. I will never believe that from a technical standpoint all ordinary users must use third party DNS.
I had an ISP that intercepted my requests, and would return bogus (ads) results if the resolution for that particular address failed. Moving off of port 53 was enough to bypass it, but it was easier to just tunnel my DNS requests. The sad thing now is google thinks I need extra captchas because of my weird DNS geo-location. I can't win.
> ISPs need to realize that they are strictly a utility now. They used to provide TV service, but nobody wants linear TV and can just send their money directly to the company that produces the TV content and have it delivered over IP.
Right, I think Comcast realized that. That's why they _are_ the company the creates the TV content. They own Universal Studios, NBC, MSNBC, CNBC, USA Network, Syfy, Dreamworks, etc.
So it's even stupider that they are scrambling in the ISP business. Relax, bean counters, the rest of your mega-holy-shit conglomerate is doing just fine.
> ISPs are dying to find some sort of "value add" to get more revenue, but it's just not there.
Yes it is. It's Google Fi. Comcast could do Google Fi better than Google does, because they could make every Comcast modem an access point for it.
Then for a few bucks on top of your cable internet subscription you give people unlimited mobile data on unlimited devices whenever they're connected to a Comcast access point, which is 97% of the time because they're everywhere, and metered data using some deal with Sprint/T-Mobile/whomever the other 3% of the time.
How much money is on the table when you allow families to cancel their wireless plans on four, five, six different mobile devices yet still have service everywhere?
My ISP has done something similar for years. They give you a router which doubles as a "wifi hotspot", where anyone with a valid ISP login can use that network at no cost beyond their regular internet bill. It just becomes a part of what you get with their service.
This doesn't quite work out as well as Google Fi because they don't have the other half of the equation which is to jump to a mobile provider when there's no valid wifi connection. My area has 2 choices for broadband and even with a majority of customers in this area there's still large gaps to the point where if I walk around the neighborhood there's tons upon tons of dead zones.
> This doesn't quite work out as well as Google Fi because they don't have the other half of the equation which is to jump to a mobile provider when there's no valid wifi connection.
That's the key though. Even if you have 97% coverage without it, if that other 3% could be when your car breaks down on the side of the road, people aren't going to be willing to switch off their existing wireless service which does work there.
But add existing wireless carriers as metered data which costs nothing unless you use it, and you satisfy that blocking factor and get the customer.
> much money is on the table when you allow families to cancel their wireless plans
and how much money do they stand to lose on those plans that got canceled? Making a service more efficient means they get less money, and without the threat of competition, a business won't do this.
Xfinity Mobile is a Verizon Wireless MVNO and already offers international roaming (at high but not Verizon-levels-of-high) rates. If Comcast chooses to go after the roaming market in a way that Google Fi has, it could.
Unless something drastic happens ISPs are going to win. They're increasingly charging per bit for wire data (where there is no inherent scarcity), lobbying to prevent competitors, and have taken public funds and failed to reinvest in their infrastructure.
Lobbying and changing laws to apply reality distortion fields on all their customers and the competition is cheaper than investing into R&D or infrastructure and provide a better service or innovate and compete.
If it’s true for ISPs, it’s true for other big players and is a symptom of a failing system.
What do you mean there is no inherent scarcity? There is only a certain amount of capacity in any infrastructure. When you pay for a bit on a network with metered bandwidth, you are paying for the fact that you used up that infrastructure for a short amount of time. It’s exactly the same concept as paying more to deliver a larger package.
On a network with scarce capacity, flat rate billing (vs metering or caps) means that light users are subsidizing heavy users. A real world illustration of this fact is that ISPs actually prefer flat rate because it is easier to market. On networks with lots of capacity they sell “unlimited”, while on tighter networks (like the cellular networks), they almost always use metered billing.
I’m not sure why some people (mostly only on Reddit or HN) are confused about this, but one theory I have is that they don’t understand the oversale of bandwidth. If you want to get a dedicated 1gbps per second of internet bandwidth, that will cost over $2000/month in most areas. So what all ISPs do is to sell that same gigabit to many subscribers. You can get hundreds of subscribers on 1gbps.
But they advertise “up to 1gbps speeds” instead of “1gbps with a 100x oversale ratio” because that would confuse people. So maybe people think that means they are actually supposed to be getting a dedicated 1gbps. If that was the case, then it wouldn’t make sense to bill for data. But most people don’t want to pay $2000/month for their internet.
The real issue that you are complaining about is that ISPs in many areas are monopolies and feel no need to upgrade infrastructure.
To start, realize stating that a dedicated 1gbps link is $2000 may be true but it does not justify the price, which is what I'm speaking to.
What observations I had were: 1) the price of dedicated links went up in time disproportionate to their capacity; 2) network operators began moving to a per-bit charging model, forcing users to bid against each other, and; 3) at least here, non-DC dedicated links can be had much more cheaply than a link in a DC.
There is scarcity but for practical purposes the scarcity is so far off that I'm not sure it makes sense to consider it. The true scarcity is related to the availability of land and the size of the data links, neither of which are yet very limiting. Regulation related to right of way (or more commonly) city involvement in high level planning could be blamed for creating market failures if these costs really are that high.
Otherwise, in regards to #1, we would expect a downward trend of dedicated network link price while capacity goes up, following storage density and transistor density (or more appropriately computational throughput). But we don't see this, even though throughput does increase, and integration costs decrease.
For #2, going to a bidding model could imply scarcity, but as there are unjustified monopolies often there is no increase in capacity so it seems prudent to assume this is done solely to increase price.
And with #3, well, certain usages are priced higher than others, also implying some kind of bidding. This might reflect space scarcity but seeing as the price tends to not go down even if the customer provides their own network equipment and pays for its space it just seems like a reluctance of the DC operator to perform upgrades.
The fairest model I can imagine has users paying the amortized cost of the network equipment and its installation and maintenance.
Yeah, there are always going to be pricing shenanigans because people are not willing to pay the actual costs.
The actual costs of an ISP are the $1MM one time cost to dig up the streets in your neighborhood, and an ongoing cost of electricity for their datacenter, Juniper licenses, peering, customer support, etc. The problems the ISPs have is that no customer is going to pay that one time cost, so they have to figure out some way to make you pay it without that $1MM bill the first month.
I have worked for two ISPs and the internal debate is always around figuring out how to do this. At the last place I worked, our install costs were relatively low; the city already built tunnels under the streets for fiber, so it is financially viable for someone to just pay the ISP to run the fiber, and then pay the incremental costs. It is certainly not free (everyone wants their cut, including the building that you're already paying rent to), but is the kind of money that a medium-sized business or a condo co-op could find if they were so inclined. Individuals have no chance of finding the money, though, so something has to give.
(Where I'm going with this, is that paying per bit does seem reasonable to me. It's what AWS does; if you want to lower your bill, send fewer bits. But it's still a bit of a hack, not reflective of the actual costs. The first bit you send over your line costs a million dollars, but the rest are practically free. You're just paying to keep some routers powered on, and for them to be upgraded when other customers want to send more bits.)
Being able to sell your browsing history is one way to cover the infrastructure costs, but it's unfortunately not a workable idea because as we've seen, companies like Google and Cloudflare can just obscure that data from the ISP.
I hate to be all pro-government, but can you imagine how inefficient it would be if UPS, Fedex, and the USPS all had to build nationwide road networks in order to be able to deliver packages to your house? Instead, the government built the roads, and it made the economy stronger. Being able to drive to work or have someone send you a package from across the country is quite valuable. I don't see any reason why the last-mile Internet shouldn't be built that way; the city can get traffic to a peering point, and from there, you can pick what ISP you want to connect you to the actual Internet. But unfortunately, we half-assed that, instead paying private companies to built the last mile, and now they want to milk their nearly-free infrastructure. The idea of Internet to the home is a mature one, and the reality of a mature business is that it becomes a commodity, and that is going to drive profits down. Comcast, Spectrum, etc. should all be finding new ways to make money, and the easy one, "spy on our customers", is not viable. Sorry, investors. Too bad for you.
Isn't that $1MM upfront cost one of the major reasons why bonds exist? You get your $1MM bond now, and you repay with the future revenues you get from your customers. You make the calculation and that debt payment, plus ongoing costs, plus profit is what you charge your customers. Eventually you 'pay off the mortgage' and then revenue really goes up, but with the rate of technology upgrades, you will always be 'paying your mortgage'.
You still eat the risk, though. Maybe 5G is a thing and all your customers cancel next year. Now you have a 30 year payment on something that generates no revenue. That's where the fear is.
Isn't that business in a nutshell? You wouldn't have a much of a business if everyone went to 5G bond or not. Competition and doing risky things is part of business and partly why bankruptcy law exists.
If your business implodes due to 5G, the bond is written off as part of bankruptcy and the lender eats the loss while taking over the business assets.
Also seeing how low range 5G is, it basically looks like cell phone providers become wifi access point providers, which smells really close to what a wired ISP does anyway, which sounds like another ISP competitor.
My thought was that the AT&Ts of the world could easily build their fiber networks for 5G backhaul. My former employer's thought was that the AT&Ts of the world would pay them to do that, though. I don't know who's right.
The biggest problem I've seen at ISPs is the general unreliability of all the ISP-grade networking equipment. The routers are bad, the OLTs are bad, the VoIP phones are bad, the CPE is bad, the WiFi routers are bad, and the WiFi clients are bad. If I were an ISP, I'd be working on fixing all of that; everything that interferes with your customer wanting to send a packet and the host on the Internet receiving that... that is the area to optimize. But I don't think the funding exists. (It did when I was at Google Fiber, I worked on CPE. But Google lost interest, and at the smaller ISPs, it's all about buying as much crap as you can get off the shelf and letting your customers integrate it.)
> The biggest problem I've seen at ISPs is the general unreliability of all the ISP-grade networking equipment.
I wouldn't generalize so much. It only seems that way, and only in countries where ISP markets are monopolized, because nobody cares about doing it well. Where there is enough competition things are completely different: ISPs are willing to test different GPON vendors, OLT/ONU, wifi routers, etc., figure out which features and configurations are reliable, which aren't, fix things that work poorly and so on, because quality becomes the reason people switch ISPs. Competitive industry also means lots of people with experience and expertise and lots of shared knowledge.
It doesn't need to be monopolized, oligopolized works too. And it's not impossible for a monopoly to care and focus on these things, it's just highly unlikely.
For about $80 million my city of 2 million people got fiber on every main street, not digging open holes but drilling 6 inch tubes with lots of dark fiber for rent too. We have Gigabit Internet for $10/month in 80% of the city. Europe, no ISP lobby.
>I don't see any reason why the last-mile Internet shouldn't be built that way; the city can get traffic to a peering point, and from there, you can pick what ISP you want to connect you to the actual Internet.
Yet not everyone can own a Taxi. Which in an economics way make sense, it also makes sense to avoid avoid too much traffic in the street.
The equivalent on the internet would be to block certain usage, either for economics reason (well we certainly can't have a million content producer, they need to live) or for traffic reasons (Netflix is using too much traffic... we maybe should limit the number of slot that can be used for medias streaming).
I'm all in for a centralized fiber networks, but its goals should be clear and it has to be transparent over what it does to achieve it. It also needs to be outside of the government control the most possible, because Snowden has proven that governments entities can't be trusted with such important data. Having it at the municipal level with some regular audits could be good enough.
Well, there definitely is bandwidth scarcity. Also, unmetered bandwidth is just one major reason botnets are so trivial. Nobody knows or even has incentive to know that their smart toaster is involved in DDoS attacks so now we're all centralizing behind Cloudflare to mitigate this reality.
Now, I'd like to see a fair price per bit, but I'm not convinced the world is better without it.
ISPs are increasingly charging for total throughput on top of point-in-time bandwidth, though. For example, to get actually-unlimited throughput with Comcast you have to pay an additional $50/month: https://www.xfinity.com/support/articles/exp-unlimited-limit
Most broadband access providers don't know how to build, operate or scale out compute infrastructure and the systems around it. They've tried, they've failed, they've blown a lot of money in the process. It's not their core competency and most have given up. They've been taken in by vendors (or acquisitions) in the last 10 years selling them solutions "just like the cloud companies" but can't manage to make it work in practice and entice the select set of customers who could take advantage of being in close proximity to users and less network congestion. The conversations I've heard go on with these companies display how they're a cargo cult and they fetishize buying the same hardware that cloud providers use.
> ISPs are dying to find some sort of "value add" to get more revenue, but it's just not there.
ISPs are some of the most profitable businesses on the planet. If you look at any of the fundamentals for financials of AT&T, Verizon, Comcast, etc their gross profit margins are 60%
Some are. Verizon bought EdgeCast CDN which has a Cloudflare style WAF. They now operate under the Verizon Digital Media Services monicker, but it’s been a bit of a challenge for them to integrate.
And that's backfired for Verizon a little bit. Think about it this way. Verizon is now a CDN: they have to get as much peering as they want. Verizon (as701) the big eyeball network has no plans to peer openly. It complicates peering negotiations because edgecast (separate asn) can't give up the access that everyone wants to Verizon. Similar happened when tata (old teleglobe) bought bitgravity cdn and constrained their peering relationships.
Yeah, I believe netflix puts boxes in a huge number of IXPs for just this reason. That said, it's not currently practical without netflix's current scale. The "as-a-service" part provided by an ISP is an actual value-add and could indeed make money.
IP is just too versatile. All customers want from their ISP is the dumbest possible pipe, because they can get all other services over that pipe and pick the best deals. If you aren't interested in infrastructure, then you shouldn't be an ISP.
(That said, I don't see why companies like Comcast aren't pursuing things like becoming Cloudflare or AWS. "Click here to host your docker container 1ms away from 87% of the US population," seems like a service that could make a lot of money.)