Hacker News new | past | comments | ask | show | jobs | submit login
Starlink adds another 100Gbps connection to the SeattleIX (seattleix.net)
247 points by walrus01 on Feb 3, 2022 | hide | past | favorite | 185 comments



I'm very glad to see the SIX getting some attention on HN. I have a lot of respect for how that IXP is operated -- Chris Caputo runs a tight ship. Their one-time fee structure is very friendly to small participants and very unique for an IXP of that size -- that alone should be lauded. SIX governance is very open and transparent. I can't recall the last time they incurred a major unscheduled outage. I wish more large IXPs were run like the SIX.


You made me curious to look up how SIX survives then. Apparently they entirely depend on donations, many of which are not monetary, but essential infrastructure (racks, power, fiber connections etc)


This model works well till some big companies get involved and the bean counters of said companies say "wait, we're spending how many millions of dollars on a donation of security guards and servers to a non-profit? Can't we just donate a bit less and other people will have to cover it?"


> This model works well till some big companies get involved and the bean counters of said companies say

Combined with relatively low one-time port fees, the SIX's funding model has worked for 25 years - including dozens of large companies and bean counters: https://www.seattleix.net/contributors


It probably helps to have two of the largest tech companies next door.


Aside, but super surreal...

I went to that page, see the top item is Starlink, the next link is "Welcome Danga Interactive AS32150". That's kind of a small AS compared to Starlink's 146K, but then I realize that used to be my AS, tummy.com.



tummy.com just shut down a few months ago and sold our old IPv4 swamp addresses and the ASN to him.


How are these transactions conducted? If I wanted to buy some IPv4 and an ASN, where would I go and how much would, say, a /28 cost me?


For a /28 you'd have to go to someone that has at least a /24. Probably Colo a bare metal box. If you want to buy a /24 you could go to arin, assuming you're in north America but even that is quite unlikely with the amount of up space left. You'd probably have to buy third party from a broker. Last I checked that was in the 4$ per IP but I haven't been following for a few years.

At this point, most of the free up space is picked up by the big boys and you gotta do business with them.

Airn has a few senarios where you can pick up ipv4 addresses but it's pretty though at this point and you usually would need to be an established business with an existing ipv4 and a dual stack ipv6 setup. The ipv4 they are giving out is basically preferred to providers that are dual stacking and need additional ipv4 space to roll out ipv6.

Honestly, if you were to start today, there's basically no chance unless you gett off your wallet and pay a 3rd party at auction. And those ups are usually going to be trash. You'll spend a year getting them off blacklists and able to function.

So theoretically, you could get some but in practice, you're best bet is to rent them from Someone that already has them and put a router or server in their Colo.


The prices have risen a lot since you last checked. It's getting close to $40 per IP these days.


I wonder how do major cloud providers reflect this appreciation of their assets on their books,:)


That is a very good question and not only for cloud providers but also for businesses with /8 ranges and even smaller ones which currently are valued at $1+ million I'm not aware if there is any business that really registers them as assets or any tax authority requiring this but I'd also be very interested to hear more on this


Lol, sounds about right. We've know that hockey stick graph is coming. Probably should have checked before I shot my mouth off.


it's way above $40 now, more like 50-55 and pushing towards 60


This. Bad for newer companies since big guys already have it and everybody still needs v4.


I can't speak to how all transactions are conducted, but there are auctions on one marketplace, IPv4 Global auctions, with cost per IP in the $50-60 range https://auctions.ipv4.global/. I also really enjoyed Flyio's blog post about acquiring addresses https://fly.io/blog/32-bit-real-estate/


Too bad there is no IPV4 ETF, would be a good investment even now.


You can't buy smaller than a single /24 through the officially approved arin, ripe and apnic transfer processes, since that is the minimum size of individual netblock you can announce into global routing tables.

An asn is not something you buy from a third party unless acquiring a whole corporation that already has one, since there is no real shortage of them, you'd get your asn from your regional registry.

Expect about $50 to 55 per IP now.


> Expect about $50 to 55 per IP now.

So 254 * 50 or about USD 12k to 13k? That’s the only cost and you get to own 254 IP addresses? Are there ongoing costs?



I believe the first and most crucial step was Brad asking his wife if she was ok with it.


Should've sold it to Starlink, maybe if they had some more addresses they wouldn't be using CGNAT.


Wow, that's very cool!


I can't see Brad's face anymore and not have nightmares filled with

CAN'T BE HUSHED HERE

CAN'T BE HUSHED HERE

CAN'T BE HUSHED HERE CAN'T BE HUSHED HERE



In other words, starlink added 5 guy-at-large’s worth of ports.


He's not just a random guy. Back in the day, he used to run LiveJournal, for instance.


Ah yes, good ol Brads Markup Language mmm fun times :)


wow!


Starlink is such a scam all my relatives signed up for the $600 beta and now more than a year later still no way to use the service. All the while they keep promising it's only 1-2 months out.


Admittedly, frontloading the deposit to determine market interest and where to scale with actual dollars is a bit scummy (see: Cybertruck/Model S/3/X/Y/everything Tesla does deposits) but the service IS live in a lot of markets and DOES have real, actual users. Calling it a "scam" implies that those things aren't true.

But I agree that Elon time is absurd and irritating and sure feels like a scam. (I would argue FSD being sold for SIX GODDAMN YEARS on Teslas is _absolutely_ a scam.)


Though, $500 deposit on a $50,000 car is different from a $500 deposit which is about the cost for the whole thing.


They can just request a refund.

https://www.starlink.com/legal/terms-of-service-preorder#:~:....

Or are they happy to save their place in line?


But it's only a month away!


Received mine this week for a service address on the Navajo Nation. I imagine the wait is shorter for less populated rural areas, which should correlate with need. My old isp charged $60 a month for 1.5 Mb/s down and 750kb/s up. This should be a great improvement.


My relatives are in a similar situation -- only satellite available in their area, so ~800ms latency and speeds around 1 MBps, but half a mile away there is literally fiber available, so it's going to be a long time before it gets opened up for them because they aren't considered to be in an underserved area.


I'd be curious to know how stable is the connection? notice any drop while on a video call?


I've been on it since February of last year. My day job uses a remote desktop and I'm on video conference calls 4+hrs a day. Starlink was unusable until late summer because I would see 50+ outages of 3 seconds or more during the day. (3 second blips are where it seems to really interrupt zoom and vdi)

It really started coming together in the fall and I would say since November it has been very usable. I would say it's easily as stable if not more stable than the 10/.7M dsl it replaces.


This is exactly my experience. Also got it last January or February. I had to keep my DSL line for a few months, but it got good enough for me to finally drop that thing.


Thanks!


I have starlink in the northern latitudes.

The issues I have with connectivity are probably 99% trees.

When I first started it seemed to need a total view of the sky to get consistent. As the software/flotilla have improved vastly, I use it for zoom and once every couple hours of zooming I may risk a 10 second blip.

I think they are getting better software wise of detecting the usable area of visible sky and utilizing satellites better.

I would think more satellites, more tree tolerance. I currently have my dishy too close to the road since I get the best view that way, I might try moving it back to the backyard this spring to see if it can work well there without worrying about teenagers using it for batting practice or some disgruntled cable employee or it getting stolen.

If you have an unobstructed view of the sky, it should be functionally as reliable as cable internet.


My company is currently using a Starlink as the primary internet connection (with vdsl backup) for a new office we’re setting up in australia. It works great, consistent 300mbps with peaks up to 500. Anecdotally, the system is working great for us and is actually more reliable than the vdsl connection we use as the failover.


I'm guessing you have no trees around?


The deposit is 99$, not 600. And it is refundable online at any time.


They paid $600 so not sure when or why that changed.


That's not possible, it was always 99.

You can only pay the 500 for the dish once it's available for shipping.


Yes, such a scam. 200K happy customers and refundable deposit. But it's Musks' so it must be a scam.


Only 200k happy customers when it was promised to be "fully live, nationwide, for all customers" next month more than a year ago with repeated similar promises this fall.

And my relatives are not wealthy people -- this was half of their income for a particular month so at this point they'll be damned if they lose their place in line. They have no other choice -- not even DSL options in their area and the other satellite services have been so expensive and unreliable. They are completely at his mercy, and I feel like it's my fault for suggesting it.


>200K happy customers

You're defining "happy" as "not quite pissed off enough to leave". That's exactly the kind of game which (when compounded) makes fed-up people call things scams.


> You're defining "happy" as "not quite pissed off enough to leave"

No I'm not, I literally mean happy. Starlink is by far the best ROI internet option for many.


I assume they run the gamut from maximally happy, to whatever their threshold for leaving is. Which is why I used a tautological descriptor.

You're taking a plausibly-most-likely scenario and using it to cast an assumption about the emotional state of hundreds of thousands of people. Don't you think you might be talking out of your ass?


No, I personally verified with each and every one of them that their emotional state is happy and that they're not running any gamut.


"happy" would be "yes, I'd use this in favor of fiber". No satellite customer is truly happy, no matter how good the product is, unless they are on a boat or truck.


I know people who have had the service for months, outside the US, and are happy with it. How can it be a scam? Why didn't they ask for a refund? Was the refund denied or slow to arrive?


I suspect the original comment is referring to Starlink taking a (~ 15% deposit) and 'promising' 1-2 months wait time which isn't reducing so they end up still waiting 6 months later, and the expected delivery is still reported as 1-2 months in the Starlink account.

It's possible that the delay is due to lack of legal clearance in the customer country. Starlink needs to get approval to install and operate the ground station satellite links in each country. From the few authoritative statements I've seen those are sometimes delayed due to local country bureaucracy.

I ordered Starlink in the UK in February 2020 and the equipment finally arrived in May 2020. I /think/ the stated delivery estimate at the time was 2-3 months and never reduced until the order confirmation email arrived and I paid the balance, after which the equipment arrived a couple of weeks later.


They're selling pre-orders in several countries where they almost certainly won't launch at all, or won't launch for a long time, due to huge regulatory hurdles. It does seem a bit like the same scummy behaviour as airlines still selling tickets for flights that definitely will not operate, thus getting cashflow for the period between sale and refund (which can be many months), as well as real profit from the people who don't bother to request a refund.

For example, Starlink claimed in early 2021 to be launching in Viet Nam "within a year", and have been selling pre-orders, but they can't legally operate in VN unless they form a JV with a local partner, since foreign companies aren't allowed to provide telecoms services there. The relevant government agency went on record saying that Starlink hasn't even applied for permission, yet they're still selling pre-orders.


I agree about the needless misleading of expectations. Some, but not all, is - I suspect - well-meaning public sharing of ambitions based on most favourable ideal timelines - something very common across most Musk-led businesses. I usually read that stuff as a partial sharing of stream-of-consciousness ideas and targets before there's a fully formed plan of execution - where reality often imposes impediments!

In your Viet Nam scenario and similar cases I wonder if it is (partially) a result of the Starlink plan to use direct satellite to satellite LASER links to relay data in low earth orbit to a more remote existing ground station?

As in, if I recall correctly, SpaceX originally expected to be launching 'block 2' satellites with LASER links in mid 2021 but suffered internal hidden delays (that led to Musk's internal SpaceX existential warning in October 2021 and firing of some senior staff) so the first batch were only recently launched in early 2022.

Those LASER links being in place are likely a prerequisite for serving territories where SpaceX cannot obtain favourable permissions for ground stations.


I don't think SpaceX will actually do this. They'd be explicitly breaking the law in those countries by providing services to people there (the mere fact that there is a technical means to do so without installing a local ground station doesn't change that). In several countries (I think including Viet Nam) it's illegal to even own an unlicensed satellite terminal, so the users would be at risk too.


If you order something online with estimated delivery "in a few weeks" and a year later it still isn't here, it's a scam, regardless of how easy it is to get refunded.


You can see a lot of Starlink’s peering at https://www.peeringdb.com/net/18747


I barely understand what this is. Wondering would you find these through traceroute or something? huh

Or people populate them at will


peeringdb runs on trust, any ISP could put in non factual entries, but it would be useless to anyone.

it's meant to be used by network admins at other ASNs that want to set up peering sessions across IXes and PNIs.


I'm not sure what they are actually using this for.

All Starlink customer traffic egresses via Google's network, and all their ground stations are along Google fiber paths.


> all their ground stations are along Google fiber paths.

actually most of their earth stations are colocated near long haul dwdm regen huts from older, more traditional big telco players like what is now lumen (former qwest/centurylink), level3, zayo, etc. The fact that they can buy transport circuits from those huts to the nearest major cities to meet google is useful.

by no means does google really own/run/control most of the long haul dark fiber and dwdm networks in north america. or even a good portion of it. the predecessor entities that have now been rolled up into lumen or zayo do.


I meant Google fiber paths in the logical sense. Most people don't care to understand the distinctions in how bits on glass are purchased.


In most of those places it's not even a Google fiber path in the logical sense, where Google doesn't have a DWDM chassis at the Regen hut, site or an iru in any dark strands there.

Starlink is buying transport circuits from the traditional telcos who operate on the route to get to a major city with a Google POP.

Like, if you go out to the Prosser Regen hut site and starlink earth station, that's all legacy level3, now lumen.


Perhaps to have a decent negotiating position next time their contract with Google comes up for renewal?

You get much better prices if Google can see you already have most of your own infrastructure set up and ready to migrate to with not much effort.


> All Starlink customer traffic egresses via Google's network

That’s not true. Customers that were getting public IPs without CGNAT were coming from a non-Google ASN.


That doesn't mean anything, Google will announce your routes for you if you want. [1]

[1] https://cloud.google.com/network-connectivity/docs/router/ho...


The google asn would still show up in the path if that were the case.


> That’s not true.

My two Starlink terminals respectfully disagree. It has been mentioned numerous times on the Starlink forums. It's IP space previously used by Google Station.


both are happening simultaneously in different areas, there are still many starlink terminals behind cgnat inside the google ASN, there are now some starlink terminals that have been giving out dhcp leases of public addresses owned/controlled by the starlink ASN, and there are also starlink terminals that have been seen doing cgnat and with an exit point in the starlink ASN.

it really depends on where you are and whether your terminal has been selected by (some unknown criteria) for spacex's network topology changes.


Do you have public IP addresses assigned directly to your router? This was showing up on the people who got public IPs.


Not anymore, customers are being migrated to AS14593. I'm one of em.. I'm in 170.203.91.0/24.


Amazing that this is run by a non-profit. Wish more of the internet was like this!


The internet is a lot more like this. The web isn't.


It is the norm in Europe for IXPs to be run by non-profits. See this for more info http://drpeering.net/HTML_IPP/chapters/ch12-9-US-vs-European...


It's the norm in the US too.


as somebody who is a DE-CIX "customer" i find that really funny. maybe for small IXes in europe, but big players like LINX and DE-CIX are doing their thing too.


I may have missed your meaning, but to be clear, LINX is not for profit and DE-CIX is wholly owned by a not for profit association.


It sure doesn't behave like a non profit.


Huh, Netnod in Stockholm is also owned by a non-profit, I had no idea. But its still very expensive to connect to afaik.


A lot of the internet is things like this


For anyone who wants a bit of history, Jared Reimer did a presentation for the SIX's 20th anniversary. This was at NANOG in 2017:

https://www.youtube.com/watch?v=N2ChhZcH2ok (slides: https://archive.nanog.org/sites/default/files/4_Reimer_Seatt...)


You can see a San Francisco IX's members here, not as gigantic as this one, but you have Indies like monkeybrains right next to Netflix and Google https://www.sfmix.org/participants


I hope they ship soon. I preordered Starlink and put down the deposit in Feb 2021. Still nothing.


Availability is presently limited by density of CPEs in a given geographical area. They don't want to ridiculously oversubscribe it and give people bad ping, packet loss and traffic speeds, because that's exactly what geostationary consumer priced vsat services as known for. If you can't get it shipped to you and are still in the pre-order queue, your area is out of capacity until more satellites are launched or the satellite+terminal firmwares are better improved.

There's a number of places that are not out of capacity and if you put in the "right" street address it'll take you to the immediate full price payment order page.


But Starlink doesn't support changing your location yet - or does it?


It always has, or has for a very long time. It's not guaranteed that when you change your location that there will be service available at that location however.


It does, but you have to apply and ask if there's capacity in the area you want to move it to in advance.


they are going to oversubscribe it. they're only slow to add users because they can't make terminals, not because they're trying to maintain an unrealistic QoS.


This is certainly not true. You can see tons of people in different countries who literally ordered starlink a week ago getting it ahead of lots of people who preordered like a year ago because the ones who are getting quickly are likely in under utilized cells along the path of existing trajectories. It is highly likely that those who have been waiting (with someone they know nearby having starlink) is because that particular cell is over subscribed and they don't want more connections in there until they have more satellites up.



That can't really be true. I ordered it in November and it shipped the same day. I was just in a location that had capacity.


Is that an assumption on your part, or is there associated evidence? If there is, that’s something I’d really appreciate knowing.


Every residential last mile isp in the history of commerical ISPs since 1993 has been oversubscribed, the question is at which ratio, and how that technology is implemented.

the problem with terrible oversubscription on geostationary consumer grade vsat services is based on the fact that raw transponder kHz on a geostationary satellite is extremely expensive.

starlink appear to be taking a very different perspective since they absolutely know about the abysmal reputation of things like viasat/hughesnet/wildblue/etc in the consumer-facing ISP space, and won't ever let it go that way. because it would be a brand name killer.


I see your other comments and you seem very sharp, but I don't understand why you think starlink will be any different, and especially why the initial launch period is anything like a mature service. for instance, do you disagree with anything here? https://twitter.com/LionnetPierre/status/1482863459834511363...


I worked in network engineering for geostationary two way satellite for years and have been a starlink beta customer for over a year now. I also have some inside info from network engineering folks in WA state that I can't disclose here.

If one fully understands the economics of geostationary it helps to better understand the tech behind starlink. I don't rule out the possibility that it goes bankrupt and ends up in a situation like the 2nd corporate incarnation of iridium, but the tech works great.


I also have more than a decade of experience with satellite internet, and geostationary satellites in particular. Nobody has ever said starlink will be technically impossible; the arguments from the start have been that it is not financially solvent. Temporary VC money obviously doesn't tell us whether they're going to be profitable, but I haven't seen any real analysis that shows they have anywhere near the number of subs they need to become profitable. And increasing the subs will surely decrease the quality of the service, because that's always what these satellite companies do.

The fact that they just released a more expensive terminal to me means that they are trying to tackle the enterprise market where the cost per bit is significantly higher than with consumers, and those users will for sure get priority over consumers. It also means they are losing money too fast and they needed to at least have a solution that will potentially have higher returns in the short-term.

I agree the tech works great for what it is, but I also don't think it's been out anywhere near long enough for people to make judgements about how well it works. I don't think we will have enough longevity data on the terminal for at least a couple more years, so it's not clear if performance will degrade or if the motor will seize due to whether.

I think the users that got in now are having a great honeymoon period, but I don't think this will last, and we should not be comparing it to existing, mature services.


I agree it's a risk inherent to the ISP business model generally. I was asking if there was any evidence that was specific to Starlink "trying to maintain an unrealistic QoS".



Weird. I put my deposit down in April 2021 and got my offer in December 2021, but I ended up declining it - no way to use it right now.

Maybe it’s related to your location. I’m in Vancouver.


I've heard a lot of people say they're still waiting.

Musk seems to have figured out that he can collect deposits for way more people than he could possibly deliver to, leveraging it as free operating cash. The cybertruck is a great example - millions of people gave him money for something that hasn't left prototype stage and will never be produced because it couldn't possibly meet federal vehicle standards in the US or Europe.


Deposits are refundable at any time. If people are ok with leaving their money in Tesla's hand for years, then they are clearly indicating that they believe Tesla will deliver. If any evidence comes to light that shows otherwise, they can take their money back in a blink.


I dunno. I've got it, it works fine. I really doubt the constraining factors are with network load or regulation, though; if you can source the components for a couple hundred thousand routers and radar dishes, I'm sure the rest of the industry is dying to hear where you're looking.


Cyber Truck is a style I don't think that will be an issue with meeting Federal Standards and EU Standards


The wait list is maybe a million people? Enough deposits to pay for 1% of the satellite constellation is worth something but I doubt it's a notable factor in how they operate.


I’m just thinking the bonnet is going to be plastic and that’ll clear about half or so of the concern, isn’t that the case?


Same with SolarCity, too.


Musk is likable but im personally in no hurry to buy anything any of his especiialy his hyped up robot cars .. here's one almost killing a pedestrian (https://www.youtube.com/watch?v=Lnp0VChf1_k) ... execute one algorithm to avoid car yet oops we forgot to program it executing additional ones in this situation .. let's ship out an update. Progress is going to be a killer!

Starlink seems to be more hype too like his HyperLoop.


That's got nothing to do with "robot cars". That's just a bad driver. Why are you trying to spread FUD?

Even the video description is incorrect.

> Note the driver blows the horn as he enters the intersection. He obviously cannot stop the car. Probable FSD or Autopilot crash.

FSD/Autopilot does not stay on or prevent human interaction. Slamming on the brake turns it off, even assuming was on. This is a "unintended press of the accelerator pedal" type crash. Human error, and actually a good argument for why removing the human from the loop would be a good idea.

> Starlink seems to be more hype too like his HyperLoop.

Starlink is serving over 100,000 customers with good service all paying $100/month. Hardly hype.


> FSD/Autopilot does not stay on or prevent human interaction.

I'm not aware of Tesla specifics, but i've previously read several stories on HN of car computers who wouldn't let the driver be in control. As a computer person who has moderate understanding of mechanics/electronics, i believe unless you have a physical kill switch to take control back from the computer, there is literally no way software can be safe enough that even in a crash you can regain control.

I mean apparently you've been around for a while on these forums. You're surely educated that software can never be trusted, especially with life-endangering situations.


> I'm not aware of Tesla specifics, but i've previously read several stories on HN of car computers who wouldn't let the driver be in control.

That can't happen. People misremember things when they're in a panicked state. Or they're exaggerating to defend themselves. The instant you even slightly tug on the wheel autopilot and autosteer of any kind will disengage and revert to a traffic aware cruise control state (with an audible chime). And any application of the brake disengages autopilot/autosteer and also automatic cruise control with an audible chime.

> I mean apparently you've been around for a while on these forums. You're surely educated that software can never be trusted, especially with life-endangering situations.

I'm not sure what you're saying. People aren't "trusting" it yet which is why it's easy to disengage and override.


> That can't happen.

Yes yes i've heard that one before. I've had many bugs "that can't happen" over the course of my lifetime. Fortunately none of them involved a car so i'm still around to talk about it. A few years ago, somebody would have laughed at the idea of Boeing producing fallible equipment, especially since we as a species have decades of experience of air travel. Now would you still say it's unthinkable that Boeing could produce faulty hardware/software?

A link that gets posted on HN quite often contains many such bugs "can't happen" bugs, that aren't even caused by AI : https://beza1e1.tuxen.de/lore/index.html

I also remember some threads about high-range medical equipment (in actual hospitals) killing people due to software bugs.

> And any application of the brake disengages autopilot/autosteer and also automatic cruise control with an audible chime.

Is that a physical kill switch that triggers a sensor to produce an alert/chime? Or does the sensor ask politely the computer who actually controls the car to disable the autopilot? In the latter case, i know of no way to make this safe: for example what happens if the sensors requesting human control back die, short-circuit, or otherwise malfunctions? Or if the program enters certain kinds of memory violations? Or any other software/hardware fault?

> I'm not sure what you're saying.

I'm saying i've seen enough bad engineering (not always, but sometimes along with bad faith) across all fields i've been even remotely involved in. And i'm saying i don't even remotely trust the >100 micro-controllers you find in a modern car whose schematics and source-code we can't inspect. I mean i don't fully trust mechanical hardware either, but at least with it symptoms and failure modes can be easily reproduced and debugged.

And i'm definitely saying i would never ever trust a machine-learning algorithm with life-making decisions. More on this topic:

- James Mickens at Usenix on why AI is usually a terrible idea: https://www.youtube.com/watch?v=ajGX7odA87k

- jwz's blog (another HN favorite) is also full of examples of AI failures ; a fun harmless example: https://www.jwz.org/blog/2021/06/using-ritual-magic-to-trap-...


That video is amazing. Play it back at low speed, the airbags are fully deployed before the hood even begins to crumple.

Additionally, we see just how a real human reacts to a real present danger: confusion. After identifying the threat the pedestrian instinctively runs into the path of the vehicle, just to run.


Yep and they’re fully deployed within one frame of this video.


Autopilot crash so your saying autopilot (aka robot driver/car) is at fault and yet I'm spreading FUD?


It's not an autopilot crash. That's the FUD you're spreading.


If anything Starlink is (along with SpaceX) the least hypey Musk project, it has some minor rough edges but seems to actually be doing a good job of fulfilling its goal. Comparing it to Hyperloop doesn't seem reasonable to me.


And it has to be remarked that Hyperloop was solely a concept Musk published to share it, as he personally had no plans of implementing it himself. The only other involvement of his with it is sponsoring a yearly event where universities can compete on a one-mile test track on the SpaceX site in Hawthorne.


starlink is possibly the most hyped on the financial side. there are virtually no scenarios where they can make money and still maintain the plans they have now. I know people will disagree, but we'll have to see in a few years.


>SpaceX - Starlink (AS14593) has added 2x100G bringing their capacity to a total of 2x2x100G

Why not 4x100G?

>Welcome Apple Inc. (AS714) at 2x1x100G.

Why not 2x100G?

>Akamai Technologies, Inc. (AS20940) has added 3x100G bringing their capacity to a total of 3x100G and 2x100G

And why not 5x100G.

How does one read these numbers?


As another said it is referring to LAG (Link Aggregation Groups or "EtherChannel").

From a logical construct perspective each "2x100G" interface is represented by a single "200Gbps" layer 3 interface. When referring to 2x2x100G, it really is meaning 2 layer 3 interfaces, consisting each of 2x100G ethernet interfaces in the link aggregation group.

Hashing of L3/L4 header data provides for consistent flow -> lag member mapping to avoid OOOs and other historical multi-link issues.


These are all likely LAG-style circuits, bonding multiple physical circuits into a single circuit. This increases the overall bandwidth available without increasing management and routing overhead needlessly, as individual circuits may do.


1yr plus PacNW user; this is likely adding needed backhaul for this high demand region.


I follow Starlink's IP space and connectivity closely, if anyone is interested, see this: https://old.reddit.com/r/StarlinkEngineering/comments/ofly6k...


100Gbps today is like 300 customers. Hmmmmmm


Residential customers don’t use anything close to maximum bandwidth regularly.

It gets better as you with faster connections and more users. At 100Mbps you can roughly do 20:1 over subscription, bump that to 1Gbps and 50:1 generally works fine. StarLink is aiming for roughly 50-100Mbps connections so this likely represents around 20,000 to 40,000 customers per 100Gbps link.

“SpaceX - Starlink (AS14593) has added 2x100G bringing their capacity to a total of 2x2x100G. They are connected to the route servers.” In other words they hit 40,000 to 80,000 customers and are preparing for another 40,000 to 80,000.


Note that it's hard to quantify this evenly between ISPs, because isp A might have a hundred gig port to an ix and meet all the major cdns and Netflix and Google (YouTube video traffic) over the ix exclusively, while ISP B might have also a hundred gig port but also set up many PNIs for direct cross connect and peering with the largest cdns and sources of traffic.

Meaning that ISP B will need to upgrade its ix port at a much later date relative to subscriber count, but its PNIs and cross connects are opaque and hard to know about from a public/external third party perspective.


That is customer that are currently being covered. It is 40k to 80k at both West Coast, Midwest and Northeast so that is Times 3 plus the rural area of the West and how many that is. Then we have Europe. So I feel like there is a lot of bandwidth for millions. I can't find the coverage per satellite. How much ground do they each cover?


> How much ground do they each cover?

https://starlink.sx/


they announced they hit 145,000 customers, so that math doesn't add up. they will need roughly 50:1 to be anywhere near profitable.


They don't transmit between satellites right now, and even if they did they'd still want to minimize that bandwidth and keep any transmissions that they can one hop.

One station feeds about one state on average, and I doubt they're feeding everything through that one connection.


No, it's not, you can fit an absolute shitload of individual 1 gigabit per second symmetric last mile residential customers in the actual usage of a 100Gbps transit port, before it starts hitting 65Gbps in evening peaks and you start thinking of needing to upgrade.


A metric shitload or an imperial shitload?


Nautical shitload, obviously, it's measured by the fucktonne


An absolute shitload. That's a metric shitload plus 273.


Aside from oversubscription only a fraction of any customer's traffic will traverse the IX. The ratio will depend on who Starlink is peering with on the SIX (not necessarily everyone) and the traffic flows to/from those ASN's.

That said there are some very significant content peers present on the SIX: Akamai, Amazon, Apple, Edgecast, Facebook, Google, Microsoft, Valve, among others. Hurricane Electric is also there and has an open peering policy despite being a very large ISP with a significant percentage of Internet routes and traffic.


Starlink has BGP sessions with the SIX route servers so from that alone you know they peer broadly on the exchange.


More like 15,000 if you average 1TB per customer per month with 50%-ish burst.


You really think your ISP reserves 300Mbit of their bandwidth just for you?

I got curious and checked the local IX (smaller EU country), one of the largest ISPs has 4x100G connections, but their combined maximum ingress+egress almost never goes beyond 100G.


I think my ISP would treat that as a million customers.


Read your contract for gigabit fiber. Here in France the ones i've read best promised 512Kb/s available bandwidth at any time, for consumer offers advertizing between 100/100 and 1000/250Mbit/s.

Dedicated gigabit bandwidth is still in the thousands of euros monthly, or in the hundreds in optimal situation (if you're already in a well-connected colo).


Retail consumers do not receive dedicated bandwidth. ISPs will oversubscribe the promised amount 25x-100x because not everyone is using it at peak capacity 24x7.


I don't think this is for customers, they are using Google's network for customers. More likely it's for the Starlink building.


Does 2x100G mean 100G each way?


No, at this level, connections are symetric. The next question is probably why is it reported as 2x2x100G? That's most likely two separate connections which are each an aggregated connection of two 100G connections. This is likely to two separate routers on the Starlink.


> The next question is probably why is it reported as 2x2x100G?

At a certain size you don't want 1x100G in case one link breaks, so for a large company they get 2x100G with each fibre going to a separate router. If one router goes down (planned or unplanned) you don't go offline. So now they probably added a second redundant pair.

You can see SIX's gear here:

* https://www.seattleix.net/topology

They have two Arista 7500-series chassis switches, one 8U and the other 12U (amongst other things):

* https://www.arista.com/assets/data/pdf/Datasheets/7500RDataS...

One of the line card options is 7500R-36CQ, which can handle thirty-six 100G ports.


and the biggest chassis can handle 12 of those cards.


The 7512R at SIX can handle 12. Arista has a 7516R that can hold 16 cards, for a total of 576 100GbE ports.

That's not even Arista's highest capacity switch; the 7816R3 can handle 576x400G:

* https://www.arista.com/en/products/platforms

(I'm sure other vendors have similar product offerings.)


More importantly it's probably to 2 redundant routers on SIX side also so that both can handle HW failure.

On the SIX side the ports are probably already in a redundant chassis but having 2 ports probably still scores you connections to two fully independent chassis in different failure zones.


The last time I got a tour of the SIX, there were switches on two different floors in the Westin building. This is more for capacity than redundancy. The original location is literally a janitor's closet in a hallway and it's so tight that exhaust fans are built into the door. The second location was in proper four-post racks in a small room. It is possible they have ports on both switches for redundancy but more likely both are on the newer location. The SIX is also built into other buildings and they could connect to more than one, but there is typically only 200G backhaul to the other sites. https://www.seattleix.net/topology

2x100G to an IX are usually connected as separate ports, each with their own assigned IP ont he IX subnet. BGP multipath can enable load balancing across both ports. Otherwise you might have some peers configured on one port and others on the second. These days I like to have all peers on both ports and use BGP multipath.

If both ports are connected to a single switch on the IX side and the same router on the customer side then LACP can be used to combine them into a single 200G port with one IP. This is a less common configuration unless you are doing more than two ports (4x100G as 2x 2x100G lags for example).

There is another consideration: what happens if there is an icmp or arp flood on the IX? It's not supposed to happen but sometimes it does, even on the most major and well-run exchanges. In that case you might want to have all of your IX ports on one of your routers, keeping the other one "clean", just in case your control plane policing doesn't mitigate it.

Different networks place different priority on IX ports. In most cases it's ok if it's down for a short time (traffic will just shift to paid transit) so redundancy is not essential. In other cases it may be too much traffic to reroute so you care more about IX reliability and redundancy.


Just want to say really great summary. I know these things but I bet many folks reading this glean an insight or two.

Been a lot time since I have done all this stuff though. I sort of miss it sometimes (though not the pager that comes with the territory).


> More importantly it's probably to 2 redundant routers on SIX side also so that both can handle HW failure.

Strictly speaking, IXPs do not work at OSI Layer 3, but usually at Layer 2.

All the Layer 3 routing logic happens in the routers of the members/customers of the IXP. The IXP is just there to provide a neutral interconnect to everyone without any filtering or policy.

The IXP runs router reflectors where people can choose to advertise themselves, and then other folks see who is in the neighbourhood and make their own routing policy choices.

The folks at Network Startup Resource Center (NSRC) have a series of videos on "IXP Design and Implementation":

* https://learn.nsrc.org/bgp

They're by Philip Smith, who helped set up LINX (London, UK) in the early days:

* https://nsrc.org/bios/PhilipSmith.html


This shows there are 4 BGP peers at SIX https://bgp.he.net/AS14593#_ix


Two of those are on the normal vlan, and two are on the jumbo frames vlan. Both vlans are available on the same physical ports.

Confusingly they list each of those as 200G capacity, not sure if that's a typo or it really is 2x 2x100G.

https://peeringdb.com/net/18747


200G because it's two 100GbE ports in 802.3ad link aggregation


Most network links imply duplex. From this (and the description detail in the OP), I would say they have added 2 links (to the 2 they had previously), each link will have 100Gbps up and down through the IX.


No, 200 each way (2 100gbit links).

Not really sure why this is a big deal - IX bandwidth is very very cheap. It looks like seattleIX charges $7500 as a one off payment to add a 100gbit link: https://www.seattleix.net/pr20191218).


Mostly, network connections aren't so public, so it's interesting in general. Also, upgrading their IX capacity may be a sign that their product is maturing or has significant use. As an upstart ISP, Starlink needs to seek out the connections to CDNs that their customers demand, and an IX connection is a good place to get them at substantially less cost than transit.


Transit is very cheap these days if you're buying 100G ports.. the economics of whether to peer or not aren't as clear cut as they once were.


>IX bandwidth is very very cheap

I would describe it as "capacity", not bandwidth. That fee is just to physically plug into the router. You have to make separate agreements with the other organizations also plugged into that router for peering or paid transit.


Is this a direct 'connection' to the satellites?


No, this is at one part of the most fundamental core terrestrial portion of the internet, how ISPs talk to each other.


An Internet eXchange Point (IXP, or IX in some languages) is a physical room or building where network operators meet and exchange trafic. The internet flows on physical cables, but depending on where the data goes different tariffs apply.

As an ISP, you will pay a transit company to transport your traffic to areas/operators you don't have connection to (very expensive, especially for small ISPs who get charged >1€/Mbit/s). So there's an incentive to establish direct connection with other operators your traffic may need to reach: that's what an IXP is for. Typically, you pay a membership fee to pay for services (electricity, network cabling), and then you can exchange traffic with everyone there for free (peering).

So there's really nothing new, unexpected or exciting about this announcement. It's just an ISP getting ready to exchange more traffic with other ISPs.


This is a great news, giving some love on SIX


What is the significance of it? Wouldn't adding bandwidth be expected?

(Do even Y Combinator companies get as much coverage on HN as Musk's?)


Yeah, given that they're adding customers continuously they should be adding peering and transit proportionally.


There is zero significance: it's just expensive cables.


[flagged]


Please don’t turn Hacker News into Reddit.


How is this comment not appropriate? Does it need to say something more thoughtful? Accusing his companies of selling smoke is a very valid accusation.


It made me chuckle. What makes it funny is that it is almost a direct quote from Elon to investors this week.

It would have been less controversial if they added a link for context. Most readers here won't know that Elon just doubled down on his FSD claims.

"After years of delays and hype-filled tweets, Elon Musk now claims that Tesla’s full self-driving feature will release in 2022 and that the technology is safer than human drivers. The exact quote from the most recent Tesla investors call is,

  “I would be shocked if we do not achieve Full Self-Driving safer than human this year. I would be shocked,”
https://mobilesyrup.com/2022/01/31/elon-musk-says-self-drivi...


[flagged]


I don't agree, there are a lot of thoughtful and nuanced conversations on HN all the times.

By the way, from the guidelines:

> Please don't post comments saying that HN is turning into Reddit. It's a semi-noob illusion, as old as the hills.

https://news.ycombinator.com/newsguidelines.html


Thank you, yes I am aware of that guideline. At the risk of excessive pedantry, I didn't actually say that Hacker News "is" turning into Reddit. You are correct that there are a lot of thoughtful and nuanced conversations on Hacker News. In fact I'd say that it's an understatement—this community is remarkable in how it has managed to defy entropy despite its scale.

Even so, it's not perfect. Certain topics seem to unearth some rather bitter trolls—and I find the lack of push-back to be striking in contrast to how similar behaviour is responded to in other topics.

(As for the guideline, I'm sure whoever wrote it a decade ago was well-meaning in their intent. But in 2022 it reads as reductive, maybe a bit passive-aggressive and, dare I say it, perhaps even a wee bit narcissistic. "Of course the Hacker News audience isn't turning into Reddit, you noob. It's impossible. Our audience is forever perfect.")


> Thank you, yes I am aware of that guideline. At the risk of excessive pedantry, I didn't actually say that Hacker News "is" turning into Reddit

Hello fellow "pedanter" :-) I did pick up your nuance. To be clear, I was replying to this part of my parent message "It feel like it already is. Gone […]" But sorry anyway, I didn't mean to annoy you.


I'm the one who needs to apologise. I did eventually realise my error but I was so satisfied by my own diatribe that I decided to leave it as-is.


Against a strong reflex of "never change a running system" (because your first paragraph is spot-on and meta like this is already watering down the quality), I wonder if offering "inappropriate for hn" flags to the posting person as lightning rods to prevent that kind of discussion on the main channel could be an improvement.

Categories like "reply jokingly/consent without adding much/dissent without adding much/reply meta" that could be flagged while posting, and filtered out for reading unless explicitly enabled (per flag).

That would most likely ruin hn in one way or the other. Why? Because it must have been tried hundreds of times before and those attempts lie in obscurity while hn works. But how exactly would it fail? "What if" wargaming through possible failure modes is surprisingly fascinating to me (and difficult: in what way exactly would it fail? no idea!).


Effectively banning these types of comments is what makes hackernews into exactly what normies think Reddit is like because it trains the readers not to look for sarcasm.


Nonsense, it was a straight-up troll. The fact a comment can masquerade as sarcasm doesn't somehow rehabilitate it.

There are a million permutations of unproductive contribution which are already "effectively banned" and those haven't turned Hacker News into Reddit. And your belief that readers need to be trained to look for sarcasm is rather paternalistic and a touch condescending. (Sarcasm has been largely ruined by the internet anyway; see: Poe's Law.)


He did recently say publicly that he would be shocked if they didn't get to safer than a human driving this year.

So what about my comment was trolling?

https://electrek.co/2022/01/31/elon-musk-tesla-full-self-dri...


Sarcasm on the internet is usually more trouble than it's worth, Poe's Law and all that.


Please don't post unsubstantive or flamebait comments here. We're trying for something else.

https://news.ycombinator.com/newsguidelines.html

We detached this subthread from https://news.ycombinator.com/item?id=30189253.



> Elon says they will reach safer than a human self driving this year.

Ha! Don’t worry. He will continue to say that for the next few years




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: