For $5/mo, you can steal $38,000/month from your enemies

[nikolqy]

It's actually sort of funny. Not for the receiving end, which would likely be able to negotiate or just suspend their service instead of owing $38k, but still. There should be a documentary on this if I'm being honest.

Today's topic: Content Delivery Networks that charge per request.

It's a common practice but it's horrific for smaller companies that can't negotiate contracts.

Fastly, CloudFront, Google Cloud CDN and more all charge for requests made to CDN deployments.

Vultr. Linode. Digital Ocean. -and more. $5/month for a not too terrible server and 1TB of egress. Not a threat until you spam someone's CDN deployment. And no, I'm not talking about 1TB. I'm talking about sending 51 billion requests a month to CDN endpoints for $5/month. Want to mitigate that? That'll cost 10x the amount per request for Google Cloud Armor or Amazon WAF (not kidding). I'm sure this actually is't a common practice, but it makes you wonder about the companies that switch from enterprise CDNs to Cloudflare.......

HTTP stress testing software like wrk is wickedly powerful and insightful. WRK can easily send 20k requests per second per core. Find a resource small enough and it's game over for the receiving end. It can easily be used as a tool for your worst enemies. The only way to mitigate it is to host your own solution, like Varnish etc. or negotiate a contract with the CDN provider, which will costs hundreds or thousands of dollars a month. Not a likely solution for small to medium sized businesses.

Thoughts? Comments? Stories? Ideas?

[tothrowaway]

Small business here. I get hit with DDoS attacks (sometimes as large as 1 Gbps) frequently enough that I won't touch cloud services. I do use Backblaze and Wasabi for user uploads but I proxy them through Nginx (with local caching). My host, OVH, does OK filtering some of the obvious bad traffic, but application level DDoSs get through.

I don't really understand the CDN business. I've got a ~130ms ping to my web apps from where I am right now, and can't say I've noticed it compared to the 60ms ping I had before. If I put my stuff on a CDN, it speeds up loading static assets the first time the page loads (nice). But then I have to worry about the CDN going down and taking me with them, or getting my account disabled by some algorithm in the fraud department (see Cloudflare posts from last week). Seems like a high price to pay.

[ThePhysicist]

Most CDNs would be able to filter such traffic, especially if it comes from a single VM. On the other side, most cloud providers are also quite serious about these things and will cut you off once they notice you're using a VM to DDoS other systems, so you won't be able to do that for very long.

[yellow_lead → ThePhysicist]

I dunno, it's fairly easy to buy VMs with cryptocurrency now. And who's to say they aren't legitimate requests? Especially if it's HTTP, it may not look like a DDOS

[ThePhysicist → yellow_lead]

Most cloud providers react very fast to abuse, they do not have an interest in ruining the reputation of their subnets as that will make life difficult for all of their customers. So they won't ask nicely if your 10 million requests per hour to a single machine are really legitimate, they'll just shut down your account to protect their network.

[serf → ThePhysicist]

> So they won't ask nicely if your 10 million requests per hour to a single machine are really legitimate, they'll just shut down your account to protect their network.

as someone who has had accounts locked down due to perfectly acceptable behavior and had to plead my case with various cloud providers : I concur, they act first and ask questions later.

[Komodai → ThePhysicist]

Some don't, especially not the ones mentioned in the original post :)

Vultr, Linode and Digital Ocean are all absolutely terrible at dealing with abuse.

[nikolqy → ThePhysicist]

Believe it or not, I found out that using Cloudflare Warp allows you to get significantly higher rps rates for cloudflare sites and others. It's a pretty terrible flaw.

[Genbox]

While S3 is not a CDN as such, it does have a feature called "Request payer" which means the requester pays for the request. It won't mitigate anything for public files, but it will mitigate DoS attacks through a third-party.

[wowi42]

We did an article about it 1 year ago!

https://blog.kalvad.com/ddos-on-demand-how-to-properly-load-...

[nikolqy → wowi42]

What a fun blog post! I'll try this on my own stuff. Shockingly, HAProxy protects extremely well with their http-request-reject, tcp deny/reject and http-silent-drop. Would be interesting to test if HA Proxy could actually mitigate these attacks on a single machine, as long as the port isn't fully saturated with requests!

[mdrzn]

I'd say you gotta try it and see how fast you get shut down from the VM or blocked from the CDN. Might not be as easy as it seems.

[dyingkneepad]

It's not stealing, you won't get to see the 38k, unless you are the owner of the CDN and they are your customer.

[nikolqy → dyingkneepad]

But you're taking away their profits that would have been spent on people's salaries etc. It's like stealing something and giving it to someone else, except it's mandatory in this case.

[ipaddr → nikolqy]

You can do many things that force businesses to make decisions that cost them money but it is not calles stealing.

Often 1800 numbers are paid for by the minute. Calling them costs them money. Lobbying for restrictions on businesses cost money.

All of these represent legitimate business. Requesting 38 billion does not and could be considered fraud

[dyingkneepad → nikolqy]

Any sort of damage you cause will have that effect, stealing is a very tiny subset of that where you get to profit. I don't know what's the Legal English term for "causing damages". Vandalism?

[altdataseller → nikolqy]

I mean, theres probably more effective ways to steal from a competitor. Like sign up and book fake demo meetings with their sales team.

[anyfactor]

How does region locking works with CDN? Does out of region requests counts as a chargable request?

If a content is delivered through a CDN can dynamic IP blocking with edge computing prevent requests from being "counted"?

[nikolqy → anyfactor]

Edge computing costs too though, more per request since it’s all serverless. Equinix has some good bare metal servers, and Data Packet has very cheap unmonitored bandwidth bare metal servers. Oracle has great servers and only charges $0.0085/GB transfer.

[pestatije]

I think this is not the right way to present a weakness... ok it's called Hacker News, so fair enough.

[epsteindidntki]

Seems too easy.

[nikolqy → epsteindidntki]

It is. I mean there's plenty of other software out there. Cloudflare has a blog about it. I tested wrk on CloudFront and they really do count Every. Single. Request. It's insane, but I don't think it's common enough to really hear about it. Cloudflare doesn't protect against DDoS for the most part though. Spamming a non-cached content on an average Cloudflare site is enough to crush it. Especially if you're logged in already. As much as these services claim to do, they really don't without extra fees and complex configuration.

TL;DR Do your own content distribution and app firewall solution. It's cheaper and it'll perform better. That's why Facebook and the big guys aren't using cloud services. Even Google uses their own private networks. CDN's are a scam if they're charging per request.

[schappim → nikolqy]

>> Even Google uses their own private networks.

Of course they do.

[Proven]

Doh... That's why people use Cloudflare.