The problem is that there isn't an easy solution to this. We have made efforts to make things better for Tor users (see my comments in the linked ticket above) but at the same time there is abuse coming from Tor which we have to deal with and we see the amount of abuse increasing.
As I said in the linked thread we are about to allow our customers to whitelist Tor exit nodes so that Tor users do not get challenged when coming from an exit node that's been used for a lot of abuse. That will allow any customer to decide what's appropriate.
There's no easy solution because Tor refuses to do anything that would allow for anything that works within current technology. This is not a new problem. If the Tor guys really want to stop being frozen out of large parts of the internet they need to
1) Tackle abuse
2) Go work for one of the large networks for a while so they see things from the other side, and then do a lot of research until they find a solution they know can make everyone happy.
In fact Tor have repeatedly gone for solution 3:
3) Attempt to publicly shame employees of big networks and get them to lower their abuse defences by implying they're idiots who hate privacy, whilst simultaneously demonstrating little understanding of what abuse teams have to deal with. This never works.
Tor is a trivial fraction of anonymised traffic to most major networks (VPNs are far more popular). It can be banned entirely with ~zero impact on any large business. Taking approach (3) just annoys the employees and makes them less likely to want to donate free engineering time to the Tor project (because that's effectively what they want/need). We can see this clearly happening on the linked thread.
E.g. comments like this one:
It's also unreasonable to maintain a "reputation" for a Tor exit node.
... just send a powerful message that Tor isn't serious. Sorry for the harsh words, but I've been round the block on this one too. Tor want to run a network that ignores abuse yet is openly welcomed by the worlds biggest content providers. Not gonna happen.
I grew up in a country where the Internet was censored and I have been a Tor user since the days of the dangerous toggle button in Firefox. So it's been like 7-8 years? I hope I can offer a counter argument.
This is not "business" as much as it is about CloudFlare centralizing the Internet around them. We should speak up against it and rightly so.
When I started using Tor, it was the only way I could access the free Internet. You mention VPN, VPNs may be popular but not everyone can pay for them or afford them or even has a credit card in many countries. With CloudFlare running on many websites, it has become impossible to read any content on them because the CAPTCHAs are impossible to solve or are a nuisance (who has figured out what a street sign is?). And while Tor gives users to access the internet freely for zero cost, CloudFlare is helping doing the opposite of that.
And please suggest what should an open network do about abuse and rather, why should they be the ones doing anything about it? Not to mention, what kind of abuse would reading an article constitute? I am not even logging in, just reading stuff has been made impossible due to CloudFlare.
I have the utmost respect for people fighting censorship, whether they're on the inside breaking out or on the outside breaking in.
But the idea that only Tor can be used to fight censorship is a false equivalence. If you look at the usage data for e.g. Turkey when they stepped up censorship a few years ago, Tor usage increased a bit, but it was a tiny amount compared to HotSpot Shield (a VPN product). Most users evading censorship in places like China or Turkey are happy to trust the VPN provider in return for superior performance or app compatibility, because they know little VPN providers aren't going to snitch them out.
Yes, normally people have to pay for VPNs whereas Tor is free. I think it'd be great if there was a collaborative network of people who donate their bandwidth to a kind of decentralised VPN (single hop, authenticated, with abuse checks, unlike Tor). But ultimately running these sorts of networks takes effort and time, and most organisations don't have access to the various grants and such that Tor has.
> please suggest what should an open network do about abuse and rather, why should they be the ones doing anything about it
Keep it under control, same as all other networks are expected to do. There are all sorts of networks that range from very closed (e.g. university networks) to very open (e.g. open signup clouds, Google, Microsoft email networks etc). But none of them have a get-out-of-jail-free card, not even the biggest.
Why should they be the ones to do anything about it - because that's how the internet works. If your network spews abuse onto the platform then it's gonna get blocked and blacklisted. If you don't want your users to all be lumped in the same bucket, then you have to do the differentiation yourself.
The linked thread already explains why simple GET requests can be problematic all by themselves.
Surely that would be a 'better' use of your time than claiming someone who is lamenting that CF won't even return simple read-only get requests without sending even larger captcha pages first is "implying [CF are] idiots who hate privacy, whilst simultaneously demonstrating little understanding of what abuse teams have to deal with".
1) Tackling abuse requires a consistent definition of abuse that transcends cultures and can be implemented without major privacy and/or censorship implications.
2) The fact I can avoid the large networks and mitigates the abuse problems simultaneously implies the issue is largely a values issue rather than a technical one about how one defines abuse.
3) 99.9999999% of the spam I have to mitigate comes from "big networks". I've literally had 1 piece of spam in 2016 come from Tor. I've only ever had DoS/DDoS from compromised servers on "big networks" who fail to notice for day(s) and require manual notification. So when you say "abuse problems", it clearly means something different to you than it does to me.
4) The "big networks" are not interested in cooperating with relatively cheap solutions like the one CF is now implementing until they are publicly called out. I asked for that 4+ years ago and was ready to move an account at $DayJob to CF until I realized how unresponsive they were.
Sometimes yes, sometimes no. In practice, for example, webmail spam filters tend to be (virtually) culturally neutral because they're driven by voting over large populations. In cases where someone writes a specific policy, yes, it will be culturally biased. That may be inevitable.
If you're talking about email spam, then Tor blocks SMTP outbound, and big webmail services all treat send attempts from Tor with suspicion, so yeah you'd not see much direct SMTP spam from them.
And yes, Tor tends to already be blocked by any reasonable abuse filter, whereas the big networks are too-big-to-block, so that's where spammers focus their efforts. But that's why the big networks have full time outbound abuse teams (what I used to work on) and do spend significant sums of money on trying to keep their networks clean.
You can argue that they're not doing enough, but when it comes to stuff like that, there is always more that can be done.
Just to be clear, I'm not talking about just email spam [email spam is actually something we outsource the handling of].
I'm talking about side projects that are little more than forums to 9 figure/year eCommerce companies I find big providers to be more of a problem with comment, review, etc. spam.
Whether they spend a large amount of money mitigating it or not, the fact that the quality of Tor [proportional to traffic] is not substantially worse compared to a VPN provider on those networks in my experience doesn't fill me with confidence that your solution would actually work.
> But that's why the big networks have outbound abuse teams (what I used to work on) and do spend significant sums of money on trying to keep their networks clean.
Any suggestions on how to get them to actually respond within a reasonable amount of time and with a substantial response (rather than a brush off) to things like people testing out credit cards to place fraudulent orders via their IP Space / VPNs?
Ah, you're talking about VPN providers. Sorry. I thought you were talking about networks like Google, Microsoft, Amazon, Facebook etc.
Some VPNs do fight abuse. Others are basically like Tor: they ignore it. The best thing you can do in those cases is indeed just block their IP space. Cross-checking the global routing tables can be a good way to enumerate their IP space. Sorry.
A) Provide VPNs themselves to the general public. [e.g. Their web proxies]
B) Provide infrastructure used by VPN providers.
So yes, I was including those two.
Microsoft provides infrastructure to VPN providers as well via Azure, so they also qualify.
Facebook would not.
Other providers that qualify:
Every hosting company that provides dedicated or VPS. [e.g. Linode, OVH, Hetzner]
I just don't bother publicly calling people out on what appears to be a systemic problem. A problem that, frankly, Tor is no worse than any of the other actors.
To me, the odd behavior is the fact CF seems to treat them as bad actors despite the fact they are an insubstantial portion of the problem.
Yeah, but in my experience, their extensive anti-abuse teams still end up with as much abuse [proportional to overall traffic] as Tor, if not more.
So to me, I don't understand why people complain about Tor when the real problem is the fact the larger networks simply take too damned long and throw far, far greater problems at people.
To me, the idea of blocking/throttling Tor more heavily than any other provider seems patently absurd for that reason.
I understand other people may have different experiences but that doesn't create a great deal of sympathy from me when a simple white list option takes 4-5 years.
Some VPNs don't and they get treated the same as Tor i.e. CAPTCHAd a lot, throttled, outright blocked (I've seen it happen).
Others do keep logs and use them. Guess what? They're not pointless. Lots of people don't require absolute anonymity against everyone - they just want to get over a particular firewall or shield their IP address from one particular network. But they don't include the VPN provider (or host country) in their threat model.
I'm sure the Tor network can't be used to seriously DDoS CF. So the abuse of which you are talking about must be just the spam (can be unpleasant for your clients but safe for CF) and the anonymous hacking attempts (again unpleasant for the clients but safe for CF).
My solution would be to not block Tor by default, instead let your clients do that if and only if they want to block it. Or, at least, let them make an informed decision when setting up the account instead of a default choice.
I think reading information using Tor should never be disableable. Adding an option to disable it is encouraging people to remove the freedom for people to read texts anonymously. How is that a feature? It's an even more serious issue than censorship, because it's attempting to force people who use Tor into ignorance.
As for writing to the site (POST), the option should be labeled "censor users in oppressed regimes". That might give it enough emotional weighting.
As I said in the linked thread we are about to allow our customers to whitelist Tor exit nodes so that Tor users do not get challenged when coming from an exit node that's been used for a lot of abuse. That will allow any customer to decide what's appropriate.