Sites routed through CloudFlare are practically unbearable to use over Tor and ReCaptcha 2.0 has actually made things worse. The new challenges take more time to solve and most times you're required to complete multiple challenges before being accepted.
I'm glad to hear that this is being worked on, but I'm pretty pessimistic that there will be a good resolution. There seems to be a fundamental conflict of interest between preserving anonymity and being able to identify potential abuse. As CloudFlare grows and absorbs more and more of the Internet, the inconvenience and potential for tracking will only increase. The problem is really more of a structural one than a technical one - this is what happens when one entity controls vast portions of the infrastructure of the Internet.
I'm hopeful. The more we do abuse detection by intelligent means the more we are able to spot abuse without using CAPTCHAs or other tools like that. We are about to allow our customers to whitelist Tor so that sites that want an immediate solution for Tor users can disable the CAPTCHA check.
Forget Tor, what about VPNs and shared networks? CloudFlare has a great way to make the web unusable. The number of small blogs that seem to use it has become infuriating.
I typically browse via a VPN, and CF present their capture blocking on every CF site visited. This wouldn't be too bad, but the captcha is often infuriating and the CF page + capture is consistently one of the slowest loading.
Lumping every user of a VPN or shared network in with the spammers is short sited, and "allowing customers (of Cloudflare presumably?) to whitelist" rather than presenting any possible solution for the innocent browser is typical of what the internet has become.
If someone is on a large shared network - VPN or university NAT, someone will have virus, spam or other crap on their machine. There needs to be a solution that ensures the other hundreds of people on that net can be minimally impacted. Perhaps a way to captcha ONCE per xx hours as a browser, rather than 5x in 5 minutes as is often the case now.
It's got so annoying I find myself wishing someone made an Adblock filter list for CF sites.
OK, that was careless writing. Better would be "Of equal importance". Net result right now is Cloudflare is equally unfriendly to both classes of users.
I'd imagine the solutions similar, though someone on Tor would probably care far more about using a CF cookie for captcha bypass solution than someone on a large NAT.
with a simpe cron job like this ^ you can benefit from cloudflare without blocking TOR.
btw cloudflare if you see this. I suspect you could make all of this drama disappear in an instant by adding a convenient firewall control panel option called "whitelist TOR". Then all of the grumbling would be redirected towards the individual websites who neglected to apply this option.
btw cloudflare if you see this. I suspect you could make all of this drama disappear in an instant by adding a convenient firewall control panel option called "whitelist TOR". Then all of the grumbling would be redirected towards the individual websites who neglected to apply this option.
You mean the thing I said we are about to do both in this thread and in TFA? :-) We are going to allow customers to do that. Very soon.
one other thing since you're here: you could have a second button: whitelist the top 20 most popular private VPN services, and college dorm NAT environments.
I know that sounds annoying because it's not a simple task to compile and maintain such a list, but I suspect it's the kind of work that a single non-technical intern could have as a partial responsbility. If you left this control in the "allow" position by default then I bet these VPN providers would start policing their own networks as a means to win your blessing to stay onto the whitelist.
Then consumers would have a better idea about which VPN providers are respectable at least in terms of whether or not they nix the most obvious forms of breach attempts/scanning emanating from their nodes. That may sound far-fetched, but I just wanted to throw it out there.
I'd like to see a better solution for CF/Tor interactions but, to be perfectly honest, Tor is too small a portion of CF's user base for them to put serious effort into this and the fact their solution is more of the same seems to largely reinforce this point.
I honestly refuse to use them when I have a choice in the matter because I don't have much faith in the dark mass of data that they have access will be used correctly. The fact they need to be called out publicly by the TBB team pretty much reinforces this belief.
The problem is that there isn't an easy solution to this. We have made efforts to make things better for Tor users (see my comments in the linked ticket above) but at the same time there is abuse coming from Tor which we have to deal with and we see the amount of abuse increasing.
As I said in the linked thread we are about to allow our customers to whitelist Tor exit nodes so that Tor users do not get challenged when coming from an exit node that's been used for a lot of abuse. That will allow any customer to decide what's appropriate.
There's no easy solution because Tor refuses to do anything that would allow for anything that works within current technology. This is not a new problem. If the Tor guys really want to stop being frozen out of large parts of the internet they need to
1) Tackle abuse
2) Go work for one of the large networks for a while so they see things from the other side, and then do a lot of research until they find a solution they know can make everyone happy.
In fact Tor have repeatedly gone for solution 3:
3) Attempt to publicly shame employees of big networks and get them to lower their abuse defences by implying they're idiots who hate privacy, whilst simultaneously demonstrating little understanding of what abuse teams have to deal with. This never works.
Tor is a trivial fraction of anonymised traffic to most major networks (VPNs are far more popular). It can be banned entirely with ~zero impact on any large business. Taking approach (3) just annoys the employees and makes them less likely to want to donate free engineering time to the Tor project (because that's effectively what they want/need). We can see this clearly happening on the linked thread.
E.g. comments like this one:
It's also unreasonable to maintain a "reputation" for a Tor exit node.
... just send a powerful message that Tor isn't serious. Sorry for the harsh words, but I've been round the block on this one too. Tor want to run a network that ignores abuse yet is openly welcomed by the worlds biggest content providers. Not gonna happen.
I grew up in a country where the Internet was censored and I have been a Tor user since the days of the dangerous toggle button in Firefox. So it's been like 7-8 years? I hope I can offer a counter argument.
This is not "business" as much as it is about CloudFlare centralizing the Internet around them. We should speak up against it and rightly so.
When I started using Tor, it was the only way I could access the free Internet. You mention VPN, VPNs may be popular but not everyone can pay for them or afford them or even has a credit card in many countries. With CloudFlare running on many websites, it has become impossible to read any content on them because the CAPTCHAs are impossible to solve or are a nuisance (who has figured out what a street sign is?). And while Tor gives users to access the internet freely for zero cost, CloudFlare is helping doing the opposite of that.
And please suggest what should an open network do about abuse and rather, why should they be the ones doing anything about it? Not to mention, what kind of abuse would reading an article constitute? I am not even logging in, just reading stuff has been made impossible due to CloudFlare.
I have the utmost respect for people fighting censorship, whether they're on the inside breaking out or on the outside breaking in.
But the idea that only Tor can be used to fight censorship is a false equivalence. If you look at the usage data for e.g. Turkey when they stepped up censorship a few years ago, Tor usage increased a bit, but it was a tiny amount compared to HotSpot Shield (a VPN product). Most users evading censorship in places like China or Turkey are happy to trust the VPN provider in return for superior performance or app compatibility, because they know little VPN providers aren't going to snitch them out.
Yes, normally people have to pay for VPNs whereas Tor is free. I think it'd be great if there was a collaborative network of people who donate their bandwidth to a kind of decentralised VPN (single hop, authenticated, with abuse checks, unlike Tor). But ultimately running these sorts of networks takes effort and time, and most organisations don't have access to the various grants and such that Tor has.
> please suggest what should an open network do about abuse and rather, why should they be the ones doing anything about it
Keep it under control, same as all other networks are expected to do. There are all sorts of networks that range from very closed (e.g. university networks) to very open (e.g. open signup clouds, Google, Microsoft email networks etc). But none of them have a get-out-of-jail-free card, not even the biggest.
Why should they be the ones to do anything about it - because that's how the internet works. If your network spews abuse onto the platform then it's gonna get blocked and blacklisted. If you don't want your users to all be lumped in the same bucket, then you have to do the differentiation yourself.
The linked thread already explains why simple GET requests can be problematic all by themselves.
Surely that would be a 'better' use of your time than claiming someone who is lamenting that CF won't even return simple read-only get requests without sending even larger captcha pages first is "implying [CF are] idiots who hate privacy, whilst simultaneously demonstrating little understanding of what abuse teams have to deal with".
1) Tackling abuse requires a consistent definition of abuse that transcends cultures and can be implemented without major privacy and/or censorship implications.
2) The fact I can avoid the large networks and mitigates the abuse problems simultaneously implies the issue is largely a values issue rather than a technical one about how one defines abuse.
3) 99.9999999% of the spam I have to mitigate comes from "big networks". I've literally had 1 piece of spam in 2016 come from Tor. I've only ever had DoS/DDoS from compromised servers on "big networks" who fail to notice for day(s) and require manual notification. So when you say "abuse problems", it clearly means something different to you than it does to me.
4) The "big networks" are not interested in cooperating with relatively cheap solutions like the one CF is now implementing until they are publicly called out. I asked for that 4+ years ago and was ready to move an account at $DayJob to CF until I realized how unresponsive they were.
Sometimes yes, sometimes no. In practice, for example, webmail spam filters tend to be (virtually) culturally neutral because they're driven by voting over large populations. In cases where someone writes a specific policy, yes, it will be culturally biased. That may be inevitable.
If you're talking about email spam, then Tor blocks SMTP outbound, and big webmail services all treat send attempts from Tor with suspicion, so yeah you'd not see much direct SMTP spam from them.
And yes, Tor tends to already be blocked by any reasonable abuse filter, whereas the big networks are too-big-to-block, so that's where spammers focus their efforts. But that's why the big networks have full time outbound abuse teams (what I used to work on) and do spend significant sums of money on trying to keep their networks clean.
You can argue that they're not doing enough, but when it comes to stuff like that, there is always more that can be done.
Just to be clear, I'm not talking about just email spam [email spam is actually something we outsource the handling of].
I'm talking about side projects that are little more than forums to 9 figure/year eCommerce companies I find big providers to be more of a problem with comment, review, etc. spam.
Whether they spend a large amount of money mitigating it or not, the fact that the quality of Tor [proportional to traffic] is not substantially worse compared to a VPN provider on those networks in my experience doesn't fill me with confidence that your solution would actually work.
> But that's why the big networks have outbound abuse teams (what I used to work on) and do spend significant sums of money on trying to keep their networks clean.
Any suggestions on how to get them to actually respond within a reasonable amount of time and with a substantial response (rather than a brush off) to things like people testing out credit cards to place fraudulent orders via their IP Space / VPNs?
Ah, you're talking about VPN providers. Sorry. I thought you were talking about networks like Google, Microsoft, Amazon, Facebook etc.
Some VPNs do fight abuse. Others are basically like Tor: they ignore it. The best thing you can do in those cases is indeed just block their IP space. Cross-checking the global routing tables can be a good way to enumerate their IP space. Sorry.
A) Provide VPNs themselves to the general public. [e.g. Their web proxies]
B) Provide infrastructure used by VPN providers.
So yes, I was including those two.
Microsoft provides infrastructure to VPN providers as well via Azure, so they also qualify.
Facebook would not.
Other providers that qualify:
Every hosting company that provides dedicated or VPS. [e.g. Linode, OVH, Hetzner]
I just don't bother publicly calling people out on what appears to be a systemic problem. A problem that, frankly, Tor is no worse than any of the other actors.
To me, the odd behavior is the fact CF seems to treat them as bad actors despite the fact they are an insubstantial portion of the problem.
Yeah, but in my experience, their extensive anti-abuse teams still end up with as much abuse [proportional to overall traffic] as Tor, if not more.
So to me, I don't understand why people complain about Tor when the real problem is the fact the larger networks simply take too damned long and throw far, far greater problems at people.
To me, the idea of blocking/throttling Tor more heavily than any other provider seems patently absurd for that reason.
I understand other people may have different experiences but that doesn't create a great deal of sympathy from me when a simple white list option takes 4-5 years.
Some VPNs don't and they get treated the same as Tor i.e. CAPTCHAd a lot, throttled, outright blocked (I've seen it happen).
Others do keep logs and use them. Guess what? They're not pointless. Lots of people don't require absolute anonymity against everyone - they just want to get over a particular firewall or shield their IP address from one particular network. But they don't include the VPN provider (or host country) in their threat model.
I'm sure the Tor network can't be used to seriously DDoS CF. So the abuse of which you are talking about must be just the spam (can be unpleasant for your clients but safe for CF) and the anonymous hacking attempts (again unpleasant for the clients but safe for CF).
My solution would be to not block Tor by default, instead let your clients do that if and only if they want to block it. Or, at least, let them make an informed decision when setting up the account instead of a default choice.
I think reading information using Tor should never be disableable. Adding an option to disable it is encouraging people to remove the freedom for people to read texts anonymously. How is that a feature? It's an even more serious issue than censorship, because it's attempting to force people who use Tor into ignorance.
As for writing to the site (POST), the option should be labeled "censor users in oppressed regimes". That might give it enough emotional weighting.
Just tried to browse with the TOR browser my websites (funlabo.com, defouland.com, stratozor.com) which have "Essentially Off" firewall settings on CloudFlare: I was indeed greeted by a bit annoying Google Captcha for each domain.
That said, I am thankful for the services provided by CloudFlare, notably on the performance side and you do have an option "Off" for this if you pay for an Enterprise plan. I still recommend them.
So if I'm understanding this correctly, CloudFlare offer free CDN and DDoS protection services to websites on the (unmentioned) condition that they make those sites essentially unusable over Tor? Wow. No wonder the Tor developers consider this an attempt to enable mass surveillance.
They're not unusable, it just becomes a bit more inconvenient to use them. In my experience, you need to solve 2-3 captchas per site per new circuit (i.e. every 10 minutes, IIRC, although TFA mentions this might have changed recently).
Exit node IPs end up on a lot of block lists. This is not specific to CloudFlare. It's in the nature of a project like Tor that some of its users use it for things like comment spam, which is one of the things CloudFlare tries to block. This might very well be what most of their customers want - a lot of non-CloudFlare sites block exit nodes completely. They're also adding a whitelisting feature for those who want to disable those features for Tor users.
> on the (unmentioned) condition that they make those sites essentially unusable over Tor
Most people don't see it as a condition like this. It's more like your website will be served through the CF network instead of your normal server. This has a lot of effects: increased security (DDOS and others), increased speed (distributed CDN, caching all your static content), (non js) analytics, free SSL (their certs, this was before letsencrypt) etc. A part of that incresed security is blocking traffic from "suspicious" sources, like TOR. Which your average small business with a website has no problems at all with.
Why would someone put a tor server behind cloudflare in the first place? If you are using cloud flare, you are using it to protect yourself from abuse (DDOS, etc). Doesn't allow GET requests mean you can DDOS things behind cloudflare then?
I'm skeptical that cloud flare can somehow be roped into 'corporate censorship' -- they IMHO clearly were not founded or intended to enable some sort of nefarious intent. Quite the opposite in fact.
TLDR: The title of this article is dubious or even trolling
Running an onion service behind CF (apart from being basically impossible) would lose many of the useful features of onion services. The topic being discussed is accessing clearnet websites hosted behind CF using Tor. Speaking personally, I've had to solve endless amounts of Captcha every fucking day. It actually makes me reconsider how important the site I'm trying to view is, because CF is trying to make it hard for me to read information anonymously.
This isn't what anyone except you is talking about. The issue is being unable to access CF secured clearnet websites using the Tor Browser Bundle (or any other way to route your traffic through Tor).
> I'm skeptical that cloud flare can somehow be roped into 'corporate censorship' -- they IMHO clearly were not founded or intended to enable some sort of nefarious intent. Quite the opposite in fact.
The fact that CloudFlare makes it challenging for users from a particular network to access common resources is, in itself, a form of censorship. This is especially problematic due to the nature of people who tend to use Tor, i.e. those who seek some level of anonymity for various reasons. I don't believe CloudFlare is malicious or intends to block legitimate Tor users. But it is censorship nonetheless, hence the title.
I would encourage Tor users blocked by CF to use archive.org or archive.is save functions instead of solving captchas and teaching Google how to be more human.
I'm glad to hear that this is being worked on, but I'm pretty pessimistic that there will be a good resolution. There seems to be a fundamental conflict of interest between preserving anonymity and being able to identify potential abuse. As CloudFlare grows and absorbs more and more of the Internet, the inconvenience and potential for tracking will only increase. The problem is really more of a structural one than a technical one - this is what happens when one entity controls vast portions of the infrastructure of the Internet.