Not everyone can spend $500 on lawyer billable hours per SMTP destination multiplied by N number of destinations.
I also think that the likelihood of success in sending legal threats to somebody that demand they accept your SMTP traffic will not stand up in court, if you ever escalated it that far.
As somebody who runs postfix MX on the receiving side of things, I can guarantee you that the day I receive a legal threat from some unknown third party with which I don't have a pre-existing business/contract relationship, demanding that I accept their email, is the day that I blacklist their entire organization and tell them "okay, I'll await service of your statement of claim".
You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs? Companies with which you don't have a signed service order contract and/or master services agreement?
You say you're a mid sized company. I think you're running a huge legal risk of angering a Comcast or AT&T size entity that has much deeper pockets and legal resources than you. The day that one of those giants calls you out on your bluff is going to be very expensive.
On an ISP-to-ISP relationship level, this is not how you solve SMTP flow traffic problems. I can tell you that if I went to a NANOG conference representing my AS and proudly told other people "oh yeah, we've started sending threats from our lawyers to $OTHERISP1 and $OTHERISP2 because they won't take our mail traffic", that I would quickly be treated as a pariah.
> You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs?
It's not a "network engineering problem" if administrators and managers are the ones making the decisions to provide no reasonable recourse for a ban, on top of actively ignoring or denying legitimate requests for removal. If the third party hasn't broken any rules, then the only resort is getting attorneys involved, because there are literally no other options if the provider refuses to act in good faith.
> Companies with which you don't have a signed service order contract and/or master services agreement?
If you are providing an email service and your customers are not receiving the emails they're expecting, all because you refused to acknowledge a removal request, then it shouldn't matter if you have a relationship with the third party. You've failed your own customers, and for no legitimate, logical or even conceivable reason, other than to assert some personal dominance over another group of engineers that you see as "lesser."
None of this is an engineering problem. It's an asshole problem.
> all because you refused to acknowledge a removal request
But the receiving MSP can't acknowledge the removal request, in the scenario reported by the author. They are not the ones operating the blocklist. That's UCEPROTECT, not Comcast or whoever.
You have this backward, as well, because that's not how any of this works. The UCEPROTECT list is a literal text file that gets ingested by the mail provider. The provider is under zero obligation to use the entirety of the list and, in fact, is still 100% responsible for maintaining their own list in a way that complies with international laws, ICANN rules, service agreements, etc. UCEPROTECT even offers a very blatant disclaimer on their site.
> USING OUR BLACKLISTS, YOU ARE AT YOUR OWN RISK. WE WILL NOT BE HELD RESPONSIBLE FOR ANY DAMAGE OR PROFIT OR LOSSES. SHOULD THIS BE UNACCEPTABLE TO YOU, YOU ARE NOT ALLOWED TO USE OUR BLACKLISTS.
> The UCEPROTECT list is a literal text file that gets ingested by the mail provider.
I don't know whether that is true; but I do know that it is usually used as a DNSBL - a DNS lookup for an IP address, that answers whether that address is or is not in the list. In general, mail providers do not "ingest" entire blocklists.
> responsible for maintaining their own list in a way that complies with international laws [etc.]
Actually, anyone can publish a list of anything; unless that publication amounts to a contract (statement of purpose, assertion of fitness for purpose), then I'm not aware of any international "law" that says you can't put anything you like in a publicly-acccessible list. And anyway, there's no contract without an exchange of considerations - baiscally, you have to cough-up if you want to assert a contract.
> even offers a very blatant disclaimer on their site
Have you ever read a FOSS licence? UCEPROTECT are simply stating that as a free user, you can't hold them responsible for the accuracy of their lists; in the same way that FOSS authors disclaim responsibility for fitness-for-purpose.
> In general, mail providers do not "ingest" entire blocklists.
Completely false. They ingest lists[1] into their own local daemon. There is not a major mail provider on the planet who is querying a third party DNSBL every time an email comes in. It's also more than a little ridiculous that you admitted to not knowing if something were true, but then decided to confidently (and incorrectly) explain it anyway, rather than taking ten seconds to look it up.
> Actually, anyone can publish a list of anything; unless that publication amounts to a contract (statement of purpose, assertion of fitness for purpose), then I'm not aware of any international "law" that says you can't put anything you like in a publicly-acccessible list.
Both wrong and irrelevant. I explained how Comcast is responsible for maintaining their own blacklist, which is NOT public, and as an ISP they are absolutely under all sorts of legal regulations, in addition to having service agreements with end users and other network providers. Yet, you responded with a "point" about how UCEPROTECT (a completely different company) is allowed to post whatever they want in a text file. Do you seriously not see the disconnect between what people are actually saying and how you interpret them? Because it's beyond frustrating.
> Have you ever read a FOSS licence? UCEPROTECT are simply stating that as a free user, you can't hold them responsible for the accuracy of their lists
Entirely irrelevant. The UCEPROTECT disclaimer is not for the end user -- it's for the ISPs. If someone sues Comcast, they're not trying to hold UCEPROTECT responsible for creating a text file, they're trying to hold Comcast responsible for acting in bad faith by not allowing reasonable recourse for removal from Comcast's copy of the text file. So, the end user in this situation is not a "free user of UCEPROTECT," but instead a paid user of a network provider with a service agreement in place.
> If you are providing an email service and your customers are not receiving the emails they're expecting, all because you refused to acknowledge a removal request,
And on this I concur with you, because if the sender of the email is actually sending spam and has ended up on some smtp-receiving deny lists for well founded reasons, they are indeed failing their customers. Failing them through their own lack of procedure and policy, such as not having a click to remove link in the mail, and not removing addresses/putting addresses into a scrub-before-sending list.
If that is the case, they have nobody to blame but themselves for that and should not be sending legal demands to other peoples' ISPs such as at&t or Comcast.
You're assuming facts not in evidence, such as that we don't have policies for unsubscribing, or that we're sending spam in the first place. To be clear, we've never complained to anyone except on behalf of customers who complained to us that they weren't getting our mail.
It’s pretty easy to envision a situation where a lawyer sends a quite friendly and factual email to a company, that is literally identical to the one the IT head would have sent, but because it’s coming from a lawyer the recipient uses completely different internal routing to process the request. So someone actually takes the request seriously.
Seems both plausible and a reasonable thing to do for a company large enough to have a legal department.
People pay attention to lawyer letters. You’ve pretty much confirmed as much by noting that letters from a lawyer are so concerning to you that the mere mention of one makes you assume it’s a threat.
If you got a letter from an attorney asking for something nicely, and it was a reasonable request you would automatically reply “SUE ME” just on principle? What’s the principle?
In the American legal system, if somebody spends the money to take the time to have their lawyer hand craft and send me a letter about something such as this, I'm going to take it as a threat whether or not it specifically contains one.
The implication is that if you do not do whatever is demanded in the letter, the next step will be the client of said lawyer escalating the situation to paying their lawyer to actually sue you.
I mean, not really. In the world of companies talking to other companies lawyers are involved all the time. In some scenarios (real estate transactions, or M&A, for example) it would just be completely routine for two companies on exceedingly good terms to communicate back and forth via attorneys.
Your main premise seems to be that the recipient should take all this very personally. But it’s not personal, these are businesses, discussing a business related topic.
The VAST majority of business to business communications that involve attorneys don’t go anywhere near an actual filed lawsuit.
An intentionally initiated transaction where everyone already knows each other, and knows in advance that lawyers will be involved (as you say, m&a, real estate, etc) is a very different thing than receiving a demand letter from a previously unknown party out of the blue.
Like I said in my original reply here, we are talking about sending threats to third-party isps, with which the originator of the smtp traffic has no existing business or contractual relationship.
I made all kinds of good faith efforts personally to contact someone at AT&T through their forms and phone numbers, without ever even speaking to anyone who could resolve the problem if they wanted to. Lacking any other way to get through to them, legal seemed like the only recourse.
I would think that if you were the ISP owner blocking our mail to your customer, a single phone call from the customer or from us would convince you that our IPs were not a spam threat. That's certainly the way I prefer to deal with anything. But when dealing with something as monolithic and totally deaf as AT&T, lawyers were the only way.
I would think an attorney wouldn’t get involved until other avenues were exhausted. As alluded to in the comment. If it’s out of the blue, check your spam folder.
Yes that’s the idea. They’ve spent a bit of money to make a sincere and well-founded request that will cause you to consult your own lawyer who will explain to you why you’re probably doing something wrong and need to do what the letter asks of you.
The fact that you would automatically react in this way is itself a reason to send it to the lawyers and have it go through a different department and chain of command.
Make it a legal and business decision instead of a technical one. And find an audience that knows the practical benefits of quick resolutions to reasonable requests.
And I say this as a person that would likely respond in a similar punitive and/or principled way to what you described upthread. Which is exactly why I leave it to the lawyers.
>> Make it a legal and business decision instead of a technical one.
I'd go further and say that choosing to unblock some IP addresses out of a block you don't like is always going to be a business decision. The whole rationale for blocking in the first place was that a sender is untrustworthy.
Untrustworthy senders generally don't have a team of lawyers, and if they do (and they're actually sending spam) you can show what spam they sent and keep them blocked.
All the legal letter does is force someone who's otherwise too busy to look at this particular case for the 5 seconds it takes to determine that it's not a threat. Sad to say that being head of IT for a company for a decade doesn't get you the same level of respect; it might if they even bothered to open your email, but there's something about postal and legal letterhead.
But lawyers don’t cost nearly as much outside the US. We moved from the US to the NL a few years ago. Legal representation is pennies compared to the US. So just because it’s expensive where you are, doesn’t mean it is everywhere else.
It doesn’t mean they are thinking to sue you. Not even close. When you actually get served, that’s when they’re thinking about it, because actually serving you is a few bits of paper and requires a few brain cells. Sending you what amounts to a form letter? No brain cells required.
The rationale for involving legal is to place some accountability and consequences where they belong.
Currently, countless people essentially commit countless abuses for free because the actor is hidden behind a machine or a process. But somewhere it's a humans decision to institute an abusive protocol, and it seems pretty fair fo me to make that human accountable for their action. Not just email but all kinds of things.
You are probably merely a dick but still a legal dick if you wantonly block email for yourself. But the second you are responsible for even one other person's correspondence reaching them, I say you should be legally culpable for any failure to deliver.
I question whether you or what other percentage of the commenters in this thread represent any specific ASN with its own IP space that it cares about keeping clean, and have bgp relationships with other ISPs.
Or whether they're actually end users only.
Have you actually encountered this problem as a service provider in the past and implemented solutions to it, or are you just sharing your opinion as a possibly-frustrated end user of email?
I avoided the problem by only ever sending email on behalf of customers, not receiving. IE, saas app can send out quotes and invoices and things but any replies go to the end users own address not any of our servers, and didn't host our own email.
What I didn't do was ignore the problem or think there was no problem. Things still got blocked elsewhere but I (personally or my company) didn't do it.
I recognized that handling someone elses correspondence is a non-trivial responsibility. Yes the job is hard, and so you either decide that is your whole job, or you outsource it to someone whos whole job it is. You don't just do a poor job because it's hard and not your all day every day job.
> I say you should be legally culpable for any failure to deliver.
So you think an MSP's advertised policy should be "We guarantee that anything sent to you will be delivered, including spam"? That no MSP should provide spam-filtering, at risk of legal culpability?
If that's not what you mean, then presumably you are requiring all MSPs to block only spam, and to deliver all legitimate email. But that is impossible, because nobody has figured out a way of reliably distinguishing spam from ham.
If I received such a letter from your legal department, I would laugh, and reply: "I am awaiting your writ." If I got the note from a recipient or their postmaster, I'd be much more inclined to try to help, but I never try to negotiate with lawyers brandishing threats.
The MSP's policy can be whatever they want as long as it IS advertised.
If they say "We drop 10% of messages at random, and you knowingly choose to take that, then that is just a stupid arrangement you should never agree to, but they aren't doing something unexpected.
I decline to believe you are as stupid as that remark.
Good faith best effort is perfectly reasonable. Not knowingly and intentionaly discarding mail is all that's required.
If you're the mailman, you do not have to garantee that you will never lose a single letter in a car accident.
But you can certainly garantee, absolutely, that you never go through the bag and throw away all the spam and sometimes mistake a legit letter from a lawyer for lawyer spam.
You can ceetainly garantee, absolutely, that you never apply utterly thoughtless rules like "we got these scam letters from Nigeria so now we just throw away anything from Nigeria."
You should absolutely be responsible for other peoples stuff that is in your hands while it is in your hands. It's not yours to dispose of, even when part of your explicit job is to filter. If that sounds onerous, that's why you get paid money for the responsibility. If it's too hard to bear this responsibility properly, then you have no business doing that job. Do the job right or don't do the job. There is nothing unreasonable about those two choices. It is not at all required to do the job, but poorly or carelessly.
> The MSP's policy can be whatever they want as long as it IS advertised.
No.
Suppose the MSP uses bayesian filtering? How would one go about advertising a policy that depends on bayesian filtering? You'd have to publish the contents of your filter table. The only people who could benefit from that would be spammers, who could use the data to customise their spam.
In general, telling the world what your filtering policies are is just going to cause spammers to try to sidestep your policies. The only policy that you maybe ought to publish would be along the lines of "We filter out spam; that sometimes results in false positives. Sorry."
Did you read the whole article? The OP wasn't being blacklisted by an operator. He was being blacklisted by a company that charges you a $25/month fee to not be blacklisted by lazy operators who program their systems to curl their for-profit blacklist.
People have been making legal threats at, and trying to sue, RBL operators since 1997 or so. It's a well known thing. All I say is "good luck" if you think legally threatening a maintainer of a list of IP CIDR prefixes that are used by a third party is going to solve your problems. It hasn't worked for the last 25 years and I don't see how it'll start working now.
No I think the solution is to not get a $5/month vps and expect it to have a good reputation. Maybe if you went with the $100/month hosting provider you wouldn't have needed to spend $500/month on legal threats. Internet addresses aren't fungible. It's a well known concept in telecommunications. One of the reasons why people have always paid a lot more money to have 212 numbers versus 646 numbers. It's the reason why if you want to set up a respectable business, you don't rent property in a high crime neighborhood. Browsers and email clients should ideally have more transparency about the ASNs and service providers who host the websites that people are visiting, so that everyone can develop a mental model of which ones are associated with good things, because right now the only people who are able to have any kind of awareness of this thing are operators who regularly monitor traffic.
This is completely valid, but if you go hunting for an IP block that has never been used for spam at this point you're going to be looking for a long time. It should not be the #1 consideration when choosing a hosting provider just because someone abused their IP block in the past (possibly before they even owned it). In any case, trying to run mail off a VPS would be stupid and that's not what I'm saying. I'm saying we don't all need to capitulate to paying Amazon or Google to forward our outbound mail, and it would be better if we did not, even considering the struggles attached to bucking the trend.
> Not everyone can spend $500 on lawyer billable hours per SMTP destination multiplied by N number of destinations.
You likely wouldn't do that - just get a template version that gets reused, just like you pay once for a contract / t&c you reuse with multiple parties.
> You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs?
If the remote mailserver is explicitly bouncing your mails, it's not a network engineering problem.
If you're able to complete a TCP 3 way handshake, it's not a network engineering problem.
If you're being prevented from completing a TCP 3 way handshake because of a filter list on the MX server, it's not a network engineering problem. Admittedly, you can't tell that one without asking a network engineer (on the remote network) to validate that for you, or asking the application owner to advise.
Contrary to popular advice (thanks MSFT), you probably shouldn't contact your network administrator. We probably have priv 15, but we dont necessarily have root.
A year later I bought a domain that had expired under my country TLD. It turns out that domain for some reason was previously added to that list.
Now, as you can see, the man behind that Github repo has decided to archive that repo and therefore make it read only.
As you can tell from both pull requests and issues on that repo, people has asked him to remote legitimate domains (e.g. *.urbandictionary.com). But those calls remain unanswered. It is fruitless to contact the author.
So this random dude causes real problems for legitimate business and individuals and we should just accept it?
Obviously he doesn't act on friendly geek requests. It seems lawyering up would be the only recourse in this situation. I find it analogous to somebody standing on a soap box in a village and announcing: "Don't trust James. Don't trust Mary either. There's problems with Charles" and James, Mary, and Charles have no way of stopping his libel.
I don't have the funds for legal action, but it is obviously wrong that he can announce "these domains are bad" and offer no way of fixing mistakes. He should take down the repo, but oh that sweet sweet Github karma probably discourages him from doing so.
> I find it analogous to somebody standing on a soap box in a village and announcing: "Don't trust James. Don't trust Mary either.
You're speaking of some random domain-list on github - that isn't even maintained? Taken from someone's private pihole?
Anyone using a list of that kind to block is simply incompetent. Even as part of a scoring system, it's pretty silly. Before adding a blocklist, a postmaster needs to familiarize herself with the list's policies. Are list entries aged-out? How quickly? Do they use spamtraps, or user-reports? Or is it just the whim of the list-maintainer? Do they block individual addresses, whole domains, or entire allocations?
> So this random dude causes real problems for legitimate business and individuals and we should just accept it?
So you're having problems sending mail to a domain where the postmaster cares more about rejecting spam than she does about receiving legitimate email. That's a matter for your recipient to take up with their MSP. And if the recipient wants to receive mail from small-time domains, they need to accept that they're going to receive some spam as well; but maybe they need to switch to an MSP that only rejects on strong evidence.
My point is that it's your recipient's choice to use an MSP that blocks using some crazy list they found on github.
Some postmasters will block everything from selected countries; at one time I would block everything from Romania, because none of my users had correspondents in Romania, and email from Romania at that time was 100% spam. But I wasn't providing service to the public. I knew all my users.
Different MTAs have different users, and different patterns of abusive email. So if you want to use a custom blocklist, make your own, based on your own incoming spam (and then you can honour removal requests yourself). Otherwise use a public blocklist, based on multiple spamtraps in multiple ISPs.
So yes, you should just accept it. You don't have a right to have mail delivered by any MSP you send to; they're private organisations or individuals, and they're entitled to determine what their own policies are. In the world of email, nobody is entitled to protection from the foolishness of others.
>> per SMTP destination multiplied by N number of destinations
The number of people using anything other than google, apple, microsoft or yahoo for personal email these days is vanishingly small; and of the remainder, almost all of them are on major broadband providers. If you were some local ISP blocking us, that wouldn't be worth the trouble of sending a legal letter. I'd just tell that particular customer to contact you and ask why you weren't delivering our mail.
This only leaves other corporations that provide their own mail services to people who work there, and if those people want to get their mail they can call IT.
1. Host your mx somewhere that isn't on any blacklists. This means a small to medium sized isp, where you can directly contact the people who run the core network operations there, and who truly do care about kicking off abusive other customers very quickly. Ideally I would go with an ISP in your own region and home business area. Best chances of success if it's a hosting ISP where random customers cannot sign up online with just a name and a credit card, but it's more of a "contact us for a custom price quotation for your colocation needs" type of hosting operation.
2. Possibly run all your outbound smtp through a trusted third party service that you pay for such relay. Leaves a bad taste in my mouth but that's where we are at in 2022.
3. Be absolutely certain that your own smtp, spf, dkim, dmarc configuration is flawless and you've never been a source of spam.
Host your mx somewhere that isn't on any blacklists. This means a small to medium sized isp, where you can directly contact the people who run the core network operations there, and who truly do care about kicking off abusive other customers very quickly. Ideally I would go with an ISP in your own region and home business area.
That's a nice idea. In fact, it's what my businesses have done for years. I have personally met several of the senior staff at the service I use and I know that they are both technically excellent and very serious about preventing abuse on their network.
We haven't been able to deliver mails to customers at certain mail services for years. 100% blackholed every time. And it's the same usual suspects that others have mentioned in this very discussion.
The problem is that the hosting service we use has many customers. Occasionally one of those customers makes a mistake and one of their systems gets compromised for a short time, until they or the hosting service detects the unusual traffic and intervenes. And that's enough to get someone's IP block on a blacklist it will evidently never leave.
This is of course absolutely no different to any hosting service like AWS or Azure, except that no-one is going to blanket block all servers from an organisation of that scale even though the exact same problems happen there too.
The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness. It's usually wrong to assume that everything in a /24 is controlled by one botnet, and it's not that hard to check whether it was just one or two particular addresses that were compromised. But if you're an ISP and you want to take the nuclear option to every spam threat, at least be willing to listen to your own customers when they complain that they're expecting mail, and there's absolutely NO reason to assign "group punishment" to everyone using the same service provider. I think the thought was that that would make service providers more accountable, but it's totally unfair to use everyday customers as pawns in a war between ISPs.
> The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness.
Yes, in that situation it's laziness, but on the part of your ISP. ISPs already spend huge amounts of time and money dealing with spam, hacking attempts, phishing attacks, etc. If your ISP is irresponsible and isn't doing their job keeping those things from leaving their network then your ISP is gong to find their entire IP space blocked and that's 100% reasonable. Why should we ever accept traffic from an ISP that refuses to keep their corner of the network clean when it's just going to cause problems for us and our users?
That's the situation for every ISP on the internet. Keep your users in line, keep trash off your network or else no one is going to accept traffic from you. You could call it laziness, but it simply isn't worth it. We'd rather spend our time cleaning up abuse on our own network and working with ISPs who are doing their job than dealing with the problems we get from bad actors.
If you own an IP surrounded by a bunch of spammers and it's giving you trouble step 1 should be to contact your ISP and tell them to get their shit in order or you'll take your business to an ISP who does their job. Step two is to switch to a new ISP if they don't. No one has the right to force us to accept traffic from anyone else. It's every ISPs responsibility to make sure the traffic leaving their network isn't more trouble than it's worth. Good ISPs are rewarded because users will give them their business and stay and bad ISPs are punished because users will drop their service when they see they are blocked.
The goal isn't to punish the poor sucker who signed on with an irresponsible host, but to cut down on the number of bad ISPs on the internet and the amount of work we have to deal with coming from them.
That seems a lot of rationalisation for a situation where a genuine sender on another system sends legitimate mail to a genuine recipient on your system, that mail is not properly delivered, and it's your fault.
There is a reason that collective punishment is considered immoral by civilised cultures. It hurts the innocent and often fails to achieve its original goal anyway.
> There is a reason that collective punishment is considered immoral by civilised cultures
Rejecting email submissions isn't punishment, collective or otherwise. It's something you have to do if you run a mailserver. In the same way, I'm not punishing trespassers if I secure my front-door with a deadlock.
> That seems a lot of rationalisation for a situation where a genuine sender on another system sends legitimate mail to a genuine recipient on your system, that mail is not properly delivered, and it's your fault.
It's how the internet stays functional. If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture. Bad actors have been ostracized from communities for as long communities have existed. If you want reliable service, choose a responsible ISP. Blacklists have enabled the internet and email to remain useful for most users most of the time. Without them, email wouldn't be usable. If you have a better solution for spam, the whole world would love to hear it.
If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture.
1. It's way more than 0.001%. Like, several orders of magnitude more.
2. What overwhelming amounts of spam/attacks? Those of us using traditional mail systems with traditional spam filtering seem to be doing OK not getting overwhelmed by incoming spam without the kind of "help" you advocate.
3. You don't get to decide what's acceptable to everyone else. Except that apparently you've decided you do, which is why regulation is needed to remove that ability from service providers who can't do their jobs properly and hurt others as a result.
If you want reliable service, choose a responsible ISP.
This is a poor argument. Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them. The decent ones will identify the problem and block it reasonably quickly, but there is plenty of evidence that they can still be blacklisted and it can still be difficult or impossible to get removed from those blacklists again after the problem is fixed.
The kind of policy you advocate punishes small ISPs just for being ISPs. I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
Or maybe the rest of us should apply the same policy to organisations that do what you advocate. If they won't deliver mail reliably, we won't forward any mail to them at all, so their mail service becomes useless. Except of course it's some of the biggest mail services that do this, so just like no-one's going to block incoming mail from AWS or MailChimp, no-one's going to block outgoing mail to Google or Microsoft.
If you have a better solution for spam, the whole world would love to hear it.
Block actual spam sources and provide a reasonable method for removing blocks that are no longer necessary. Don't carpet bomb whole chunks of the Internet just because there are a few bad actors around. Don't fire and forget. It's really not difficult and plenty of small organisations operate just fine on this basis every day.
The majority of all email has been spam, for more than a decade.
> You don't get to decide what's acceptable to everyone else
If you operate a mailserver, you do actually get to decide what kind of stuff you are willing to accept, and from who. "Everybody else" does not get an automatic right to inject data into my computer.
> The kind of policy you advocate punishes small ISPs
Not at all (perhaps you meant MSPs?) Blocklisting Google was a reasonable policy, at one time. Blocklisting MailChimp is perfectly reasonable now.
> Block actual spam sources
Of course, good plan. Unless the sender's ISP is in the habit of moving spammers from one address to another, so they can evade blocks. Then you have to block the ISP, or eat their spam.
At times that number has been even higher with over 90% of all messages sent over the internet being spam. If 90% of all the messages in your inbox were spam how long would you continue to use it? Email systems can't bear the costs that spam forces on them. Even with tools like blacklisting which you think shouldn't exist that cost is measured in tens of billions annually. source: https://www.aeaweb.org/articles?id=10.1257/jep.26.3.87
If not for the ability to filter common sources of spam, email would never have survived as a viable means of communication.
> It's way more than 0.001%. Like, several orders of magnitude more.
Whatever the actual number, it's clearly acceptable to us because blocking irresponsible networks is standard practice. We depend on it.
> What overwhelming amounts of spa
Again, 80-90% of all mail is spam, costing billions. If you're able to run a mail system without blacklists that's great for you, but it clearly doesn't work for everyone.
> You don't get to decide what's acceptable to everyone else.
That's the beauty of the internet. I don't have the power to force an abusive network to do their job and prevent spam from leaving their network and that abusive network can't force me to accept mail from them. No one can force anyone to do anything. All we have is a loose set of standards and expectations and it's up to each network to decide what to accept or not based on how well those standards and expectations are followed.
> Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them.
A responsible ISP identifies those users and prevents them continuing to cause problems. If they refuse to do that their reputation suffers and they will get blocked. If they do their job too slowly or too poorly they will be blocked.
Is it possible for a responsible ISP to end up on blacklists? Yes, it is, and there are blacklists that don't maintain their lists well. That's fine too because no one is forced to use them. It's still the case that every network has the choice of what blacklists they will or won't use and how they use them. They can whitelist blacklisted IPs they decide to trust and they can use blacklists to greylist instead of block.
> The kind of policy you advocate punishes small ISPs just for being ISPs
Nope. Even small very ISPs can staff their internet abuse departments adequately and implement anti-spam technologies to prevent their IP space from becoming a safe haven for hackers and spammers. If they choose not to do that they will and should be blocked.
> I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
I'll agree that there are problems when certain services (either cloud providers or mail providers like Gmail) become "too big to blacklist". We've had that problem with AOL and we have it now with Google. Personally, I'd prefer to hold them to the same standards as everyone else, but the problem of the largest players throwing their weight around giving them unfair advantages exists in every industry and until someone comes up with a solution for it, we're all just stuck playing along.
> Block actual spam sources
If your ISP is a safe haven for spammers and hackers their IP space is the spam source.
> provide a reasonable method for removing blocks that are no longer necessary.
So your alternative to blacklists is just more blacklists that are run better? I think everyone who depends on blacklists would like those blacklists to be better at detecting spam sources and better at clearing unnecessary listings. The good news is that badly run blacklists don't tend to get widely adopted because they cause more trouble for ISPs than they are worth.
If some network won't accept your mail and you're convinced that your ISP is acting responsibility and that it's the blacklist that's wrong, you can have the person you're trying to reach contact their ISP to get your mail server whitelisted. If an ISP sees that a blacklist they use is catching too many messages that it shouldn't they'll adjust their thresholds or stop using that list.
It's not a perfect system, but it's the best one we have.
Did you actually read that, and the sources it cites, before posting it? If you had you might have noticed that it's full of the worst kind of junk stats. Several of the sources cited, the ones that supposedly support your arguments here, don't even say what the piece you linked claims. They literally have completely different numbers. Not that it matters since there is no indication of methodology used and the exact figures are clearly impossible for anyone to measure accurately. Some of the other "sources" are just links to organisation home pages without identifying any specific research or analysis at all.
If 90% of all the messages in your inbox were spam how long would you continue to use it?
As someone old enough to remember the time when that was actually the case, obviously we managed. But this is distorting the argument again because you are implying a false dichotomy where the alternative to overly aggressive blacklisting policies such as you advocate is all of the spam reaching our inboxes. Clearly that is not realistic as less aggressive defences are still highly effective and have consistently been so for a long time.
No one can force anyone to do anything.
Really? Then where can I sign up for a mail service that will reliably deliver both my incoming and outgoing legitimate messages without undue monitoring or interference with my own business? I contend that possibly no such service currently exists.
Personally, I'd prefer to hold them to the same standards as everyone else, but the problem of the largest players throwing their weight around giving them unfair advantages exists in every industry and until someone comes up with a solution for it, we're all just stuck playing along.
Which is exactly why some of us are in favour of statutory regulation to compel anyone participating in such an important technological ecosystem to be a good citizen.
So your alternative to blacklists is just more blacklists that are run better?
I don't believe I have ever suggested anywhere in this discussion that using blacklists to block traffic from proven spam sources was unfair or inappropriate. My objection, which seems to be in line with the submitted article, is to big mail services that think spraying fire into a crowd of 250 indefinitely because there was once one bad person there is a reasonable response to the problem. There is huge collateral damage being caused and the defenders of this policy are trying to sweep it under the carpet and use highly debatable arguments of necessity to justify their damaging policies.
This is not the best system we have. That's the point being made here.
> As someone old enough to remember the time when that was actually the case, obviously we managed.
I'm also old enough to remember that and we managed by blocking huge amounts of IP space. Even massively popular services like AOL have blocked the IP space of entire ISPs or entire countries from being able to send them email. Eventually spam filtering improved, things like SMTP auth, DKIM etc caught on and wide range blocking could be scaled back somewhat, but I doubt it will ever go away entirely.
> Really? Then where can I sign up for a mail service that will reliably deliver both my incoming and outgoing legitimate messages without undue monitoring or interference with my own business?
Use your own servers and you can do whatever you want. Again, you can't force others to accept email from your mail servers, but you can choose to accept or reject whatever you want from others. No one can stop you from sending mail from one mail server you own to another mail server you own.
> Which is exactly why some of us are in favour of statutory regulation to compel anyone participating in such an important technological ecosystem to be a good citizen.
You can't really regulate the internet. If you could enforce regulations on a global network made up of discrete but interconnected networks we could just make spam, phishing, and hacking illegal on the internet, enforce that law/regulation and there would be zero need for blacklists. Because laws and regulations don't work on the internet we instead have to come up with blacklists, filtering technology, and other tricks to keep the internet even semi-functional.
> My objection, which seems to be in line with the submitted article, is to big mail services that think spraying fire into a crowd of 250 indefinitely because there was once one bad person there is a reasonable response to the problem
It's the only one that works. I've seen with my own eyes ISPs who didn't care enough to invest at all in abuse handling, but were forced to because of being blacklisted and in order to keep their customers they had to clean up their network, pay attention to abuse notices, participate in feedback loops, and slowly rebuild and maintain their reputation as responsible network operators.
If you limit blocks to individual IP addresses than spammers just cycle IP addresses. ISPs that ignore anything sent to their abuse@ address (if they even have one) never have any pressure to invest in preventing spam and can just keep accepting money from spammers and hackers and give them new IPs whenever they need to.
IPv6 makes the problem much much worse since a single spammer would get a huge amount of IPs to burn through before they have to bother their ISP about it. Blacklists themselves could become so massive and cumbersome that restricting larger and larger ranges may be the only option.
Can you imagine what would happen if we applied your argument to other important communications channels like postal mail or telephone calls? Sorry, someone in your old friend's city was using a robodialler so now none of the local phone service providers available to you will accept calls from anyone in that area code.
We absolutely can regulate the Internet on this kind of issue. We don't have to regulate everywhere in the world to make a big improvement, just businesses above a certain size that operate a commercial email service. If our governments can effectively lean on social networks enough that they add warnings to potentially misleading comments about science, they can lean on email services to do better with this problem. They only difference is that there is an obvious and unambiguous way the mail services could do a better job.
And again, just to be crystal clear, I am not arguing for giving real spammers a free pass. I am only arguing for credible, realistic measures to try to avoid the huge numbers of false positives we get from mail filtering today.
> Can you imagine what would happen if we applied your argument to other important communications channels like postal mail or telephone calls?
The only reason we don't is because unlike email, it's the sender who pays not the receiver. Telecoms do monitor and block outbound international calls if the connection times are excessive, if they occur at unusual hours, or if they going to certain "blacklisted" countries where phone fraud is common. They do it because hackers will break into a business's PBX and use it to place a bunch of international calls and the business suddenly gets a massive phone bill. They call their phone company about the changes, the phone company waves the changes (once) but that leaves the phone company on the hook for them. When false positives happen, the business has to call into the phone company and explain the calls were legit and they will be whitelisted and similar outbound calls will be allowed going forward.
I wouldn't oppose using regulation in the US against US based mail services if it meant forcing them to do a better job preventing spam from leaving their networks, but I'd be hesitant to support legislation forcing them to accept more spam. Maybe the largest ones could be pressured to invest more money in handling the influx of spam after they accept it, but I'm guessing there would be costs to consumers such as long delays in delivery, or "free" services like Gmail suddenly requiring payment or closing their services for good. At the ISP I work for now we stopped hosting our own mail servers and outsourced email services to a third party because spam filtering was too expensive and time consuming, and now we're looking at possibly no longer offering an email product at all and telling all of our customers to migrate to services like gmail and yahoo. Killing our email service today would eliminate a lot of problems in terms of help desk calls, phishing attacks, and spam problems. Make it too much harder for people to provide email service and there may only be giant providers left.
Other guy sounds like a giant dick-wad - we should not be wholesale blocking IP ranges without recourse to "unblock".
Whatever the other guy thinks about it being "necessary" or whatever, there is not commonly a way for a user to whitelist a service. And services providing email dont normally take that sort of signal into account, either.
Once you are operating a large system that is used by many people, you become a public utility - furthermore, at that scale we can generally find where you live and come lock you up. This kind of thing is 100% regulatable.
Either let users choose what mail they receive, or implement regulation forcing compliance. If that doesnt happen, and you snub my lawyer like the irresponsible mega corp you probably are, guess thats one more reason for me to polish off my shotgun and takeout the dickwads running the megadoom corp.
> The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness.
It's not laziness; the intention of collateral blocklisting, as with UCEPROTECT L2 and L3, is punitive. It's to incentivize the sending MSP to remove their spammer (or move them to address-space where they can be blocked without causing collateral damage).
It's not laziness; the intention of collateral blocklisting, as with UCEPROTECT L2 and L3, is punitive.
Unfortunately the people it punishes are the legitimate users of both systems who only wanted the system to do its job and let them communicate. The bad actors will just move on and abuse another system instead.
> who only wanted the system to do its job and let them communicate
"The system" you are referring to consists of a bunch of private networks. Your opinion about what the job of those networks is may not coincide with the opinions of the operators of those networks.
Email is not a public service, and there is no entitlement to send whatever "vital business communications" you like to anyone you want. It's not even reasonable to require a postmaster to state what their rejection policy is; that would just tell spammers what they have to do to evade your blocks.
If email doesn't work for your business, then switch to another channel, such as huge billboards or whatever. Ranting about blocklists isn't going to help, people have been doing that for two decades.
Could you get a lawyer to draft you one template that looks scary while also not leading to much follow up unless you really want to be whitelisted by that particular entity?
I also think that the likelihood of success in sending legal threats to somebody that demand they accept your SMTP traffic will not stand up in court, if you ever escalated it that far.
As somebody who runs postfix MX on the receiving side of things, I can guarantee you that the day I receive a legal threat from some unknown third party with which I don't have a pre-existing business/contract relationship, demanding that I accept their email, is the day that I blacklist their entire organization and tell them "okay, I'll await service of your statement of claim".
You actually think that the best answer to a network engineering problem is to make legal threats at third party ISPs? Companies with which you don't have a signed service order contract and/or master services agreement?
You say you're a mid sized company. I think you're running a huge legal risk of angering a Comcast or AT&T size entity that has much deeper pockets and legal resources than you. The day that one of those giants calls you out on your bluff is going to be very expensive.
On an ISP-to-ISP relationship level, this is not how you solve SMTP flow traffic problems. I can tell you that if I went to a NANOG conference representing my AS and proudly told other people "oh yeah, we've started sending threats from our lawyers to $OTHERISP1 and $OTHERISP2 because they won't take our mail traffic", that I would quickly be treated as a pariah.