Email sent to a friend who was considering using a blacklist to reduce the amount of spam they receive. Having worked for several ISP's and run large mail farms here's my personal take on the whole thing.
See also: AdamShand/2003-05-07, ChallengeResponseSystemsConsideredHarmful
Just personal experience/preferences as a long time email admin. Blacklists are evil and I *LOATH* them, please don't ever use them or encourage anyone else to use them. Unless you are AOL (who recieves something like a terabyte of spam a day) blacklists cause more problems and cost more money (in support and trouble shooting) then spam.
In that vein things like TMDA can be useful personal tools but i don't like them as general policies. In other words, use them to sort things into folders but don't bounce messages based on it. I also consider them rude, if I get a message from someone I tried to email with a message that says "XXX uses TMDA and you have to respond within X hours with X code blah", I just delete the message and don't bother emailing them. Why should the onus of communication be on the sender?
Personally I *only* use statistical systems and I've been very happy. My combination of razor and mozilla's bayesian filters catches about 99% of my spam. Razor alone cathes about 40-60% of my spam which was enough to make my spam bearable ... mozilla has made spam mostly disappear.
Razor occasionally catches a mailing list message as spam (it's happened maybe 3 or 4 times in the last 8 months) but i can live with that. Mozilla has only ever tag's things as spam that I mistakenly tag as spam, and usually that's cron messages.
I can't use things like spamassassin because I really do get messages from people I don't know with weird and stupid subjects (people in asia, africa etc who want to start cwn's).
And a response to a friends response to the above:
> Professionally I have considered using blacklists, I am curious to hear > a bit more on why you loath them so.
For example. Pixelworks has at a couple times in it's life had open relays (mostly due to aquisitions which had crappy networks), which have been listed. Further our /24 is part of a block of AT&T (12.154.*.*) which black lists like to block (through no fault of ours).
Everytime we have an employee complain about mail prolems we have to go through the dance of finding out if we've been blocked somewhere and getting ourselves unlisted.
These days, one of the troubleshooting steps you have to go through when you have mail problems is trying to figure out if anyone is blocking you cause they think your a spammer. Way to create centralized points of brokeness for a decetralized service Batman!
> I have at this point never used them but they seem to offer a little > light at the end of tunnel for my poor users that get swamped with > spam. Several of my issues are that I dont have the luxury to get
From a users point of view I don't like blacklists because false positives are inevitable, that the false positives are indescriminate.
For example, with razor I do get occasional false positives but it'll never be a message sent directly to me, because no one has ever seen that message before. You only get false positives on messages sent to lists.
Blacklists are actually more likely to block users emails then list email because popular lists are generally on up and up servers while lots of people have legitimate, but dodgy, mail servers.
> newer more powerful email clients on all my users machines (otherwise > they would all be running OS X and its default mail reader), my > environment requires that I implement solutions at the server level, > educating users has proven to be less then stelar. Anyway, I have > little to no experience with blacklists other then keeping my mail > servers out of them. Input appreciated.
My recommendation would be to setup a server with spamassassin and set it up to not use blacklists but to use razor and it's other heuristics.
When it thinks a message is spam just have it add a header or tag the subject line. Then employee's can use normal filter rules to catch suspected spam and dump it into a SPAM folder which they can review manually occasionally. If you have an IMAP server then you can even do server side filtering for them into the SPAM folder.
The big advatage of this is that the rules remove spam into a seperate folder, but if something does go wrong and they are expecting a message they can go dig throug their SPAM folder to look for it and still get it without assistance from you or having to ask a client to send it to their hotmail/yahoo account.
Date: Sat, 17 May 2003 16:29:29 -0700 To: politech@politechbot.com From: Bob K <bk@msgbase.com> Subject: A "Nice Blacklist?" No such thing.
Declan: This may or may not be appropriate for the list, but I thought I would write it anyway. I've tried to keep it short.
Our ISP has been plagued by SPAM. I don't know of any ISP that hasn't been. But we recently shut off our use of Realtime Blackhole systems in favor of in-house SPAM-control. Instead of an effectiveness of 60% or so, we are now trapping more than 90% of unwanted mails, and in a way that our end users have total control.
We shut off the RBLs because we incurred a huge costs as a result of our support for them. When the RBLs began to support the blocking of entire Autonomous Systems instead of targeting SPAMmers directly, they lost my support. I had a carrier whose AS numbers were put into the RBL. Like a good anti-SPAMmer I rejected the carrier and moved to another. My ISP is fairly large, and including customer support costs the carrier change cost my company around $35,000.
Within 90 days, my new carrier was placed into the RBL system. Again I was mandated by the RBL operators to switch carriers. This time I refused to be extorted by a handful of people with way more power than they deserve. They misuse their power and do so without conscience. I say this because their use of the term "collateral damage" hides a tidal wave of harm to innocent ISPs and their more innocent customers. Some of the RBL operators have become so obsessed with their tools that they have started to create more harm to Internet users than the SPAMmers they want to protect those users from. A SPAMmer clots mailboxes. An agenda operated RBL takes the mailbox away. Plus, I have run out of choices for carriers. My area only has a few that support it, and all of them are AS-wide being blocked. So I tend now to view RBLs with disdain and disrespect. I do so because they don't respect the needs of honest people and honest companies who need unfettered Internet mail.
All of the anti-SPAM efforts have created a lost perspective I think. That is, people should control their own mailbox. A system that makes arbitrary decisions about what content should and shouldn't be permitted is a loss of freedom of choice. So our new system affords them as much or as little control over their SPAM as they would like. Plus, it gives them the opportunity to retrieve messages they filtered by mistake. An RBL will drop the communication with a SPAM source. The new system accepts the mail and places it in a temporary holding area where it may be easily retrieved.
Yes, this is taking resources to do, but I knew this perspective would entail some costs. Frankly, the cost of some disk space and processing power is a lot less expensive than having to change carriers every time an RBL invokes personal vendetta. It also doesn't leave me wondering if the next carrier will end up in the RBL forcing me to change again. And again...
So, there is no such thing as a nice RBL. They are more harmful than helpful, they are less efficient than internal methods, and they take freedom of choice away from ISPs and more importantly, their customers. As long as legitimate RBLs maintain their support for those RBLS that have gone rogue, or are completely inept, they are more criminal or problematic than SPAM.
Date: Sun, 15 Jun 2003 12:19:54 -0700 (PDT) From: Joe St Sauver <JOE@OREGON.UOREGON.EDU> Subject: Re: "Nice" Spam Filtering Respones To: declan@well.com, brad@crisp.net
Hi,
Your recent post to politech was passed along to my by a colleague... had a few comments for y'all (interposed inline below):
# A) Of the Open Relay blockers, most people seemed to like ORDB ( #http://www.ordb.org ). It scours the net looking for open relays, just #like Orbz used to do.
I would encourage you to also check out the mail-abuse.org RBL+ (see http://mail-abuse.org/rbl+ ). Not free, but pretty cheap (at for .edu/nonprofit type folks). It does a nice job on open relays and some other classes of content.
# B) Of the proxy blockers, there was no clear consensus, but #opm.blitzed.org and proxies.relays.monkeys.com seemed to be the favorites.
I've been looking at the open proxy problem a little, and I think I'd suggest Wirehub/Easynet instead. Feel free to see:
http://darkwing.uoregon.edu/~joe/open-proxies-used-to-send-spam.html
http://darkwing.uoregon.edu/~joe/proxies/ (this last link is for a paper talking about the Open Proxy Problem that I presented at the Internet2 Member Meeting in Arlington a month or two ago; PDF and PowerPoint formats are provided)
# C) Of the manual spam blockers, ones that add known spam sources manually, #the Spamhaus SBL ( http://sbl.spamhaus.org ) is by far the most recommend, #and probably fits the bill of the "nicest".
Yep, the SBL is definitely the correct choice there.
# D) There is actually one aggregate. blackholes.easynet.nl contains both a #list of open proxies and the spamhaus sbl, but not an open relay blocker. # #2) Additionally, there are two other methods for blocklists, but I'm not so #sure they fall under "nice". The first is country blockers. These block #all e-mail from the designated country. ( china.blackholes.us #korea.blackholes.us nigeria.blackholes.us ) As a business ISP, I'm not so #sure I can just go and block whole countries, but I'll wager they would #stop a good chunk of spam.
I would urge ASN-based blocks rather than country based blocks. There are definitely ISPs that don't give a damn (including Chinanet, China Netcom and Kornet, among others, see http://darkwing.uoregon.edu/~joe/spam-friendly-carriers.html ), but those ISPs don't necessarily fully occupy a given geographic region. 
#The second is blocking "dynamic" and "dialup" #IP's. Essentially, these sites try to track IP's that belong to dialup and #cable modem users. As someone who runs a home server off his cable modem, #I think this is a bad idea, but others might want to consider it.
We handle these via local /etc/mail/access rulesets -- works great for us for the most part.
#3) Lastly, everyone seems to love SpamAssassin. One person even sent me a #message ten times saying I should use SpamAssassin and probably just didn't #know how to use it properly, despite my original message stating #SpamAssassin was not what I was looking for.
I guess I must be the one exception. I discuss a number of the reasons why I'm less than enthusiastic about content based filtering as a solution at http://darkwing.uoregon.edu/~joe/spamwar/ (presented at the Northwest Academic Computing Consortia meeting a week or two ago).
#The problem is managing its #use for 20,000 people. Different people will want different levels of #SpamAssassin. I use it myself, but I have to order it in procmail #carefully, otherwise it will mark all of my nightly root-mail and other #cron jobs as spam.
delay_checks allows one to exempt certain addresses from filtering if you're using sendmail. This should be done to insure that RFC 2142-mandated addresses don't filter complaints/requests for unblocking, etc.
Regards,
Joe
Source: http://www.politechbot.com/p-04856.html