Thanks for visiting! Remember that nowadays, (most) blocklists don't really govern deliverability and inbox placement. Want to learn more about email marketing best practices, email technology, and deliverability troubleshooting? Then you'll want to check out my other site, Spam Resource.
Showing posts with label dnsbl. Show all posts
Showing posts with label dnsbl. Show all posts

Shutting Down Blocklists

As I often do, today I'm receiving reports about a DNSBL (which I've previously warned was dead) is returning false positive entries for those still using it today.

What does this mean?

Security Sage Update

It seems today as though the Security Sage domains have expired and/or replaced by "placeholder" pages by their registrar. Net result: Bad things. If you were still using their BL, you're probably having problems receiving inbound mail right about now.

DSBL Current Status: DEAD

DSBL, the Distributed Sender Blackhole List, seems to have gone missing. The list appears to have been in operation since at least May, 2002.

Help, we're listed on ORDB!

I've received multiple queries about this today, so I figured it would be wise to put up a quick message about this.

ORDB is a long dead blocking list, gone for more than a year.

Recently, they started "listing the world" -- meaning everybody using ORDB is now blocking 100% of inbound mail. Blocking lists do this to shed themselves of any excess DNS query traffic from sites who haven't yet ceased querying their data. It can very much be considered a slap in the face -- hey, we tried shutting down the nice way, but since you're not listening, we're going to make all your mail bounce.

But what does that mean? Why am I listed?

You're not actually listed on ORDB. ORDB is returning a "yup, they're listed" answer for any IP address that people check. Meaning the whole world is listed. Everybody, not just you. It's not because they hate you, it's because they want people to stop querying their DNSBL.

If you received bounces from somebody that suggests that you're listed on ORDB, here's what to do:
  1. Call that person on the phone, if you can. Tell them all of their inbound mail is probably not working, and won't work, until they stop using ORDB. Point them to this page for more information.
  2. Don't worry. The person who bounced your mail is suddenly now having problems receiving any mail at all. They're likely to figure this out very quickly and fix it. Try your mail again, in a day or two.

Status of rbl.spamhaus.org: NOT A BLOCKING LIST

My friend Mickey Chandler pointed out recently that he's been seeing some unusual bounces that look like this:

Host blacklisted - Found on Realtime Black List server blocklist.address.is.wrong.spamhaus.org

Status of blackhole.securitysage.com: DOWN

The RHSBL (right hand side blocking list) blackhole.securitysage.com appears to have been created by Jeffrey Posluns and appears to have been around since at least August, 2004.

I received a report today indicating that a mail administrator has been unable to reliably query the blackhole.securitysage.com DNSBL zone. With the help of my friends, I was able to confirm this issue.

It looks to be a DNS issue. What we see from here is that the zone blackhole.securitysage.com is delegated to nameserver blackhole.securitysage.com. The two DNS "glue entries" for the zone are servers that aren't configured to be authoritative for the zone, so no results are returned. Ultimately, this points toward a DNS configuration issue with this domain and/or sub-domain.

The popular anti-spam filter SpamAssassin has been tracking this issue since at least October 8, 2007. On October 17th, SpamAssassin decide to remove support for this list (implemented in the DNS_FROM_SECURITYSAGE rule), due to the ongoing issues with accessing this DNSBL.

As a result of this ongoing issue, I recommend against using the blackhole.securitysage.com blocking list. If you continue to check against this list; queries are likely to time out and it could delay the receipt of inbound mail. Use of this list while this issue persists is likely to provide no blocking or filtering benefit.

I, and others, have contacted Security Sage and Mr. Posluns, making him aware of the issue and asking for more information. I'll be sure to update this page with more information as I have it.

11/03/2007 update: I've seen no response to my email to Mr. Posluns, nor to a friend's email to Security Sage's support address. I emailed that support address today, and my attempt bounced. The error message suggested an SPF failure. The fact that I publish a working SPF record, and other information in the bounce, suggest that it is in error. I guess that means either nobody's home, or they don't want anyone to contact them.

5/26/2008 update: Way back in November, I talked to Jeffrey Posluns. He is no longer actively involved with Security Sage, but was kind enough to nudge the folks running things, in hopes of making things better. It fell off my radar, until a few days ago, when I was alerted to the fact that Security Sage's domains have expired.

Net result: Broken blocklist. Has a wildcard listing, meaning that if you use their list, you're probably negatively impacting your own email delivery.

My recommendation: Stop using this blocklist immediately and permanently. Even if they do somehow manage to pull things back together, they don't have a good track record of staying online.

PSBL: Easy On, Easy Off

The Passive Spam Block List, or PSBL (psbl.surriel.com) is a spamtrap-driven anti-spam blocklist that has been around since at least June, 2003. Created by Rik van Riel, who explains on the PSBL website that “the idea is that 99% of the hosts that send me spam never send me legitimate email, but that people whose mail server was used by spammers should still be able to send me email."

The passive nature of the list means that there's no probing or poking of remote servers on the internet (which tends to make ISPs very angry and was a significant issue back in the days of testing for open relays). It also means that there is no debate or argument with listees. As the PSBL website states, “Want to remove your mail server from PSBL? Go ahead.” No need for lawsuit threats, arguments over why listing is denied, or anything of the sort. Anyone can remove any entry for any reason.

Sounds scary, doesn't it? In theory, bad guys could game the system, and rob PSBL of its ability to stop spam. Thankfully, the data shows that this isn't something to worry about. PSBL is a pretty neat tool that can help system administrators filter or reject spam in a way that makes it very easy to prevent false positives. And even though it doesn't take a line as hard as Spamhaus or Spamcop, it manages to block some spam that they do not.

Success Rates
PSBL's success rate seems to greatly vary from week to week. Over the past ninety days, its overall effective rate is 41.4% against the spam hitting my spamtraps. Over the past thirty days, it has been 36.5% effective against spam.

False Positives
False positives are often non-zero, but generally very low. For the past eleven weeks, consistently under 1%. I suspect that this is due to the “easy on, easy off” removal policy-- If anyone trying to send you mail receives a bounce message back from you referring to the PSBL website, it's very easy for them to have their sending IP address removed from the list.

Additive Numbers
Even though PSBL catches a lower amount of spam (on its own) than some other more well-known blocklists, it manages to catch some spam that those other lists do not. To determine this, I took the last thirty days worth of results, and looked for intersection and overlap between PSBL and other blocklists.
What I found is that about 9% of successful PSBL hits against spam stopped spam from IP addresses not found on Spamhaus ZEN. When compared against Spamcop, the numbers were even higher -- about 13% of successful PSBL hits stopped spam from IP addresses not listed on Spamcop.

This suggests to me that PSBL would be an excellent blocklist to configure second or third in your mail server configuration. That 9% of IP addresses not found on both Spamhaus and PSBL won't lead to a straight 9% boost in spam filtering effectiveness, due to lists being different sizes. But, if your data is like mine, you're likely to receive a boost of 3% or more.

Conclusion: I recommend PSBL. It helps to block spam that some other lists could miss, and it has friendly anti-false positive policies that make any revealed blocking issues easy to resolve.

The usual caveats applies here: This data illustrates how my own mail streams intersect with PSBL. Your mileage may vary, and I strongly recommend that you test and review results against your own mail streams.

Spamhaus ZEN: The DNSBL Resource Review

Spamhaus ZEN is a composite blocking list run by the Spamhaus Project. This UK-based organization was created in 1998 by Steve Linford, and is maintained by a group of employees spread across the globe.

Status of completewhois.com: IN FLUX

Update 9/30/2007: The website www.completewhois.com is operational again, but some links appear to be broken. My attempts to query their DNSBLs have all timed out. While CompleteWhois may be on the mend, it seems that it may be too soon to give the all clear.

Previous updates follow.

APEWS: Doing the Math

I'm guilty. I admit it. I've called APEWS listings "random," which isn't quite right. Arbitrary would be a better word for it. Not to mention broad, and questionable.

APEWS, the "anonymous" blocking list meant to be an early warning system for spam, generates a lot of worry from administrators and end users who find themselves listed by way of plugging their IP address into an online lookup tools like DNSStuff. Though it doesn't result in much (if any) of anyone's mail being rejected, as it's not widely used, some people still think they're being labeled a spammer, and don't know what to do about it.

They've usually done nothing to warrant the listing; the simple fact of the matter is that they happen to have an IP address on the internet, and there's more than a 1/3 chance that this IP address will be on the APEWS blocklist.

As I've indicated previously, APEWS has IP address entries accounting for about 42% of the raw numerical depth of V4 IP address space, though I'm not excluding non-routable space and overlap between some listings. When one takes those factors into consideration, APEWS seems to list somewhere around 38% of currently routable IP4 network space.

Time for an experiment. What if I take a large chunk of address space, say, 42%, and list it all? I've got detailed records of spam and ham, and it's easy to bump my corpus up against an imaginary blocklist I've just made up right here on the back of this napkin.

Here's what happens when I do that: Over the past ten days or so, my 42% listing of IP space would've captured 62.8% of spam, but also incorrectly captured non-spam 31.5% of the time.

When I skinny my imaginary blocklist down to 38% of IP4 space, I get a 62.21% hit rate on spam and 31.15% false positive rate against non-spam. (In other words, just about the same numbers.)

To me, this is evidence that APEWS seems to be blocking some spam based on the "stopped clock is right twice a day" principle. List a large chunk of IP address space, and you're going to catch a significant amount spam, though inaccurately.

It further suggests to me that if I added a few rules to start my focus points with a bit of accuracy, I could probably tune this to get a hit rate close to what I see from APEWS, with its 73% hit rate against spam, and 26% false positive rate against non-spam (21 day average ending on 9/2/2007).

The conclusion I draw from this exercise is that only the barest thought has been given to the processes by which APEWS decides which IP addresses to list and for what reason. If I can get more than halfway there with a couple hours of sloppy bar napkin math, then perhaps they haven't thought it through too deeply.

What to do if you are listed on APEWS

If you are listed on the APEWS blocking list, as confirmed by checking their website, here's how I would recommend that you handle the situation. (Who the heck am I?) 

Note: This isn’t guidance on how to avoid a blocklisting or sidestep anti-spam groups. If you have a spam issue, fix it. Don't spam, ever, for any reason. This is information is regarding how to address an issue with a list that is very aggressive at listing non-abusing IP addresses and networks, with no published, attainable path to resolution.
  • Don't despair. Be calm.

  • Do NOT post to a USENET newsgroup or to Google Groups, asking for assistance. Any replies you get will be from people who do NOT work for or with APEWS, and most of those replies will be unhelpful.

  • I can't stress this point strongly enough: Posting requests for help on the Internet will not get you any assistance. The APEWS FAQ directs people to post questions, but the only thing that happens is that discussion groups are overrun with questions, and the only people who answer those questions are (a) not involved with APEWS and (b) rarely polite or helpful.

  • APEWS ability to be used as a spam filter has been greatly reduced and restricted due to perceived malfeasance on the part of the APEWS maintainer(s). UCEPROTECT and SORBS, blocklist groups who used to publish the APEWS data, are no longer doing so as of August 13, 2007. This means that the two main channels available to administrators to use APEWS as a spam filter have been revoked. This means that if your mail bounced due to an APEWS listing before or on August 13, 2007, you might want to try to send your mail again – it would likely get through, as the list is even LESS widely used than it was up until August 13, 2007.

  • APEWS is very aggressive (meaning its use as a spam filter drives a lot of false positive blocking) and as measured by me on August 11, 2007, lists approximately 42% of the Internet. (By “the Internet” I mean IP4 address space.) In other words, they list nearly half the Earth, suggesting that anybody who actually wants to receive mail probably cannot use APEWS as a spam filter. This strongly suggests that very few people are going to block your mail because of the APEWS listing.

  • Anyone using APEWS as a spam filter is going against the advice of a multitude of other anti-spam advocates and email professionals. See my news and commentary roundup for more information and links to feedback from others in the anti-spam arena.

  • Since APEWS is not widely used, your next step should be a review of your bounce data. Have you received any bounces that reference an APEWS block?

  • If not, don’t worry about it. You just determined that you’re not having blocking issues that you can trace back to APEWS. It’s annoying that you’re listed on the website, but there’s little easy recourse available to you to address that.

  • If yes, you have received a bounce message that references APEWS, contact the site that blocked your mail. Call them on the phone or email them from a different email account (Hotmail, Gmail, etc.) Show them that APEWS is problematic and not widely used. Explain to them that you do not spam, and that APEWS has listed you even though you do not spam. Provide them links to this page on DNSBL.com with more information about APEWS.
If you want to learn more about APEWS, I've collected everything I know about this “anonymous” blocking list here on DNSBL.com.

I hope you find this information helpful. Please feel free to contact me with your comments or feedback. But, please note that I'm unable to consult with you regarding your specific situation -- I've already got a full time day job, and I'm not looking for consulting clients.

APEWS News and Commentary Roundup

APEWS, the Anonymous Postmasters Early Warning System, is an “anonymous” blocking list that claims to run in the style of SPEWS. That is to say, its goal is to be an “early warning system,” catching and stopping spam before other lists or filters have the opportunity to do so.

The APEWS blocking list was first announced by way of an anonymous posting to the newsgroup news.admin.net-abuse.blocklisting on January 12, 2007. Though this newsgroup post originated from the IP address 149.9.0.57 (registered to US provider PSI/Cogent), the list is widely believed to be run from Germany.

If you are listed on APEWS and wondering what to do, visit this page for my suggestions.

Accuracy

A quick review of the past thirteen weeks of my own stats.dnsbl.com data shows that the list has been ramping up in aggressiveness the entire time that I've been tracking it. What was barely a 20% effectiveness rate against spam eleven weeks ago is up to 80+ percent on a week-by-week basis. However, false positives have risen similarly.

The rising spam match rate is based on what I would characterize as the “stopped clock is right twice a day” principle. List enough IP addresses, and eventually you're going to stop some spam. The side effect is that you're going to block legitimate mail (and lots of it) at the same time. Against my personal hamtrap data, APEWS blocks two out of ten of every legitimate piece of newsletter or list mail that I've signed up for.

I'm not kidding about "listing enough IP addresses," either. As of today (August 11, 2007), APEWS lists just about 1.8 billion IP addresses - by the raw numbers alone, this is 42% of the entire IP4 networking space. Much of the IP space listed isn't even routable; suggesting little attention is being paid to what IP addresses are actually able to transmit traffic (email or otherwise). Also, APEWS has been growing at a very fast rate. From July 20th through today, they have added an additional 7.5 million IP addresses. These are data points that, in my opinion, suggest that the list is bloated, questionably targeted, and inaccurate.

09/30/2007 update: Click here to read about how I can similarly block around 60% of spam just by arbitrarily listing 42% of the internet.

Based on this data, and the recommendations of other trusted blocklist operators and anti-abuse folks, I personally would not use APEWS to filter incoming mail.

Controversy and Commentary

The blocklist is considered controversial by many other blocklist operators, ISP abuse staff, and anti-spam advocates.

  • Matthew Sullivan, SORBS maintainer, indicates that as of August 9, 2007, SORBS will no longer be publishing the APEWS blocklist zones via DNS.

  • Claus V. Wolfhausen, maintainer of UCEPROTECT, another German-run blocklist, indicates that UCEPROTECT will no longer publish the APEWS blocklist zones. (Previously: Claus warned that unless APEWS were to make immediate, significant changes to its policies, UCEPROTECT will no longer publish the APEWS blocklist zones.)

  • Suresh Ramasubramanian, respected anti-abuse manager for large mailbox provider Outblaze, categorizes APEWS as “meant to be used by fools.”

  • Steve Linford, Spamhaus maintainer, has suggested numerous times on newsgroups and elsewhere that APEWS is poorly run and is not widely used.

  • Kevin Liston and others from the Internet Storm Center have indicated that APEWS is using the ISC "top source" data to support blocklist entries, in violation of the data's license, and against the wishes of those who provide this data. ISC says that the data "is not supposed to be used as a blocklist as it is bound to include false positives" and that "APEWS may be a useful 'anti-spam" list if you do not mind losing a lot of valid e-mail as well."

Misplaced Newsgroup Discussion

If you read either of the two popular anti-spam newsgroups (news.admin.net-abuse.blocklisting and news.admin.net-abuse.email), you already know that both groups are often overrun with requests (example) from people who find that they are listed by APEWS. I find over 2,000 messages on these groups relating to APEWS remove requests, which is a high number considering that the blocklist is less than a year old. The blocklist group is run “anonymously.” Question 41 of the APEWS FAQ asks how one contacts APEWS. The answer includes the following: One does not. APEWS does not accept removal request by email, fax, voicemail or letters.” [...] “General blocklist related issues can be discussed in the public forums mentioned above. The newsgroups news.admin.net-abuse.blocklisting (NANABL) and news.admin.net-abuse.email (NANAE) are good choices.

This is likely why many administrators post to these newsgroups, asking for assistance, when finding their IP addresses are listed. The FAQ does warn that “abusing these newsgroups & lists by posting removal request you will make a fool of yourself,” but that doesn't seem to be a deterrent. I would theorize that this is because a lot of the people on the wrong side of listings do not understand why they are listed and do not now how to “fix” whatever issue led to the listing, as the listings are often broad and vague.

ISP Perspective

Vincent Schönau, an ISP abuse adminstrator, has related his APEWS experiences to me in email, and given me permission to share them here.

Other blacklists have employed the 'escalations' strategy in the past, but APEWS has taken it to a whole new level; a few spams from a providers ip ranges will cause all or most of the providers ip space to be listed in APEWS, with comments such as 'unprofessional / negligent provider'. What this means is that if your provider is a noticeable source of e-mail, sooner or later, it's going to get listed. Several providers of 'blacklist checks','blacklist comparisons', 'e-mail reputation checks' and include APEWS data. Apparently this is causing systems administrators who are desperate to reduce the amount of spam they're receiving to think that using it might work - perhaps because not all of those sources include the data on false positives for the blacklists. In practice, this means that several times a week, I'm spending time explaining to my users how they should work around the e-mail delivery-problems they're seeing which may or may not be related to APEWS. I could be spending this time taking action against compromised hosts in our network instead. This hurts providers who do take action against the abuse from their network more than providers who didn't care in the first place.

Others have related similar stories to me, of how long after spammers were booted, that a listing still persists. In one instance, a provider had a compromised machine, which was identified and disconnected within two hours of sending spam. Three days later APEWS listed it, and six weeks later, the listing persists, even though the issue is long since addressed.

If you are listed on APEWS and wondering what to do, visit this page for my suggestions.

SORBS: Accuracy Rates and False Positives

The blocking list SORBS (aka the “Spam and Open Relay Blocking System”) was created in 2002 by Australian Matthew Sullivan. SORBS publishes a main “aggregate zone” (dnsbl.sorbs.net) containing listings meeting a multitude of criteria beyond open relaying mail services. SORBS also publishes multiple other zones meeting various criteria.

As related previously, SORBS appears to be undergoing changes. Some of these changes appear to relate to the fact that the SORBS maintainer has repeatedly taken issue with the methodology used by DNSBL.com to measure accuracy rates and false positive rates.

SORBS has indicated that they have the ability to feed false or different data in response to queries from DNSBL.com. As such, it's unclear if recent query results are indicative of results seen by other users. Because of concerns that SORBS may be attempting to sway the data reported, it's important to share current data and information, so that system administrators can make an educated determination as to whether or not it would be wise to use this DNSBL.

Historical Information

I've been tracking data on the main SORBS zone, dnsbl.sorbs.net, since March, 2007. Here's what I've found.

  • For most of the past fifteen weeks, the DNSBL had an effectiveness rate varying between fifty percent and fifty six percent, week over week. This means that SORBS correctly blocked a piece of spam in my spamtrap about five to six times out of ten.

  • For many weeks, I believe SORBS clearly suffered from significant false positive issues. As measured by my own calculations (see here and here for more info), the false positive rate is in the 7.9% - 11.1% range. This means that if your users sign up for the same kind of mail that I did, that for every one hundred pieces of solicited mail your users signed up for and expected to receive, SORBS is likely to block seven to twelve of them.

Recent Data Changes

  • On July 9, 2007, changes were made to SORBS. As you can see from the chart above, around this time (near the start of week 12), the net result is that the effectiveness rate and false positive rates have both significantly declined.

  • Since July 9, 2007, I have not noted any additional false positive from the main SORBS zone. Because of indication from SORBS that they are able to feed false data, it is unclear if the results I am seeing are accurate.

  • Similarly, the effectiveness rate of the main SORBS zone seems to have greatly declined as well. Since July 9, 2007, it is hovering in the 18% range.

There are two possible conclusions to make here:

  • SORBS is somehow able to feed different blocklist data to DNSBL.com than to others. If so, then the historical data I have summarized above is likely to be the most accurate view of SORBS. Or,

  • SORBS has gutted its lists and the poor effectiveness rates I'm now seeing are reflective of how it would likely work for others.

It's hard to say which scenario is the more accurate one, and what future testing will reveal. I'll certainly continue to collect data, but right now, there's an open question of SORBS' effectiveness and false positives.

As of Thursday, July 19, 2007, SORBS changed the default zone mentioned in configuration guidance pages from dnsbl.sorbs.net to a domain not owned by SORBS. As a result, if any SORBS user copies and pastes a configuration snippet from one of the SORBS configuration pages verbatim, the result is that 100% of a site's inbound email will be blocked. My recommendation is to proceed with caution – if you are not sure what you're doing with DNSBL use and mail server configuration, a misstep here will have significant consequences.

SORBS has leveled the following criticism, assumably as justification for for the results published on DNSBL.com. Below is an overview and response to the points raised:

  • SORBS claims that the DNSBL.com email feed data is US-centric. This is true. The domains involved in these hamtraps and spamtraps are "dot com " domains, and have always been hosted in the US. If this means that SORBS is inaccurate as a result, it suggests that SORBS is Australia-centric, and likely will not work as well for those in other countries.

  • SORBS claims that a false positive as defined on DNSBL.com is not what everyone calls a false positive. This is true. I consider a false positive to be a requested message that was blocked. Others have different definitions. I believe the definition used on DNSBL.com to be accurate. I further believe that the most common definition of a false positive as used by regular end users or system administrators is most likely to align with my own.

  • SORBS is unable to verify false positive hits, as DNSBL.com does not provide IP addresses correlating to false positive hits. This is true. If data were provided to any blocklist operator regarding false positives, this would enable the DNSBL to whitewash over the issues by removing the IP addresses reported (and no others). This is similar to why blocklist groups do not provide spamtrap information – they do not want their spamtraps “compromised,” which would allow a bad sender to simply stop sending to spamtraps, but continue spamming elsewhere. Therefore, this information is not provided to any blocklist. (Other list operators have been more understanding.)

  • SORBS claims that the zone “dnsbl.sorbs.net” being queried by DNSBL.com is not the zone used by most users or recommended by SORBS as the main or default zone. This is untrue. It has or had clearly been positioned as the default zone or default recommended configuration choice, and remains the zone first listed, positioned as the “aggregate zone” as of July 20, 2007.

  • SORBS claims that Spamhaus volunteers have (or had) access to the SORBS database and have entered listings in the past to drive significant false positive issues. I am not associated with either SORBS or Spamhaus so I can't speak to this accusation.

  • SORBS claims that the methodology of checking mail against DNSBLs within 15 minutes of receipt is inaccurate. This is untrue. Anyone who uses a DNSBL is enabling their mail server or spam filter to check the mail against the DNSBL within seconds to minutes of receipt. If, as SORBS states, their DNSBL distribution model is such that it suffers from this methodology, then it suggests that it may be slow to respond to real spam trends. (10/29/2007 update: At a recent conference, over a beer with a colleague who builds tools to block spam for a living, I was gently chided over this bit of methodology. I was told that I was letting mail get far too old. 15 minutes is a hundred years as far as spam vector measurement is concerned; the vendor in question uses a 60 second interval at maximum. By this logic, I was being too forgiving as far as slowly updating anti-spam blocklists were concerned. This is further at odds with the criticism from SORBS.)
  • SORBS has picked a specific sender as the source for the SORBS false positive rates I report, saying that this sender is a "habitual source of spam." I have no financial interest or any other connection to the sender in question, except that I ordered pillows from them in December, 2006, and was happy with the product and service they provided. As a result, I signed up to receive mail from them, and happily do so. If I used SORBS to reject mail, that mail would not reach me. Additionally, this sender is far from the only source of false positives I found when utilizing the SORBS blocklist. (11/09/2007 update: The specific sender is/was Overstock.com. SORBS categorizes Overstock as a spammer. Matthew Sullivan (now known as Michelle Sullivan), in fact, indicated that "1000's of people who receive unsolicited commercial/bulk email from them." There are two additional problems with his characterizations here. First, Overstock.com is not listed on ANY OTHER of the approximately 47 blocklists I check, except FIVETEN (which lists many hundreds of potentially legitimate senders, and therefore, is not very useful as a second opinion here.) It's not on any of the lists that commonly do list supposedly-legitimate senders who may have run afoul of spamtraps. Second, the last mail I had received from Overstock.com was on May 25, 2007. This is significantly before the July 9th cutoff of my data, and measured false positives were on the rise even with no further mail from Overstock.com in the data set. Incidentally, I have no idea why I've received no mail since. I didn't unsubscribe.)

Additionally, SORBS has made numerous statements questioning the accuracy of data published here, and characterizing this project as something other than honest and transparent. Here's how it works: I have a feed of mail, and I check all mail received for DNSBL hits. I give internet users a live, rolling snapshot of how various lists intersect with my mail steams. That's all there is to it. I leave it to you, the reader, to decide if I've been honest and clear at every step of this process, and as always, I welcome your feedback.

(11/18/2007 Update: Added the phrase "that if your users sign up for the same kind of mail that I did" above to clarify false positive comments.)

Status of dnsbl.radparker.com: NOT A DNSBL

For a time, SORBS was found to be inaccurately referring to “dnsbl.radparker.com” on the mail server configuration pages over on the SORBS blocking list website. This appears to have been done in retaliation for DNSBL.com publishing data on the effectiveness of the SORBS blocking list. (Both domains are owned by me.)

The real problem was for potential SORBS users – if they followed the instructions verbatim, they ended up rejecting 100% of your inbound mail. Sadly, I've seen traffic, which implies that this has happened to some degree. If you're going to use the SORBS blocklist, be very careful to make sure you've implemented it correctly. Both this, and SORBS' claim that dnsbl.sorbs.net is an unsafe zone to use, suggest that the SORBS' list may not be a wise starting point for those looking to simply, safely block spam. More information on SORBS can be found here.

There has never been a blocklist with a zone name of dnsbl.radparker.com -- and if you type that into the DNSBL section of your mail server config, you will break your inability to receive inbound mail.

Changes at UCEPROTECT

Whatever your opinion of UCEPROTECT, hold on to your hat, as things are apparently about to change.

This posting to the USENET newsgroup news.admin.net-abuse.email indicates that Johann Steigenberger is no longer involved with UCEPROTECT. Going forward, Claus v. Wolfhausen has indicated that he is charge of the lists.

At first there was some concern that this post wasn’t true, that it was a deception. I’ve spoken to Claus via email, and that, along with other information, leads me to believe that this is in fact true and correct. (I’ve met neither individual in person, so I suppose this could be a giant hoax, but I’ve got no reason to believe so at this time.)

Claus indicates that UCEPROTECT will no longer list for backscatter and sender verification callouts. These two listing criteria were controversial and I am told that they resulted in numerous complaints of false positives relating to UCEPROTECT. These data relating to listings based on these criteria are being repurposed into a new blocking list at www.backscatterer.org.

He went on to say due to his intervention, UCEPROTECT has ceased publishing the controversial “anonymous” APEWS blocklist data, and that he is unsure if UCEPROTECT will again publish the APEWS data in the future. APEWS, an “anonymous” list widely thought to be created as a replacement for the defunct SPEWS, has been regularly criticized by respected anti-spam advocates such as Steve Linford of Spamhaus and Suresh Ramasubramanian of ISP Outblaze. Controversy includes listing policies considered to be broad and inaccurate, and contact/removal policies perceived as cruel to listees (by deflecting all contact away from the blocklist and toward public discussion forums where listees are often subject to abuse from unrelated parties).

I have yet to write and post reviews of UCEPROTECT or APEWS for dnsbl.com. Look for this in the future.

Status of blackholes.intersil.net: DEAD

I've recently received a few requests regarding blackholes.intersil.net. According to this message to news.admin.net-abuse.blocklisting, this blocking list was retired in July, 2006. Confusingly, the zone still contains a test entry. Regardless, I've seen no hits against it in a very long time. Therefore, if you are using this DNSBL for spam blocking, I recommend you cease doing so. If you have any further information about this list, please feel free to contact me.

Status of blacklist.spambag.org: DEAD

As of Sunday, May 27th, 2007, the blocking list Spambag, with the DNSBL zone blacklist.spambag.org, is no longer available. The website www.spambag.org does not resolve to an IP address, and there appear to be no DNS entries under the DNSBL sub-zone.

Spambag, created and run by Sam Varshavchik, developer of the Courier mail server, has been operating this list since at least November, 2001.

The list had the following listing criteria: "[Spambag is my] personal list of networks who I block from sending me mail or accessing my web servers, because I believe the networks actively or passively allow abusive or antisocial behavior. Examples of what I consider abusive or antisocial behavior are: spamming, mailbombing, mail server dictionary attacks, and web page E-mail address harvesting."

I last noted a hit against this DNSBL on May 26th, at 1:34 am US central time. Note that I was not a user of this list; I simply measure its effectiveness and status, like I do for many other lists.

This post to news.admin.net-abuse.email explains that Sam Varshavchik shut the list down, and that he felt his efforts had not been as productive as he would've liked them to be.

I would recommend removing blacklist.spambag.org from your list of DNSBLs to check, as it is no longer in operation.

Spamcop BL: Take Another Look (It’s Accurate!)

If you know me, you know that in the past, I’ve made no secret of my disdain for the Spamcop DNSBL, aka the SCBL. I’ve worked in spam prevention, deliverability, and the email realm for a long time, in various capacities. I’ve created and run at least two blocking lists that you know about. Later, I helped to design and create a system that processed thousands of confirmed opt-in/double opt-in newsletter signups a day. Combine those two details together and that’s what led me to take issue with Spamcop. My employer’s COI/DOI signup servers kept getting listed by Spamcop, based on some really bad math to measure email volume thresholds and make a determination as to what to list.

I was trying to do the right thing. I was implementing what Spamcop (and other anti-spam groups) want: confirmed opt-in/double opt-in. Yet Spamcop was listing the servers and subsequent mailings regardless. It made me really frustrated, and I was very disappointed. See, it’s not really fighting spam. It’s just blocking mail you don’t like, or don’t care about. While perfectly allowed, I am of the opinion that it’s lame to do so under the banner of “fighting the good fight” to stop spam. I’ve shared my thoughts on this topic in just about every available forum—websites, blogs, discussion lists. I know I’ve personally guided many sysadmins away from using the SCBL in the past, because it was easily, demonstrably, listing things that were obviously not spam.

In February 2007, I found that Microsoft was using the SCBL to filter/reject inbound corporate email. (Note that I said corporate email—mail sent to users at micrsoft.com, not users of MSN or Hotmail. I don’t know whether or not SCBL data is used in MSN Hotmail delivery determination, but from what I’ve observed, it doesn’t seem to be.) This started me off on another rant on how ill-advised I felt this was, based on my prior experiences with Spamcop. Some kindly folks (and some less kindly) suggested that I needed to revisit my opinion of the Spamcop blocking list, because things have changed.

After a lot of measuring and discussion, I’m here to tell you: Spamcop’s DNSBL has changed, and for the better. It works very well nowadays, as personally measured by me. The open question on Spamcop was what drove me to dive into my massive DNSBL tracking project. I started that back on March 10th. Ever since then I’ve been compiling data on Spamcop blocking list matches against both spam and non-spam. Here’s what I see:

DNSBL

Spam hits

Acc %

Ham hits

Failure Rate

Spamcop SCBL

156194

49.37%

0

0.00%

Spamhaus ZEN

255521

80.77%

5

0.10%

Spamcop+ZEN

267795

84.65%

5

0.10%

Range:

~ 74 days

Total Spam

316348

Total Ham

4999

As you can see, Spamcop helps you attack nearly 50% of spam received, while affecting no legitimate senders. Very few lists do better. Spamhaus ZEN (which combines multiple lists) does better, but will occasionally have a false positive, based on some reputational issue perceived with a given sender.

My recommendation: Spamcop’s blocking list is safe to use, and will effectively help you reduce the amount of spam you have to deal with. Where I find it particularly useful is as an addition to Spamhaus ZEN: If you block mail from entities on either list, you get a 3.8% percent boost in effectiveness. Meaning, just under four percent of my spam hits are found on the Spamcop list, but not on Spamhaus.

For historical reasons only, here are links to my previous articles on Spamcop:

Spamcop Roundup http://www.dnsbl.com/2007/03/spamcop-roundup.html Spamcop BL: A blocklist with a hair trigger http://www.dnsbl.com/2007/02/spamcop-bl-list-with-hair-trigger.html Microsoft using Spamcop BL http://www.spamresource.com/2007/02/microsoft-using-spamcop-and-spamhaus.html My Problems with Spamcop http://www.spamresource.com/2003/03/problems-with-spamcop.html

Status of relays.orbs.org: Shut down, legal troubles in 2001

Remember ORBS? Short for “Open Relay Behavior Modification System,” it was a blocking list run by Alan Brown from New Zealand. (Mr. Brown ran the second version of ORBS. The first version had been run by Canadian Alan Hodgson.)

People keep asking me about the situation regarding ORBS and its eventual downfall. It happened so long ago, that I don't feel that it would be appropriate to try to fill people in from memory alone. Instead, here's links to a lot of the articles I've found regarding Alan Brown and ORBS. If you have any others, drop me a line and I'll add them to this page.
Please note that I'm not linking to any commentary or conspiracy theories put forth by emotional, anti-blocklisting “how dare you block my guaranteed opt-in email” people. There are many blocklists run correctly and appropriately. There were then, and there are now. The lists themselves weren't the problem, and aren't the problem now. Like with any other field of study, type of product, or process, some manage it well, and others do not.

Status of rbl.cluecentral.net: ALIVE

The DNSBL “rbl.cluecentral.net” has been revived. Its maintainer, Sabri Berisha, had previously shut it down in November 2005.

This list aims to allow you to allow or block mail from specific countries, or from certain routers (by AS number). For example, if you wish to block all mail from the US, you could configure us.rbl.cluecentral.net as a DNSBL to be used for mail blocking in your email server software, and you would then block all mail from the US, as identified by Sabri’s categorization. For more information, see Sabri’s post to the NANOG mailing list, announcing resuscitation of the list, or click here visit the list’s website.

Note that while these lists may be used to block spam, they're not exactly spam-blocking lists. Rejecting all mail from China simply means that you're going to reject all mail from China, spam or non-spam. It's up to you to determine whether or not this is an acceptable compromise. I assume, like with users of korea.services.net, administrators who choose to use this list are fed up with spam from a certain country's servers, and receive little enough legitimate mail from a country that the risk of false positives is considered acceptable.