Expanded Spamtraps and Hamtraps

As always, I'm looking to maintain and improve the accuracy of the data behind the reports over at the Blacklist Statistics Center here at DNSBL Resource. Here's a quick overview of a couple of recent improvements I've made.

Just this week, I've turned up an additional spamtrap feed. This data is based on a set of domains that were no longer routed, but are found on many spammer lists. Not sure how much this will change the spamtrap data, but we will see. It's always good to mix things up.

What this means: I'm broadening my view of the net, working to keep things on the up-and-up by expanding the inbound spam feed to keep this data from becoming biased; if my spamtrap feeds were small enough that they overlapped with one list's traps significantly, but not another's, it could potentially bias results in favor of that list. I don't think this is happening currently, but I'm going to continue to change things up periodically in an attempt to pro-actively attempt to prevent this from ever happening.

On the hamtrap front, I'm now periodically testing to see if various blacklists are blocking large webmail provider outbound mail servers. So far, I'm checking AOL, Hotmail, Yahoo, and Gmail. I don't have a complete view of what all of the outbound IP addresses are for each site; only AOL seems to publish a comprehensive list. I've determined what I can based on headers from real mail that I've sent and/or received over the past week or so. Feel free to contact me if you have pointers to official, published information from the bigger sites.

What this means: If a blacklist blocks all mail from various Yahoo IP addresses, and you have friends who use Yahoo, they're going to have trouble emailing you. If that's the case, this is going to generate significant false positives. It certainly would generate false positives for me; all of my friends seem to use one of those four webmail providers.

 Some blacklists might not like that I now make, and publish, this measurement. It's true that some ISP outbound mail servers send spam sometimes, and it's true that those ISP outbound mail servers might be appropriately listed on a given blacklist. But it's also true that even though those servers might send some spam, they also send quite a bit of legitimate mail, and avoiding false positives in that situation becomes near impossible.

Ultimately, it's up to you, as a potential user of a given blacklist, to decide if that risk of false positives is acceptable. In some cases, it is acceptable. In other cases, it may not be.

Also, this makes the hamtrap measurements more likely to reflect real, one-to-one email, in addition to the newsletter and (non-spam) list mail already being tracked. I think this is a good thing.

(More good stuff is on the way...stay tuned!)