False Positives

One of the things I measure over at the Blacklist Statistics Center is false positives. What are false positives? How do I use the term, exactly?'

Ultimately, there are three different ways to define false positives, depending on whom you ask and who they are. Allow me to explain.

Here's what I think of as a false positive(1) in the context of DNSBLs: You did not receive a mail message you signed up for, and wanted to receive, because it was blocked by your use (or your ISP's use) of that DNSBL.

This is what I consider a false positive. If you signed up to receive news alerts and wanted to receive those alerts, but you couldn't receive them because your spam filter blocked that email, that's what I would call a false positive. This is very end-user focused, or recipient system focused.

That can be quite a bit different than what a blacklist calls a false positive(2). The example above might not be a false positive as far as the blacklist operator is concerned. Maybe somebody sent mail to their spam traps from that IP address. Or maybe the blacklist's policies are such that they choose to list an entire net block because of spam issues elsewhere in that net block.

I used to be a blacklist operator myself. Back then, what I considered a false positive(2) was a blacklisting that shouldn't have taken place, by my own reckoning. I primarily dealt with open relaying mail servers. If I had accidentally listed an IP address, even though it wasn't an open relay, that probably would be considered a false positive.

But, to the person whose mail is getting blocked as a result, that could constitute a whole other kind of false positive(3). Getting deep into that kind of false positive is a bit beyond the scope of what I'm doing here. Anybody whose mail has been blocked for any reason can feel it's unwarranted. Sometimes I would agree, sometimes I would not. But, that's not a debate for right here and right now.

Instead, I focus on the definition of false positives I think is most applicable to end users of DNSBLs: Mail you (or I) wanted to receive, but didn't receive, because receipt of that mail was blocked by that DNSBL.

That's the only kind of false positive I'm measuring and reporting on.