On 10/30/02, "Anne P. Mitchell, Esq." wrote:
> So far, in all of our discussions with ISPs and MSPs, the response
> has been "we may not like it, but we want to be able to distinguish
> the confirmed opt-in from not, because our users want it, and our job
> is to deliver what our users want, without delivering spam".
As an engineer type, I've always firmly beleived that a problem
should be solved as close to the source as possible. In a case
like this, where we generally only control the recipient network,
the closest we can get is the border of that network. But
blocking at the border has two problems:
- lack of reliable statistics (if you don't allow the
connection, nobody knows how many messages were
blocked)
- higher potential rate of false positives (and even
if this isn't true, it can't be disproven because
of problem #1.)
So, we move into the next layer inwards: the mail server. This
has two distinct layers of its own: the SMTP session, and
internal processing. (It could be argued that there's also
the IP stack, but that's functionally equivalent to blocking
at the router.)
Our options for blocking during the SMTP session are limited
because we only have a limited amount of information:
- IP address (reliable)
- reverse DNS (sometimes reliable)
- IDENT response (extremely unreliable)
- MAIL FROM (unreliable)
- RCPT TO (reliable, multiple)
Most of the SMTP session solutions we all use are based on the
IP address. There used to be a lot based on reversed DNS
and/or MAIL FROM, but those are easier to spoof. Generally
the only things we can do with RCPT TO are recipient-based
rate-limiting (easily gotten around) and spamtraps.
Blocking in the SMTP session is elegant because it generally
follows at least the spirit if not the letter of the standards
regarding non-acceptance of e-mail. This is WONDERFUL for
false positives because legitimate senders will receive a
bounce message and know that their mail was not delivered.
However, it doesn't allow for per-sender or per-list granularity
across the same IP or domain (or with unknown IP's) unless you
look at the unreliable (and legitimately malleable) MAIL FROM.
Next up is server-side internal processing, which is pretty
much all that Hotmail has ever done. The drawbacks are (or
should be) obvious:
- it's expensive; you have to accept the mail, run it
through various filtering programs, etcetera.
2. false positives are lost forever unless you've got
a way to generate a bounce message after acceptance
(which then means queues are clogged with bounces)
But server-side filters also allow for much more granularity,
and it's often easier to make pretty graphs to show how well
the filters are working.
A final layer is the client side, which has most of the same
problems as the post-acceptance server side. It's also even
more expensive, because the mail (including the unwanted mail)
has to be transferred from the server to the client. However,
it allows for all of the latest fads:
- whitelisting, which works great for some people
but generally undermines the utility of e-mail
- challenge/response systems, which are all fucking
annoying and undermine the utility of e-mail to an
even greater extent
- Bayesian or other smart filters, which look like
a very good solution but require ongoing
maintenance by the end user
To answer Anne's original question, the only place where there's
a reliable chance of discerning between individual lists from
the same source is in server or client filters...and it's better
in the server, because the client doesn't have a reliable way
to determine the originating IP address or domain (it has to
trust the server.)
I'm obviously ignoring the question of how to maintain the list
of block/filter (or don't block/don't filter) criteria. However,
that issue seems even more important when you get to a per-list
granularity, because new lists are started all the time.
Whew, that ended up being a lot longer than I'd expected. I'd
intended to follow up with equivalent verbiage about how to
block the spam at the source, but that can wait for now.
A few days ago, my friend Neil gave a brief answer to the question "Spamfighter, howcome you got into spamfighting?" I get asked that a lot, too -- and while my answers vary depending on my mood at the time (and usually go on way longer than the questi
Tracked: Oct 21, 10:11
Some people -- some friends -- are going to ask why I left a particular anti-spam mailing list after eleven and a half years, and what that means. Here's a bit of a timeline: 19 October 1995I subscribed.cyberNOTHING.org was only about five months ol
Tracked: Mar 17, 22:39