Rob's Blog

Update (02/02/2010): It would seem from the comments that this was a legitimate request for an autograph! MAybe I’m more famous than I think? Original post continues:

A few days ago I got a surprising email asking for my autograph (full text below). Despite getting literally tens of visitors to this blog per day, I am by no means a celebrity, so this either there is some famous person with my name (so famous I’ve never heard of them), this person is just trying their luck sending hundreds of emails hoping to get a real one or two, or this is some new scam to get my signature and use it to steal my identity.

I’ve searched for the sender of the email (Lalit Kumar Bajaj), and I can only find a few comments on blog posts requesting autographs and not much more, but it could be very new, or could be the scammers (if they are scammers) are using multiple names.

Read the rest of this entry »

I’ve recently noticed in my logs lots of visits to my posts from a Microsoft IP block (the last one was 65.55.109.226), supposedly with a referrer of http://search.live.com/results.aspx?q=[random keyword]&form=QBHP .

My posts never appear on the search results page, in fact some keywords are completely irrelevant.

The visits even show up in my Woopra log, which means the bots must support JavaScript. Annoyingly due the the low volume of visits to my site, this is really throwing off my statistics. In particular, because the bot is reporting a screen resolution of 800×600, my total stats show 14% of visitors have that resolution (which clearly isn’t accurate).

I found this post on Blogboing (and this older one), and this fuller explanation (including instructions on how to block it), which partially explains them, but the Microsoft response is clearly rubbish. As far as I’m concerned, this is just referrer spam, and another reason not to use Live Search!

I don’t think I’ll block them for now, but may do if it continues.

Disclaimer: We are resellers for Death2Spam, so I have a potential interest in recommending it, however, we became resellers because we like the service!

For years I had just accepted the ever increasing amount of spam delivered to my mailbox every day. My email address is at the top of every page on the company website and as such harvested by every spambot out there, but I’m not going to remove it – I’d rather ensure every (potential) customer finds it easy to contact me than reduce spam by removing or obfuscating my email address.

We use DNSBLs at the server to reject mail from known spamming servers, but it is hard to get the balance right between blocking spammers, and not blocking legitimate users. False-positives are not an option, especially considering our mail server hosts our customer’s email as well as our own in house.

DNSBLs block quite a few junk mails, but personally I was getting over 100 junk mails per day.

I used the standard Junk Mail Controls in Thunderbird, and that cut out quite a few more, but I was left with quite a lot I had to deal with manually.

I looked in to installing SpamAssassin onto the server, but decided against it for various reasons:

• It was important that junk emails were rejected during the SMTP conversation so that senders knew their mail was rejected, but we did not produce any backscatter. It looked like at the very least this was going to be difficult to implement, and cause the SMTP connection to be too long.
• I needed to offer users a way to customise the sensitivity level. Again, this wasn’t going to be trivial.
• The risk of false positives was too great – it wouldn’t be easy to let users recover email rejected as junk.

I decided to abandon SpamAssassin as it didn’t seem quite right for our needs.

After a bit of reading, I decided to try SpamBayes (with the ThunderBayes plugin which is no longer supported) – a great client side solution. With a bit of training this cut out a large proportion of junk – much better than the Thunderbird built in filter, and virtually no false positives. It did however take a while to process my mail each morning, and of course being a client side solution meant that if checking my mail using webmail or on my phone, I still had to wade through all the Spam.

A request from a customer prompted me to again look at server-side solutions. I discovered Death2Spam through the SpamBayes site (listed as a ‘similar project’), and contacted them for details.

Death2Spam works in a similar way to SpamBayes – it scores each email based on content and headers, and categorises it as ‘good’ ‘spam’ or ‘unsure’. Users can (and should) recategorise the emails to train the system
. Greylisting is also used, which can cause a delay on certain initial mails from obscure senders, but does a very good job.

The great thing about to Death2Spam is that is it works independently of your existing mail server, and client. It is essentially an smtp proxy, and rejects emails marked as spam during the SMTP conversation (as well as storing a copy), and only the ‘good’ (and ‘unsure’) emails are delivered to your mailbox. This means however you read your mail: webmail, desktop client or mobile, your mailbox will be spam free.

After a bit of training (using their easy web-based control panel), excellent statistics can be achieved. Mine is currently quoted at 98.8% accuracy. The number of spams being sent also seems to be going down (In the first full week of using it I had 2989 spam, total emails 3128, The most recent full week I had 128 spam, total 245). I am assuming spammers eventually remove addresses off their lists when mails are rejected.

The cost of D2S is £27 per user, per year. This may seem like a lot, but consider how much time is saved not having to go through hundreds of junk emails each day. They offer a one month free trial, which is well worth trying out. See more details on Death2Spam and the free trial here.