Skip to content

Justin's Linklog Posts

DSPAM acquired by Sensory Networks

Published May 31, 2007

whoa, didn’t see that coming. Quoting Jonathan Zdziarski via jgc’s newsletter:

…The [DSPAM] project had grown to a point where it would take others – with enough free time – to bring DSPAM to the next level as a widely accepted enterprise-class solution, and [I] decided that it would be in the best interest of the project to entrust it to someone with the technical knowhow and dedication to reach these goals. Many of you are aware of my work in the past with Sensory Networks in developing a hardware-accelerated version of DSPAM (capable of supporting multi-megabit speeds in large carrier environments). I’ve spent a considerable amount of time with SN’s team over the past several years and when we initially discussed working together, they had shown to be very excited and motivated about the project.

After careful consideration and many discussions at length, I decided to allow Sensory Networks to acquire the rights to the project, and continue development on it with their own team. SN has displayed a strong commitment to the open source community and has been working closely with other leading projects such as Snort, Clam Antivirus, and SpamAssassin. They assured me that the project will remain open-source and available to all, and at the same time the project will receive exposure in commercial environments it has not seen before, as many of you have been asking for. We’ve now completed the acquisition for the project, and I’d like to encourage you to support them in helping them move forward as it grows into new areas.

More details at zdziarski.com.

Dealing with backscatter, revisited

Published May 30, 2007

Back in January, I wrote about how I deal with email backscatter nowadays. Since then, I’ve made a notable tweak.

This is that I no longer reject “null-sender” traffic during the SMTP transaction. It turned out that it broke Exim’s implementation of Sender Address Verification, which performs the SAV check using a MAIL FROM of <>, rendering it indistinguishable from a bounce during the SMTP transaction.

Now, I’ve complained about SAV, but I have to be pragmatic anyway (Postel’s law and all that!) — so it was better to just allow other sites to perform SAV lookups against our server, and fix the anti-bounce stuff some other way.

The new method (below) does this, by allowing null-sender SMTP traffic just fine; it detects bounces in Postfix if they arrive via SMTP in RFC-3464 format, and bounces that slip past are then dealt with in a more CPU-intensive manner using the SpamAssassin “VBounce” ruleset (which is part of the now-released SpamAssassin 3.2.0, btw).

This increases the load, since some bounces cannot be rejected at MAIL FROM time now, and instead we have to wait ’til DATA — but CPU hasn’t been a problem recently, so this is ok.

Here are the updated instructions:

In Postfix

In my Postfix configuration, on the machine that acts as MX for my domains — edit ‘/etc/postfix/header_checks’, and add these lines:

/^Content-Type: multipart\/report; report-type=delivery-status\;/  REJECT no third-party DSNs
/^Content-Type: message\/delivery-status; /     REJECT no third-party DSNs

Edit ‘/etc/postfix/main.cf’, and ensure it contains:

header_checks = regexp:/etc/postfix/header_checks

Then run:

sudo /etc/init.d/postfix restart

This catches most of the bounces — RFC-3464-format Delivery-Status-Notification messages from other mail servers.

In SpamAssassin

As before, install the Virus-bounce ruleset and set it up. This will catch challenge-response mails, “out of office” noise, “virus scanner detected blah” crap, and bounce mails generated by really broken groupware MTAs — the stuff that gets past the Postfix front-line.

Dead laptop time

Published May 28, 2007

Argh. My Thinkpad’s power socket must have received a knock during the move. It no longer works with either of the two power bricks I have here — so it looks like it’s time to either (a) buy a soldering iron and some screwdrivers (incl Torx ones?) or (b) renew my IBM warranty service and send it in for some fixing :(

Bad timing.

Update: oh look, it’s working again! phew. I guess I should probably set aside some time for warranty service here anyway though…

Back

Published May 25, 2007

Hey — I’m back, rested and full of tasty, tasty Niçois and Provencal cuisine.

I got back just in time to vote, for what good that did with Bertie’s gang leading strongly in the current counts… argh!

For what it’s worth, I gave Patricia McKenna a preference, in the end. I was reminded that she’d been entirely on our side on software patents during her time as an MEP — so credit where it’s due, there; on top of that, a vote for the Greens is better than a vote going to Sinn Fein, after all, no matter what. ;)

Carbon offsetting

Published May 10, 2007

I’m off to Nice on vacation for two weeks, starting tomorrow — back on May 25th. See ya then!

In the meantime, and appropriately enough given that jet fuel I’ll be consuming, here’s some interesting stuff from my mate Eoin on carbon offsetting…

‘It’s a fecking minefield to figure out. There are many conflicting standards, some of which sound impressive but are useless in reality.

Steer clear of tree planting, especially outside Europe; even a well-run forestry in Europe will take decades to make any difference.

The best quality-mark appears to be the CDM Gold Standard. The Gold Standard is a recent introduction, a response to the weak, conflicting Kyoto standards and many ad hoc government ones. Gold Standard specifically excludes tree plantatations.

The following operators are the only ones I found that are Gold Standarded and also pass the bullshit smell test (which is far more stringent ;-) thanks to all who supplied links etc. — eoin

  • My Climate — Seem good. run out of Switzerland. Professional vibe. Mainly projects in the developing world.
  • Atmosfair — like the swiss one except smaller and German. Again, seems professional, their projects page in particular reads well. Doing a German schools project as well as developing world ones.
  • Climate Friendly — Aussies. Mainly wind power, in Oz & NZ. Again seem good, have been around for a few years. Website is decent if a bit all over the place.
  • Sustainable Travel International — more an eco-holidays travel agent than offsetting per se. Useful bookmark.
  • Puretrust.org.uk — These guys seem good. Interesting business model. They buy high quality carbon credits, from mainly Gold Standard providers, and retire these credits. Permanent retirement, I think, though this wasn’t 100% clear on their site. So they both support the providers directly by doing business with them, and also jack up the market price by reducing supply. This supply choke isn’t something that the rest of them do, at first glance anyway. Clever idea. As the market price gets higher it will put pressure on companies to reduce their emissions, not just buy their way out of it.’

Now it’s worth noting that this is the state of play as of May 2007; it’ll definitely change pretty quickly as time goes on. Good info, though.

Eircom broadband — it’s never easy

Published May 9, 2007

Argh, it’s never easy.

After this post, the consensus was that nowadays, Eircom have a pretty good quality of service for their DSL offerings, taking both price and service into account. I was happy enough to go with that, so I ordered their “Eircom broadband always on 2MB and Eircom talktime anytime bundle”, back around the middle of April.

I had a great call with the sales agent, Hazel. Everything went swimmingly, we were all set for the modem to be delivered and the service to be up and running in 10 working days — by May 1st April 30th. I asked for an order reference number and she said I didn’t need one, it was all handled in their system. Great!

Unfortunately it seems the call centre staff never got that quality-of-service memo.

Come May 1st, there was no sign of the modem, so I rang Eircom’s order line to see how things were going. To my horror, the staff I talked to told me that there was no record of my previous order, or call… it was as if that call had never taken place at all. No part of the order had even started.

As a result, I’ve had to reorder from scratch. The previous 10 working days we’ve waited counts for nothing. (The agents lie through their teeth about this, though — one agent says they’ll send it out in the “next 3-5 days”, the next agent insists that we have to wait the full 10 days, and the next says somewhere in between — anything to get us off the line within 4 minutes.)

This is bad news, since we’re waiting on the broadband to move in — since I work from home, we can’t move in until we have a good ‘net connection.

We can’t even make a complaint to Eircom about this fuckup, because they refuse to take complaints without the original order number to reference — the one that “Hazel” told me wasn’t needed anymore. Now that’s bureaucracy. Attempts at escalation just wound up with a dead end, where supervisors had no names and had left the office at 10am anyway. >:(

Best of all, their online complaints system now takes a maximum message length of 400 characters, so you can’t even provide a detailed written complaint online anymore. (That is, not unless you submit the complaint in 15 separate parts…)

What a fiasco.

So we now have to wait until May the 15th. We’ve submitted the complaint via the aforementioned 15 parts, and postally; if they don’t take action on those, we’ll complain to Comreg (and let’s see what that’s worth).

But here’s a question — assuming they fail to deliver the second order within time this time around, can we cancel at that stage? There’s a minimum contract length of 6 months, but since the service hasn’t been delivered, I would hope that hasn’t started yet. The terms and conditions document says:

“Ready for Service date” (otherwise “RFS date”) means the date on which eircom establishes the Facility for the Customer.

3.1 This Agreement shall commence on the Ready for Service date and shall be for the Initial Period. Provided that this Agreement has not been terminated in accordance with its terms or in accordance with the Regulations, this Agreement shall thereafter automatically renew for successive six-month periods. For the purposes of this clause 3, a six-month period will be calculated from the anniversary of the RFS date.

3.2 The Customer may cancel its order for the Facility at any time prior to the RFS date. In the event of such cancellation by the Customer it shall be obliged to return any Kit, which may have been provided to it by eircom. Any Kit shall be returned to eircom by posting it to the freepost address detailed in the welcome pack. In the event of any Kit not being returned to eircom within fourteen (14) days of the cancellation of the Order for the Facility, the Customer shall be charged by eircom and shall pay to eircom such sum as is set out in the Regulations as being the charge payable in respect of the non-return of any Kit.

So I guess as long as the facility — the ADSL line — is not up and running, I’m clear to cancel, right? It’s a little worrying that the “facility” doesn’t include the “kit” — ie. the broadband modem, though; if they fuck up sending out the modem, but the line is up, am I liable for 200 Euros?

In terms of who are viable options to switch to — in my opinion it’s got to be fixed wireless, since everyone else now would have to go via Eircom’s exchanges anyway, and be delayed there. So — Irish Broadband. I know they had some pretty massive problems 2 or 3 years ago, but recently I’ve been hearing good things about them, Boards.ie has some reasonably good-sounding recent experiences, and half of my new neighbours (srsly!) are using them with great results. Anyone got recent news about how useful they are with service quality and install speed for their Breeze product in the D9/D11 area?

Alternatively, Ripwave might make a reasonable stop-gap option? 120 euros is the minimum fee (6 months at 18.95 per month), which is better than the money I’m paying now to live in two houses…

Alternatively anyone know an Eircom engineer in D9/D11 that can nip over to the exchange and plug in my connection on the DSLAM? ;)

Moin Moin attachment spam

Published May 8, 2007

Here’s a new trick used by the web spammers — attachments on a Moin Moin wiki. The taint.org/wk RecentChanges list illustrates it well:

2007-05-07  set bookmark
[UPDATED]       UserPreferences         04:17   Info    ?StepStep [1-21]        
  #01 Upload of attachment 'big-cocks.html'.
  #02 Upload of attachment 'big-cock.html'.
  #03 Upload of attachment 'big-boobs.html'.
  #04 Upload of attachment 'big-ass.html'.
  #05 Upload of attachment 'bdsm.html'.
  #06 Upload of attachment 'bbw.html'.
  #07 Upload of attachment 'bang-bros.html'.
  #08 Upload of attachment 'bangbros.html'.
  #09 Upload of attachment 'baby.html'.
  #10 Upload of attachment 'asian-porn.html'.
  #11 Upload of attachment 'asian-girls.html'.
  #12 Upload of attachment 'anime-porn.html'.
  #13 Upload of attachment 'anime-girls.html'.
  #14 Upload of attachment 'angelina-jolie.html '.
  #15 Upload of attachment 'amature.html'.
  #16 Upload of attachment 'amatuer.html'.
  #17 Upload of attachment 'adult-videos.html'.
  #18 Upload of attachment 'adult-stories.html' .
  #19 Upload of attachment 'adult-games.html'.
  #20 Upload of attachment '69.html'.
  #21 Upload of attachment '3d.html'.

Great. Lots of spam. This first started appearing on Feb 27 2007, in a multi-upload attack on a single page (“FindPage”), from IP address 212.26.129.162; then reoccurred on Apr 27 and May 7 from the (insecure open proxy) proxy.drevlanka.ru.

Annoyingly my “subscribe to wiki changes” patch doesn’t catch this — these aren’t gatewayed through as “changes” via mail for review. I need to fix that in my copious free time. :(

Also, the RecentChanges RSS feed doesn’t list them, although the HTML form does.

So unfortunately, the only way I can see to block this is either to review by visiting the RecentChanges page in a web browser regularly (how retro!), and delete them retrospectively, or simply to turn off attachments entirely — which is what I’ve done, by editing “wikiconfig.py” and adding:

    actions_excluded = ['AttachFile']

It looks like quite a few other wikis around the web are running into the issue too :(

SpamAssassin 3.2.0!

Published May 2, 2007

W00t! SpamAssassin 3.2.0 has finally gone gold!

This release is a big one — it’s the first major release since 3.1.0, back in September 2005, just over a year and a half ago. Here is the release announcement mail, containing a list of major changes since version 3.1.8. There are a few major new features that I feel worth picking out in more detail and editorialising about:

sa-compile

This is a biggie. This new script takes the active SpamAssassin ruleset, and uses code contributed by Matt Sergeant to produce input for re2c. re2c in turn compiles the ruleset into a deterministic finite automaton, which can match multiple regular expressions in parallel. That’s not all, though; re2c then compiles that DFA into C code — which is then compiled into native object code. SpamAssassin will then load that object code and use it to replace the slower perl regexp tests, if it’s available at scan-time.

Now, it’s been a long time since SpamAssassin’s ruleset consisted mainly of rudimentary regular expressions matched against the body text — a good portion of SpamAssassin’s ruleset these days operates against headers, performs network lookups, analyzes URLs extracted from the body, uses the more advanced features supported by Perl’s NFA regexp engine, or so on. But even given that, the effects of ‘sa-compile’ seem to average between a 15% and 25% speedup, in my testing. That’s good ;)

Many of the commercial versions of SpamAssassin include their own body-rule speedups — but this is the first time anything similar has made it into the open source code.

Short-circuiting

Another good one for performance. There are some rules that you can reasonably assume will never hit nonspam or spam mail in a well-configured setup. For example, a hit on “ALL_TRUSTED” should mean that the message never traversed an untrusted network, therefore it cannot be spam, so why bother applying the expensive tests? It should be reasonable to “short-circuit” and immediately return a “ham” score for that mail.

This new plugin implements that algorithm — and efficiently, too, which historically has been the hard part!

I’ve been using this for a while with a ruleset like this one — in my experience, it’s cut overall CPU time spent scanning mail by 20%.

It is pretty flexible, too — there’s lot of tweakage that can be done with this functionality to suit your own setup.

Reduced memory footprint

One aim of this release has been to reduce the memory usage of SpamAssassin; the core code now uses less RAM than 3.1.x does, when tested with the same ruleset. (Unfortunately we’ve added lots more rules in the interim, so it’s a bit of a wash overall. ;)

The VBounce anti-bounce ruleset

Detects spurious bounce messages sent by broken mail systems in response to spam or viruses. More info about that here.

Apache-spamd

apache-spamd implements spamd as a mod_perl module. This was contributed by Radoslaw Zielinski, as a Google Summer of Code project last year. Thanks Radoslaw!

There are plenty more new, useful features and rules — these are just the top ones, in my opinion. Pretty cool stuff!

Patricia McKenna and MMR, again

Published May 1, 2007

Great! Patricia McKenna just called around, canvassing our area — and just got a serious telling off from the wife ;)

Catherine — unsurprisingly, given that she’s a zoology Ph.D — was fantastic, hitting every key point of the issue: that we’re both long-time Green voters who’ve been forced to not vote Green this time around, due to this MMR issue and the anti-science/pro-hokum angle it represents.

Interestingly, she claimed that her stance on MMR was always her own point of view, and that it wasn’t party policy — and that it was mentioned on the party website was a rumour put about by the PDs.

While it turns out that Dr. Ruairi Hanley, the author of this letter to the Indo is indeed a PD (didn’t realise that!), Treasa at Winds and Breezes also noted it appearing on the Green Party site, as follows:

Questioning the Benefits of Immunisation

There are significant question marks about the effectiveness of mass immunisation programs. We would launch a major study of the benefits of these programs looking at all aspects of health

So Treasa — are you a stealth PD rumour-monger? ;)

Worth noting that at no time did McKenna reassure C that her policy would not become government policy if the Greens were elected… as an elected representative, surely her own policies would influence the government’s thinking?

Screenclick devolve again

Published May 1, 2007

After a short period where things were looking up, Screenclick have once again reverted to type, by ditching the lovely simple Netflix-style queue they seemed to be using, and instead instituting some new kind of bizarre homebrew wierdness.

It looks like a queue, with a line-by-line listing of movies — but then beside each title, there are 3 radio buttons: “High”, “Medium”, and “Low”.

The instructions run as follows:

All titles are sorted in alphabetical order within their priority group
  • – High: Please deliver these titles as soon as possible
  • – Medium: Please deliver these titles as they become available
  • – Low: I don’t mind when you send these titles

So what — does this mean that if I put a title in as “High”, I’m going to receive it next, or not, or what? and what’s with the alphabetical order? WTF is going on? argh.

Anyway, I just got out “Amores Perros”, presumably due to this alphabetical ordering thing. not what I wanted at all. What a mess.

A week of Bertiespam

Published May 1, 2007

We’re in the run-up to a general election here in Ireland, and I live in Bertie’s constituency. For the past year or so, things have been pretty quiet, but in the last week there’s been a sudden flurry of activity and direct postal mail from Bertie’s office — and from many departments of local government, too:

Mon Apr 23:

  • Fianna Fail: “Fianna Fail delivers on education in Dublin Central”, tabloid newspaper.

  • direct from the office of Bertie: a photocopied letter from the Environmental Health Officers of Dublin City Council about the standards of rented houses “in my area”.

Tues Apr 24:

  • HSE: “Parents Who Listen, Protect” leaflet, a full-colour glossy handbook “on building good communication in families and communities” “as part of a national initiative on child protection”.

  • Dept of Environment: a leaflet on the “National Climate Change Strategy, 2007-2012, Main Points”. Printed on recycled paper, naturally ;)

Fri Apr 27:

  • Fianna Fail Senator Cyprian Brady: “dear resident, please vote for me” — one-page full-colour glossy.

  • Spring 2007 “Central News”, “Official Voice of Fianna Fail in Dublin Central”, a 16-page tabloid newspaper, featuring stories like “Smithfield: the Temple Bar of the Northside” (like Temple Bar, but with more winos and Children’s Court, and less stuff!)

Mon Apr 30:

  • HSE: “Need a doctor urgently? Call D-DOC out-of-hours GP service”, full-colour glossy leaflet.

  • from Bertie: Evening of Election Letter. “Good evening constituents” etc.

It’s a veritable flood of full-colour glossies! Could be worse, I suppose — I hear the PDs have been blanketing selected Dublin constituencies in free books. However I suspect grimy Dublin 7 is a little off their list (see “winos”, above).

It’s worth noting that a good half of this flood (which I’ve coined Bertiespam to describe) isn’t from Bertie’s constituency office — it’s from government departments like the HSE and the Department of Environment. It’s funny that we hadn’t heard a peep from them all year, then once an election looms — “here come the voters! look busy!” ;)

What bertiespam have you been getting?

Hog’s Chip

Published April 27, 2007

Hey Google —

Since Fido.ie is throwing errors at me, and since you’re probably a more searchable (and more global) database anyway — the Trovan FDX-B RFID transponder number 956000000659388 is that of “Hog Dempsey”, a small female black and white cat, whose owners can be contacted via any address on this page. Cheers!

HOWTO do a DOS-based BIOS upgrade without Windows

Published April 23, 2007

Wow, I can’t believe I still have to do this in 2007 — Taiwan really needs to discover FreeDOS! Here’s how to run a DOS BIOS update on a PC without using Windows (in my case, it’s a Dell laptop).

  gunzip FDSTD.288.gz
  sudo mount -t msdos -o loop `pwd`/FDSTD.288 /tmp/bootiso
  • ensure there’s enough space, and copy the app into the disk image:
  df /tmp/bootiso
  sudo cp ME051A10.EXE /tmp/bootiso
  • Then make an ISO, using mkisofs’ “-b” option to ensure it’s bootable:
  mkdir /tmp/floppycopy
  cp -Rp /tmp/bootiso/* /tmp/floppycopy
  cp -p FDSTD.288 /tmp/floppycopy
  mkisofs -pad -b FDSTD.288 -R -o /tmp/cd.iso /tmp/floppycopy
  • And burn it:
  sudo umount /tmp/bootiso
  sudo cdrecord dev=0,0,0 -pad -v -eject /tmp/cd.iso
  • Now, take the burned CDROM, and boot it.

Answer “N” to all questions when booting, otherwise you’re likely to see an error like “Cannot operate in Protected environment” when you run the BIOS update.

Thanks to the Motherboard Flash Boot CD from Linux Mini HOWTO; very helpful. I hope the next time I have to do this, they just issue a bootable ISO image instead…

Update, Sep 2013:

Wayno Guerrini emailed to say: ‘I used your recipe to update the bios on a old Dell Dimension 8400. Worked like a champ, with a couple of modifications. I am running 64 bit debian wheezy.

apparently the mkisofs has been replaced by genisoimage. Syntax the same.

instead of cdrecord I had to use wodim: sudo wodim dev=/dev/sg1 -pad -v -eject /tmp/cd.iso

Thank you. Recipe worked very well. I will point people to this article, but add the changes as appropriate to my website.’

Using qpsmtpd for traps.spamassassin.org

Published April 17, 2007

Like many anti-spam systems these days, SpamAssassin operates a network of spamtraps. One set of these run off traps.SpamAssassin.org, a server kindly donated by ISP Sonic.net.

Large-scale spam-trapping systems like this are generally run in quite a secretive manner, but we’re an open source project — so it may be interesting if I give some details of our setup. Here’s a potted history of how this spamtrap server has run over the years…

The beginning

The architecture was initially very simple. The MX was Postfix, delivering to the “trapper” user, which in turn ran procmail, which directly ran a perl script. This perl script then performed the trap actions, namely: DoS prevention, discarding viruses and malware, discarding backscatter bounces, extraction and cleanup of the incoming mails, then onward reporting, archival, and further distribution.

Given that this was a target for spam — and we want as much spam as possible here! — this would predictably run into load issues. Right at the beginning, back in around 2001/2002, I ran this on our shared server, where it pretty quickly caused trouble for delivery of other, more useful mail. It was around this time that Sonic kindly donated the server.

With dedicated hardware, we weren’t seeing much trouble — it was enough to just wait for the few hours for a traffic spike to pass, and the Postfix queue would then clear.

Clearing the queues

After a few months, though, this wasn’t enough — the queue would get consistently clogged, and the backlog became enough to result in the incoming spam being delayed for days before it made it from the MX to the trap archives. For a spamtrap, you want fresh spam, but not necessarily all spam — so I installed a cron job to simply clear the queue on a nightly basis. (I also had to restart the Postfix server, too, since it’d occasionally get hung and stop accepting connections on port 25, presumably due to load issues.)

IPC::DirQueue

The next level was an inability of the procmail/perl script end to process the mail fast enough for the MTA to keep up with the incoming connections, and follow-on problems, caused by load generated by the perl script impacting the MX’s activity. To work around these, I designed a new queueing backend, based around IPC::DirQueue. This allowed a new split architecture; the procmail-run perl script was extremely lightweight, delivering all inbound mail to a dirqueue and exiting quickly, allowing the MX to get back to the next inbound spam message, and the trap processing script was then split into a web of dirqueues, allowing each individual part of the trap backend pipeline to operate independently.

There were several benefits to this:

  1. Since dirqueues operate as a batch-processing model, load spikes become irrelevant; the load incurred is limited by how many dequeuer processes are run.
  2. The time taken in backend tasks becomes irrelevant to the MX throughput, since that is bottlenecked only by the lightweight perl script and its write speed to the “incoming” dirqueue.
  3. By splitting the backend work into multiple queues, outages in the spam-reporting systems or onward forwardings become much less of a problem, since they won’t affect inbound spam, archival, outbound delivery to other reporting systems, forwards, etc.

Again, the dirqueues were cleared on a frequent basis, to discard the “spiky” traffic and ensure we were just seeing samples of the freshest spam. The dirqueues use a tmpfs as the backing storage directory, so it never hits the disk at all.

This worked pretty well for several years — from 80 megabytes of spam per day to the current level, which is around 130MB per day. However, we still occasionally saw problems from load spikes, where high load caused the traps to refuse incoming SMTP connections — purely because the load of inbound connections is too high for the Postfix MX to accept them all in a timely fashion.

qpsmtpd

Last weekend, I had a go at a project I’d been thinking of trying out for a long time — switching from Postfix to qpsmtpd. A while back, Matt Sergeant rewrote qpsmtpd to use Danga::Socket, Danga Interactive / Six Apart’s insanely scalable event-driven asynchronous socket class, as used in mogilefsd, perlbal and djabberd. This article notes that ‘two large antispam companies’ high-traffic spam traps have used this effectively since the second quarter of 2005, delivering concurrency as high as 10,000 on some occasions’, so it seemed likely to work ;)

Sure enough, results have been great… we now have a pure-perl system handling heavy volumes without breaking a sweat, certainly compared to the previous system. qpsmtpd’s plugin system was elegant, allowing me to annotate inbound spam with more details of the SMTP transaction, write plugins to deliver mail to a dirqueue directly instead of to an MTA, and do some conditional code (ie. basic “deliver this RCPT TO to this queue”) where needed.

Full details are over on the QpsmtpdSpamtrap page on the taint.org wiki, for the curious.

Don’t worry about Blacklist.ie

Published April 17, 2007

Irish techies — wondering what the next website to put the fear into your parents will be? Here it is: Blacklist.ie. It’s been getting a bit of coverage from the Irish technology press recently, it seems, as the new site from IE Internet.

(IE Internet are the Irish internet company that puts a press release every month or so telling us how much of their mail is being filtered as spam, which Silicon Republic et al dutifully report as news, month after month.)

I got a call from my mother last week, telling me that she’d been “blacklisted”, and asking how to fix it. Sure enough, when I found out that she’d heard this on blacklist.ie, I went to the site, and her IP address was indeed listed — as was mine:

The IP address 212.2.169.61 is blacklisted.

RBLs checked:

Spam Haus not listed

Spam Cop not listed

Mailwall RBL not listed

Abuse At not listed

SORBS not listed

NJABL listed: Dynamic/Residential IP range listed by NJABL dynablock – http://njabl.org/dynablock.html

510 SG not listed

Naturally, that IP is listed — it’s entirely ok for a home-user broadband machine to appear in SORBS or NJABL as a dynablock-listed IP. (Dynablock, for those who don’t know, is a set of records for addresses which are known to be residential/end-user “dynamic” addresses, rather than mail relays — so obviously most end-user desktop machines would fall under this category.)

Unfortunately, this distinction isn’t mentioned anywhere on the blacklist.ie page… just a large, red, “The IP address is blacklisted” warning.

Worried readers might then reasonably go on to read the site’s Frequently Asked Questions list — which, incredibly, includes a helpful suggestion that you sign up with IE Internet to avoid being listed in future! I’d be curious how that’s supposed to help a home user get off the NJABL dynablock list… a little fishy, if you ask me!

Bar Camp Dublin next weekend

Published April 12, 2007

Dublin hackers/software people — don’t forget! Bar Camp Dublin is happening on April 21st — that’s 9 days from now.

It should be interesting — there are 93 attendees signed up already, and I see a good few familiar names I haven’t run into in a while! The last Bar Camp was a good opportunity to meet up for some very informal talks, and this looks likely to be the same.

Sign up here, go on…

Screenclick improve their site

Published April 11, 2007

Yay! They now have a proper queue! Also member reviews and other improvements — it seems a lot better.

Can’t figure out how to change my password, though ;)

Don’t vote Green in Dublin Central!

Published April 10, 2007

I’ve long held green views, and have always voted green — I believe climate change, damage to the environment and pollution are extremely serious problems, especially for Ireland. At the same time, I also believe that science and technology has a key place in a better, greener future — a Viridian, bright green / electric green viewpoint, in other words.

Given this, I was really shocked and appalled to hear (via the lovely C) of an interview on Today FM with Patricia McKenna, a Green Party candidate for my local constituency of Dublin Central — one I’ve voted for before, no less! — in which she revealed that she believes in the thoroughly discredited scaremongering regarding a link between the MMR vaccine and autism, and has taken the appallingly irresponsible position of not allowing her children to be vaccinated.

This blog post discusses the interview, which was broadcast on Today FM’s The Last Word show on Tuesday 13 March. Here’s an archived podcast of that interview so you can listen to it yourself, and here’s a local copy of that WMV file in case that first link expires any time soon.

Here’s a transcript of the part of the interview once the issue of vaccination is brought up. Matt Cooper is the host of the show. Keith Redmond is an opposing candidate, for the PDs. The timestamps are in minutes and seconds from the start of the audio file.

  • 8:30: Patricia McKenna: Parents have the right to choose what they opt to do, and in relation to some vaccinations, there are serious question marks hanging over them but that’s not what we’re talking about here…

  • 8:44: Matt Cooper (clearly annoyed): No its not, but now that it’s up there, couldn’t it be irresponsible for parents not to vaccinate children against serious issues (sic), if they don’t have reputable scientific facts to back up the decision not to vaccinate?

  • 8:54: Patricia McKenna: Many parents in this country have chosen not to vaccinate their children in relation to the MMR because of the links to autism.

  • 9:00: Matt Cooper: Utterly untrue, totally unproven, absolutely bogus and false.

  • 9:02: Patricia McKenna: Hold on a second…

  • 9:03: Matt Cooper: Andrew Wakefield has been utterly and totally discredited in relation to that. Anyone who doesn’t give the MMR vaccine to their children because of a fear of autism is almost in danger of endangering their child themselves. We’re going to have a rise of measles again in this country because of people not actually giving the vaccine.

  • 9:17: Patricia McKenna: First of all, we’re moving away from the issue…

  • 9:22: Matt Cooper: Yeah we are, but it’s come up now, let’s deal with it…

  • 9:23: Patricia McKenna: It’s come up, right. Eh, have you had the measles? I’ve had the measles, and I’ve got over them well, I have a strong immune system, my 10 year old son has had the measles…

  • 9:30: Matt Cooper: And you are aware that unhandled the measles can have very serious side effects?

  • 9:33: Patricia McKenna: Look — the side effects that are linked to the measles are in relation to… there are other things linked to it in relation to the child’s well being initially. Now you just look at the number of people when you were young, all of your peers I would say have had the measles as with mine, and I think we have a tendency to over-indulge in vaccinating our children and vaccinating ourselves, because what we need — our immune systems are getting weaker and weaker by the day, it’s a — I think we need to be very careful about how we actually approach this so that when medicines are necessary, we will not be immune to them…

  • 10:08: Matt Cooper (interrupting): Do you know that children have died of the measles in this country in the last 5 years?

  • Keith Redmond: because of views like that.

  • Patricia McKenna: Well I’m saying is that, as far as I’m concerned…

  • 10:18: Matt Cooper (repeats): Do you know that children have died of the measles in this country in the last 5 years?

  • 10:30: Patricia McKenna: The children that have died of the measles because of other complications (sic), not the measles themselves.

  • Keith Redmond: that have not been vaccinated.

  • Patricia McKenna: Not the measles themselves, but other complications, right? Now if you’re saying that parents should — it’s a bit like —

  • Keith Redmond: Matt, can I just come back to…

  • 10:32: Matt Cooper: Sorry, one second Keith. Would you also concede Patricia, that there is absolutely no link between the MMR and autism, that that link was a bogus link put up by Andrew Wakefield who has been completely and utterly discredited and it has done an awful lot of damage, the misrepresentation of his views in relation to the MMR and autism.

  • 10:50: Patricia McKenna: Well in relation to the MMR, I am not satisfied that it’s safe, and I am not satisfied with the idea of lumping a whole lot of vaccines — different vaccinations together en masse, inducing them (sic) to our children — but having said that, parents should have the right to choose and decide what is best for their children…

  • 11:06: Matt Cooper: But would you concede that Andrew Wakefield, who is the man that pushed that whole agenda, was exposed as a fraud?

  • 11:11: Patricia McKenna: But the jury is still out in relation to…

  • 11:15: Matt Cooper: No, it’s not.

  • 11:16: Patricia McKenna: Yeah well I’m sorry but the jury is still out in relation to how safe the MMR is. And I think it’s unfair to label all parents who decide for their own children’s safety, that they may not want to go down the route of vaccination, that they’re being irresponsible, because I wouldn’t consider myself irresponsible, I would consider I want what’s best for my child.

  • 11:37: Keith Redmond: [again says something]

  • Matt Cooper: Give Keith a chance to come in.

  • 11:41: Keith Redmond: This totally exemplifies the Greens’ approach to any kind of science. We have a woman there who knows, in her heart of hearts, that her argument is wrong but refuses to admit it because it relies on science. Now, we have exactly the same issue with flouridation — we know the science, we know the facts, and we still have this scaremongering every now and again. And the Green Party are totally irresponsible and you’re right, they are frightening parents across the country right now and it’s absolutely reprehensible.

My god, this insanity has me agreeing with a feckin’ PD!

This is luddism, pure and simple. Matt Cooper is spot on the money — children are dying in Dublin because of this “my child, my rules” selfishness and simple inability to understand the science surrounding vaccination as a public health policy.

This is appalling. To put it bluntly, there is no fucking way I’ll be voting Green if this kind of cargo-cult, anti-science superstition is the kind of shite they’re espousing these days. …and if you think I’m feeling strongly about this, you should hear my (zoologist) wife.

But it goes on — here’s a letter to the Irish Independent on this issue from Feb 9 2007, which raises another worrying factor:

… until two days ago, there was a statement on the Green Party website informing voters that there were “serious question marks about the benefit of mass vaccination programs”.

Furthermore, the party promised that there would be a “major review” of vaccination if they were returned to office.

Now that these statements have apparently been removed from the Green party website are we to take it that they are no longer Green policy?

This blog posting at Winds and Breezes also notes this. So — is this official Green policy or not?

Update: In the comments, it was noted that McKenna is pretty much acting alone in this; it, apparently, is not Green Party policy at all. I’ve updated the title to reflect that it’s only one constituency’s candidate that needs to be shunned.

Also, Conor O’Neill has a great idea over here:

I was thinking further on this yesterday and I realised what the Greens need to do in order to be taken seriously… They need to become the “Party of Science”. Proper environmentalism is based on rigorous science and strategic thinking. Every policy they define should be backed up with rock-solid science and a detailed long-term financial analysis proving why it is in our best interests to adopt them.

Man, I would love to see that!

Eircom broadband?

Published April 6, 2007

I’m moving house. Naturally, first priority after getting the keys is getting the broadband set up ;)

Current broadband: BT DSL. Supposedly “up to” 3Mbps — however, as with most DSL connections in Ireland, it’s rate-adaptive RADSL, which means it trades off connection speed against distance to exchange and line quality.

Sadly, this has really deteriorated since the last time I checked! A “bing” test between the BT-supplied DSL router and the far end looks like this:

BING    10.18.72.1 (10.18.72.1) and 193.95.142.243 (193.95.142.243)
        44 and 108 data bytes (1024 bits)
193.95.142.243: minimum delay difference is zero, can't estimate link throughput
193.95.142.243:  6.966Mbps 0.147ms 0.143555us/bit
193.95.142.243: minimum delay difference is zero, can't estimate link throughput
193.95.142.243: 19.692Mbps 0.052ms 0.050781us/bit
193.95.142.243:  4.697Mbps 0.218ms 0.212891us/bit
193.95.142.243:  3.261Mbps 0.314ms 0.306641us/bit
193.95.142.243:  3.170Mbps 0.323ms 0.315430us/bit
193.95.142.243:  2.479Mbps 0.413ms 0.403320us/bit
193.95.142.243:  2.723Mbps 0.376ms 0.367187us/bit
193.95.142.243:  2.688Mbps 0.381ms 0.372070us/bit
193.95.142.243:  2.716Mbps 0.377ms 0.368164us/bit
193.95.142.243:  2.065Mbps 0.496ms 0.484375us/bit
193.95.142.243:  1.984Mbps 0.516ms 0.503906us/bit
193.95.142.243:  1.270Mbps 0.806ms 0.787109us/bit
193.95.142.243:  1.017Mbps 1.007ms 0.983398us/bit
193.95.142.243:  1.002Mbps 1.022ms 0.998047us/bit
193.95.142.243:  1.008Mbps 1.016ms 0.992187us/bit
193.95.142.243: 983.670Kbps 1.041ms 1.016602us/bit
193.95.142.243: 993.210Kbps 1.031ms 1.006836us/bit
193.95.142.243: 987.464Kbps 1.037ms 1.012695us/bit

--- 10.18.72.1 statistics ---
bytes   out    in   dup  loss   rtt (ms): min       avg       max   std dev
   44   762   758          0%           2.524     3.858    19.083     2.194
  108   762   762          0%           2.639     4.187    58.273     3.079

--- 193.95.142.243 statistics ---
bytes   out    in   dup  loss   rtt (ms): min       avg       max   std dev
   44   762   761          0%          13.061    20.025    78.689     8.226
  108   762   760          0%          14.213    17.954    61.137     4.697

--- estimated link characteristics ---
host                              bandwidth       ms
193.95.142.243                      987.464Kbps      10.536

987Kbps is not 3Mbps any more, not by a long shot. I’d say I now have a lot of new friends adding contention at the ol’ DSLAM. I’m paying way too much money for what I’m getting :(

(Update: actually, it may not be contention. Judging by boards.ie traffic, high-contention situations in Ireland are usually faster in the mornings and daytime, then slower from 4pm-9pm as the commuters and kids get home — however, this slowdown is pretty consistent across all times of day.)

(Update 2: as of right now, late afternoon on Apr 12, it’s the worst I’ve seen it — packet rates of 600Kbps, and packet loss of 5%-20%.)

On top of this, they have the really annoying daily disconnection policy, which I have hacked around with IPv6 and a VPN, but which still manages to waste my time and cause aggravation, even after frickin’ months of pissing about.

For this, and the packaged phone service, I’m paying just under EUR 60 per month, including all call charges and VAT.

At that price, Eircom are offering a pretty good bundle — free connection, free modem, 2Mbps downstream, 256Kbps upstream, unlimited free local and national calls at all times, 5% off calls to mobiles, 10c/min calls to the UK and US.

Now, a drop to 2Mbps may seem a lot, but bear in mind I’m getting just under 1 right now! I’m pretty sure the new gaff will have similar-quality lines and exchanges. Also, if I get the 2Mbps line, and the attenuation and S/N statistics indicate that it can support 3Mbps, I can always upgrade pretty easily.

The only problem now is getting over my revulsion at buying from Eircom, ugh…

Am I missing something? Does that Eircom bundle not include line rental maybe?

About the title change

Published April 5, 2007

The eagle-eyed may have spotted a change that took place a month or two ago in the taint.org configuration — I ditched the old weblog tagline.

Previously, this weblog was titled “taint.org: Happy Software Prole”. This title had been in place since around October 2003, when Daniel Lyons wrote a particularly idiotic article for Forbes entitled “Linux’s Hit Men”, which I took umbrage to:

Here we go again — the old ‘free software is communism’ line […] The article goes on to bemoan how software companies who write proprietary extensions into GPL-licensed software, have to comply with the terms of the license. It’s all a bit of an obvious dig — but I am looking forward to the follow-up article — that’s the one where the author bemoans how commercial software companies send out their ‘enforcers’ to extort money from companies who don’t bother paying the royalties and runtime license fees their licenses require.

As an free/open-source-software guy, I happily adopted ‘happy software prole’ as an absurd tagline, in the spirit of detournement. Fast-forward to 3.5 years on, however, and I’d say most people can’t even remember the Forbes article, or that Daniel Lyons guy! So that tagline was a bit old and busted, really.

On top of this, I’d noticed something I do in my weblog reading — I’ve started renaming blogs in the feed reader from their fancy title, to simply the name of the author.

I’ve found that when reading blogs, I’m interested in who’s writing. When skimming through the feeds of a morning, having to spend 5 seconds to recall that “ByteSurgery.com” is Robin Blandford is just a wee bit superfluous, sorry Robin. ;)

As a favour for readers, I’ve saved them the trouble, and renamed the blog to be quite explicit about who’s writing; the taint.org tagline is now just “taint.org: Justin Mason’s Weblog”. Let’s face it — it’s a bit functional. Hopefully it’s helpful, though!

(And finally, it gives me the edge in the ongoing Google war against the non-me “Justin Masons” out there… and against a heart surgeon and a Texan basketball player, I need it. ;)

A recycling puzzle

Published April 4, 2007

Myself and Tom were in a taxi last night, stopped at a stop light, when I noticed something odd.

A girl, about 20 or so, walking along the path stopped beside a bag of recycled rubbish, and bent over as if she was tying her shoelace. Instead of fixing her lace, though, she quickly ripped a hole in the (transparent) plastic bag, grabbed a crumpled Fanta can, and walked off.

WTF? anyone got any theories?

Coworking.IE

Published April 3, 2007

Coworking.ie is a new community-driven coworking group-blog and promotion site, set up by Jason Roe.

Coworking’s a pretty cool idea — ‘a movement to create a community of cafe-like collaboration spaces for developers, writers and independents.’ Great news for us teleworkers.

I’ve subscribed — it’ll be interesting to track development of this concept, in Ireland and elsewhere…

New list for Irish users of MythTV

Published March 28, 2007

MythTV is a pretty great product, once you get it working — however, it can be labour-intensive, involving lots of local knowledge to deal with the ins and outs of each area’s TV provider, cable service, etc.

To that end, we’re recently set up a new mailing list: mythtv-ireland, a list for discussion of topics of interest for MythTV users in Ireland.

Particularly on-topic:

  • the NTL frequencies list for areas in Ireland

  • hacks to scrape the Channel 6 schedule from their website

  • dealing with the NTL Digital set-top box

Sign up, if you’re interested!

Twitter and del.icio.us

Published March 23, 2007

Walter Higgins says:

It’s just occurred to me why I don’t like twitter – It doesn’t fulfill any need that isn’t already fulfilled by del.icio.us. I usually post a note alongside each bookmark which lets me micro-blog (post short comments without having to think too much). If I want to signal to someone to take a look at the bookmarked item I just tag it for:[nameofperson] which I suppose you could loosely call ‘chat’. Since I gave up personal blogging, del.icio.us has fulfilled a need for short-hand blogging. Thinking about it – twitter is like del.icio.us but without the bookmarks – viewed in that light it really is hard to understand why anyone would use twitter.

To my mind, though, there’s a big difference:

  • My del.icio.us page is where I post things I’m reading, and things I think others may be interested in reading;

  • My twitter page is where I post things I’m doing, and chat.

There’s no way I’d try to hold a conversation in my del.icio.us bookmarks! ;) Different tools for different uses.

Geeking out on the ‘leccy bill

Published March 20, 2007

A good post from Lars Wirzenius on measuring the electricity consumption of his computer hardware. Here’s a previous post of mine on the subject.

With the rising cost of energy, a keenness to reduce consumption for green purposes, and an overweening nerdity in general, I did some more investigation around my house recently.

I have a pretty typical Irish electricity meter; it contains a visible disc with a red dot, which spins at a speed proportional to power usage. (There’s a good pic of something similar at the Wikipedia page).

The fuse-board works out as follows (discarding the boring ones like the house alarm etc.):

  • Fuse 7 – gas-fired central heating (on), fridge (on), kitchen power sockets

  • Fuse 8 – TV in standby, idle PVR, Wii in standby, digital cable set-top box, washing machine

  • Fuse 9 – telephone, DSL router, Linksys WRT54G AP/router

  • Fuse 10 – bedroom sockets, home office with laptop, printer, speakers, laptop-server etc.

The approach was simply to turn off the house fuses at the fuse board, one by one, and measure how long it took the disc to make a full revolution; then invert that (1/n) to convert from units of time over a static power value, to some notional unit of power consumption over a static time interval (I haven’t figured out how to convert to kW/h or anything like that, they’re just makey-uppy units).

Fuses Time/power Power/time
Baseline (all fuses on) 22.71 seconds 0.0440
Fuse 7 off 43.03 0.0232
Fuses 7 and 8 off 57.92 0.0172
Fuse 7, 8 and 9 off 84.88 0.0117
Fuse 7, 8, 9, and 10 off ~20 minutes (I’d guess) 0.0008?

(I stopped measuring on the last one and just estimated; it was crawling around.)

Breaking out the individual fuses, that works out as:

Fuse Power/time
Fuse 7 (central heating, fridge, kitchen bits) 0.0208
Fuse 8 (TV, Wii, set-top box, washing machine) 0.0060
Fuse 9 (phones, routers) 0.0055
Fuse 10 (home office, bedrooms) 0.0109

Good results already: (a) it was pretty clear that fuse 7 was doing all the quotidian legwork, eating the majority of the power, and (b) the TV equipment and internet/wifi infrastructure was pretty good at low-power operation (yay). However (c) the computer bits aren’t so great, but still only half the power consumption of the kitchen bits.

Breaking down the kitchen consumption further:

Appliances Time/power Power/time
Gas central heating on (rechecking the baseline) 20.46 0.0488
Gas central heating off 34.15 0.0292
Washing machine on (40 degree wash) 13.65 0.0732
Dishwasher on 2.53 0.3952
Dishwasher and dehumidifier on 2.53 0.3952

Subtracting the baseline:

Appliance Power/time
Gas central heating 0.0196
Washing machine 0.0244
Dishwasher 0.3464
Dishwasher and dehumidifier 0.3464

So the central heating, despite being supposedly gas-fired, eats lots of power! I guess this is the electric pump, used to drive the heated water around the house to the radiators. Ah well, I’m not skimping on that ;)

More practically: the dishwasher result is incredible. That’s 30 times the power usage of the house’s computer hardware. This is a ~7-year-old standard dishwasher; obviously green power consumption wasn’t an issue back then! We’re running it less frequently now, obviously; the odd hand-wash of bulky and nearly-clean items helps. With any luck when we move in a few months, we can replace it with a greener model.

The washing machine is about what I would expect, so I’m OK with that.

Also interesting to note that our dehumidifier is unnoticeable in the volume of the dishwasher; I could have tried to work it out properly in isolation, but couldn’t be bothered by that stage ;)

Sender Address Verification considered harmful

Published March 16, 2007

(as an anti-spam technique, at least.)

Sender-address verification, also known as callback verification, is a technique to verify that mail is being sent with a valid envelope-sender return address. It is supported by Exim and Postfix, among others.

Some view this as a useful anti-spam technique. In my opinion, it’s not.

Spam/anti-spam is an adversarial “game”. Whenever you’re considering anti-spam techniques, it’s important to bear in mind game theory, and the possible countermeasures that spammers will respond with. Before SAV became prevalent, spam was often sent using entirely fake sender data; hence the initial attractiveness of SAV. Once SAV became worth evading, the spammers needed to find “real” sender addresses to evade it. And where’s the obvious place to find real addresses? On the list of target addresses they’re spamming!

Since the spam is now sent using forged sender addresses of “real” people, when a spam bounces (as much of it does), the bounce will be sent back not to an entirely fake address, but to a spam recipient’s address.

Hence, the spam recipients now get twice as much mail from each spam run — spam aimed at them, and bounce blowback from hundreds of spams aimed at others, forged to appear to be from them.

This is the obvious “next move” in response to SAV, which is one reason why we never implemented something like it in SpamAssassin.

On top of this — it doesn’t work well enough anymore. Verizon use SAV. Have you ever heard anyone talk about how great Verizon’s spam filtering is? Didn’t think so.

(This post is a little late, given that SAV has been used for years now, but better late than never ;)

By the way, it’s worth noting that it’s still marginally acceptable to use SAV as a general email acceptance policy for your site — ie. as a way to assert that you’re not going to accept mail from people who won’t accept mail to the envelope sender address used to deliver it. Just don’t be fooled into thinking it’s helping the spam problem, or is helping anyone else but yourself.

Finally, this Sender Address Verification is different from what Sendio calls Sender Address Verification. That’s just challenge-response, which is crap for an entirely different, and much worse, set of reasons.

Something in the oven

Published March 12, 2007

Check out what’s cooking chez Mason:

Thrills and spills! I may have to cut down on the extra-curricular activities for a while, so we’d better get SpamAssassin 3.2.0 released before August 21st ;)

Spam volumes at accidental-DoS levels

Published March 6, 2007

Both Jeremy Zawodny and Dale Dougherty at O’Reilly Radar are expressing some pretty serious frustration with the current state of SMTP. I have to say, I’ve been feeling it too.

A couple of months back, our little server came under massive load; this had happened before, and normally in those situations it was a joe-job attack. Switching off all filtering and just collecting the targeted domain’s mail in a buffer for later processing would work to ameliorate the problem, by allowing the load to “drain”. Not this time, though.

Instead, when I turned off the filtering, the load was still too high — the massive volume of spam (and spam blowback / backscatter) was simply too much for the Postfix MTA. The MTA could not handle all the connections and SMTP traffic in time to simply collect all the data and store it in a file!

Looking into the “attack” afterwards, once the load was back under control, it looked likely that it wasn’t really an attack — it was just a volume spike. Massive SMTP load, caused by spammers increasing the volume of their output for no apparent reason. (Since then, spam volumes have been increasing still further on a nearly weekly basis.)

This is the effect of botnets — the amount of compromised hosts is now big enough to amplify spam attacks to server-swamping levels. Our server is not a big one, but it serves less than 50 users’ email I’d say; the user-to-CPU-power ratio is pretty good compared to most ISPs’ servers.

So here’s the thing. New SMTP-based methods of delivering nonspam email — whether based on DKIM, SPF, webs of trusted servers, or whatever — will not be able to operate if they have to compete for TCP connection slots with spammers, since spammers can now swamp the SMTP listener for port 25 with connections. In effect, spam will DDoS legitimate email, no matter what authentication system that legit mail uses to authenticate itself.

This, in my opinion, is a big problem.

What’s the fix? A “new SMTP” on a whole different port, where only authed email is permitted? How do you make that DoS-resistant? Ideas?

(Obviously, counting on spammers to notice or care is not a good approach.)

A SpamAssassin rule-discovery algorithm

Published March 5, 2007

Just to get a little techie again… here’s a short article on a new algorithm I’ve come up with.

Text-matching rule-based anti-spam systems are pretty common — SpamAssassin, of course, is probably the most well-known, and of course the proprietary apps built on SpamAssassin also use this. However, other proprietary apps also seem to use similar techniques, such as Symantec’s Brightmail and MessageLabs’ scanner (hi Matt ;) — and doubtless there are others. As a result, ways to write rules quickly and effectively are valuable.

So far, most SpamAssassin text rules are manually developed; somebody looks at a few spam samples, spots common phrases, and writes a rule to match that. It’d be great to automate more of that work. Here’s an algorithm I’ve developed to perform this in a memory-efficient and time-efficient way. I’m quite proud of this, so thought it was worth a blog posting. ;)

Corpus collection

First, we collect a corpus of spam and “ham” (non-spam) mails. Standard enough, although in this case it helps to try to keep it to a specific type of mail (for example, a recent stock spam run, or a run from the OEM spammer).

Typically, a simple “grep” will work here, as long as the source corpus is all spam anyway; a small number of irrelevant messages can be left in, as long as the majority 80% or so are variations on the target message set. (The SpamAssassin mass-check tool can now perform this on the fly, which is helpful, using the new ‘GrepRenderedBody’ mass-check plugin.)

Rendering

Next, for each spam message, render the body. This involves:

  • decoding MIME structure
  • discarding non-textual parts, or parts that are not presented to the viewer by default in common end-user MUAs (such as attachments)
  • decoding quoted-printable and base64 encoding
  • rendering HTML, again based on the behaviour of the HTML renderers used in common end-user MUAs
  • normalising whitespace, “this is\na \ntest” -> “this is a test”

All pretty basic stuff, and performed by the SpamAssassin “body” rendering process during a “mass-check” operation. A SpamAssassin plugin outputs each message’s body string to a log file.

Next, we take the two log files, and process them using the following algorithm:

N-gram Extraction

Iterate through each mail message in the spam set. Each message is assigned a short message ID number. Cut off all but the first 32 kbytes of the text (for this algorithm, I think it’s safe to assume that anything past 32 KB will not be a useful place for spammers to place their spam text). Save a copy of this shortened text string for the later “collapse patterns” step.

Split the text into “words” — ie. space-separated chunks of non-whitespace chars. Compress each “word” into a shorter ID to save space:

"this is a test" => "a b c d"

(The compression dictionary used here is shared between all messages, and also needs to allow reverse lookups.)

Then tokenize the message into 2-word and 3-word phrase snippets (also known as N-grams):

"a b c d" => [ "a b", "b c", "c d", "a b c", "b c d" ]

Remove duplicate N-grams, so each N-gram only appears once per message.

For each N-gram token in this token set, increment a counter in a global “token count” hashtable, and add the message ID to the token’s entry in a “message subset hit” table.

Next, process the ham set. Perform the same algorithm, except: don’t keep the shortened text strings, don’t cut at 32KB, and instead of incrementing the “token count” hash entries, simply delete the entries in the “token count” and “message subset hit” tables for all N-grams that are found.

By the end of this process, all ham and spam have been processed, and in a memory-efficient fashion. We now have:

  • a table of hit-counts for the message text N-grams, with all N-grams where P(spam) < 1.0 — ie. where even a single ham message was hit — already discarded
  • the “message subset hit” table, containing info about exactly which subset of messages contain a given N-gram
  • the token-to-word reverse-lookup table

To further reduce memory use, the word-to-token forward-lookup table can now be freed. In addition, the values in the “message subset hit” table can be replaced with their hashes; we don’t need to be able to tell exactly which messages are listed there, we just need a way to tell if one entry is equal to another.

Summarisation

Iterate through the hit-count table. Discard entries that occur too infrequently to be listed; discard, especially, entries that occur only once. (We’ve already discarded entries that hit any ham.)

Make a hash that maps the message subsets to the set of all N-gram patterns for that message-subset. For each subset, pick a single N-gram, and note the hit-count associated with it as the hit-count value for that entire message-subset. (Since those N-grams all appear in the exact same subset of messages, they will always have the same P(spam) — this is a safe shortcut.)

Iterate through the message subsets, in order of their hit-count. Take all of the message-subset’s patterns, decode the N-grams in all patterns using the token-to-word reverse-lookup table, and apply this algorithm to that pattern set:

Collapse patterns

So, input here is an array of N-gram patterns, which we know always occur in the same subset of messages. We also have the saved array of all spam messages’ shortened text strings, from the N-gram extraction step. With this, we can apply a form of the BLAST pattern-discovery algorithm, from bioinformatics.

Pop the first entry off the array of patterns. Find any one mail from the saved-mails array that hits this pattern. Find the single character before the pattern in this mail, and prepend it to the pattern. See if the hits for this new pattern are the same message set as hit the old pattern; if not, restore the old pattern and break. If you hit the start of the mail message’s text string, break. Then apply the same algorithm forward through the mail text.

By the end of that, you have expanded the pattern from the basic N-gram as far as it’s possible to go in both directions without losing a hit.

Next, discard all patterns in the pattern array that are subsumed by (ie. appear in) this new expanded pattern. Add it to the output list of expanded patterns, unless it in turn is already subsumed by a pattern in that list; discard any patterns in the output list that are subsumed by this new pattern; and move onto the next pattern in the input list until they’re all exhausted.

(By the way, the “discard if subsumed” trick is the reason why we start off with 3-word N-grams — it gives faster results than just 2-word N-grams alone, presumably by reducing the amount of work that this collapse stage has to do, by doing more of it upfront at a relatively small RAM cost.)

Summarisation (continued)

Finally, output a line listing the percentage of the input spam messages hit (ie. (hit-count value / total number of spams) * 100) and the list of expanded patterns for that message-subset, then iterate on to the next message-subset.

Example

Here’s an example of some output from recent “OEM” stock spam:

$ ./seek-phrases-in-corpus --grep 'OEM' \
        spam:dir:/local/cor/recent/spam/*.2007022* \
        ham:dir:/local/cor/recent/ham/*.200702*
[mass-check progress noises omitted]
 RATIO   SPAM%    HAM%   DATA
 1.000  72.421   0.000  / OEM software - throw packing case, leave CD, use electronic manuals. Pay for software only and save 75-90%! /,
                         / TOP 1O ITEMS/
 1.000  73.745   0.000  / $99 Macromedia Studio 8 $59 Adobe Premiere 2.0 $59 Corel Grafix Suite X3 $59 Adobe Illustrator CS2 $129 Autodesk Autocad 2007 $149 Adobe Creative Suite 2 /,
                         /s: Adobe Acrobat PR0 7 $69 Adobe After Effects $49 Adobe Creative Suite 2 Premium $149 Ableton Live 5.0.1 $49 Adobe Photoshop CS $49 http:\/\//,
                         / Microsoft Office 2007 Enterprise Edition Regular price: $899.00 Our offer: $79.95 You save: $819.95 (89%) Availability: Pay and download instantly. http:\/\//,
                         / Adobe Acrobat 8.0 Professional Market price: $449.00 We propose: $79.95 Your profit: $369.05 (80%) Availability: Available for /,
                         / $49 Windows XP Pro w\/SP2 $/,
                         / Top-ranked item. (/,
                         /, use electronic manuals. Pay for software only and save 75-90%! /,
                         / Microsoft Windows Vista Ultimate Retail price: $399.00 Proposition: $79.95 Your benefit: $319.05 (80%) Availability: Can be downloaded /,
                         / $79 MS Office Enterprise 2007 $79 Adobe Acrobat 8 Pro $/,
                         / Best choice for home and professional. (/,
                         / OEM software - throw packing case, leave CD/,
                         / Sales Rank: #1 (/,
                         / $79 Microsoft Windows Vista /,
                         / manufacturers: Microsoft...Mac...Adobe...Borland...Macromedia http:\/\//
 1.000  73.855   0.000  / MS Office Enterprise 2007 /,
                         /9 Microsoft Windows Vista /,
                         / Microsoft Windows Vista Ultimate /,
                         /9 Macromedia Studio 8 /,
                         / Adobe Acrobat 8.0 /,
                         / $79 Adobe /
 1.000  74.242   0.000  / Windows XP Pro/
 1.000  74.297   0.000  / Adobe Acrobat /
 1.000  74.462   0.000  / Adobe Creative Suite /
 1.000  74.573   0.000  / Adobe After Effects /
 1.000  74.738   0.000  / Adobe Illustrator /
 1.000  74.959   0.000  / Adobe Photoshop CS/
 1.000  75.014   0.000  / Adobe Premiere /
 1.000  75.290   0.000  / Macromedia Studio /
 1.000  75.786   0.000  /OEM software/
 1.000  75.841   0.000  / Creative Suite /
 1.000  75.896   0.000  / Photoshop CS/
 1.000  75.951   0.000  / After Effects /
 1.000  76.062   0.000  /XP Pro/
 1.000  82.460   0.000  / $899.00 Our /,
                         / Microsoft Office 2007 Enterprise /,
                         / $79.95 You/

Immediately, that provides several useful rules; in particular, that final set of patterns can be combined with a SpamAssassin “meta” rule to hit 82% of the samples. Generating this took a quite reasonable 58MB of virtual memory, with a runtime of about 30 minutes, analyzing 1816 spam and 7481 ham mails on a 1.7Ghz Pentium M laptop.

(Update:) here’s a sample message from that test set, demonstrating the top extracted snippets in bold:

  Return-Path: <tyokaluassa.com@ultradian.com>
  X-Spam-Status: Yes, score=38.2 required=5.0 tests=BAYES_99,DK_POLICY_SIGNSOME,
          FH_HOST_EQ_D_D_D_D,FH_HOST_EQ_VERIZON_P,FH_MSGID_01C67,FUZZY_SOFTWARE,
          HELO_LOCALHOST,RCVD_IN_NJABL_DUL,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_DYNAMIC,
          URIBL_AB_SURBL,URIBL_BLACK,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_RHS_DOB,
          URIBL_SBL,URIBL_SC_SURBL shortcircuit=no autolearn=spam version=3.2.0-r492202
  Received: from localhost (pool-71-125-81-238.nwrknj.east.verizon.net [71.125.81.238])
          by dogma.boxhost.net (Postfix) with SMTP id E002F310055
          for <xxxxxxxxxxx@jmason.org>; Sun, 18 Feb 2007 08:58:20 +0000 (GMT)
  Message-ID: <000001c7533a$b1d3ba00$0100007f@localhost>
  From: "Kevin Morris" <tyokaluassa.com@ultradian.com>
  To: <xxxxxxxx@jmason.org>
  Subject: Need S0ftware?
  Date: Sun, 18 Feb 2007 03:57:56 -0500

  OEM software - throw packing case, leave CD, use electronic manuals.
  Pay for software only and save 75-90%!

  Discounts! Special offers! Software for home and office!
              TOP 1O ITEMS.

    $79 Microsoft Windows Vista Ultimate
    $79 MS Office Enterprise 2007
    $79 Adobe Acrobat 8 Pro
    $49 Windows XP Pro w/SP2
    $99 Macromedia Studio 8
    $59 Adobe Premiere 2.0
    $59 Corel Grafix Suite X3
    $59 Adobe Illustrator CS2
  $129 Autodesk Autocad 2007
  $149 Adobe Creative Suite 2
  http://ot.rezinkaoem.com/?0B85330BA896A9992D0561E08037493852CE6E1FAE&t0

            Mac Specials:
  Adobe Acrobat PR0 7             $69
  Adobe After Effects             $49
  Adobe Creative Suite 2 Premium $149
  Ableton Live 5.0.1              $49
  Adobe Photoshop CS              $49
  http://ot.rezinkaoem.com/-software-for-mac-.php?0B85330BA896A9992D0561E08037493852CE
  6E1FAE&t6

  See more by this manufacturers:
  Microsoft...Mac...Adobe...Borland...Macromedia
  http://ot.rezinkaoem.com/?0B85330BA896A9992D0561E08037493852CE6E1FAE&t4

  Microsoft Windows Vista Ultimate
  Retail price:  $399.00
  Proposition:  $79.95
  Your benefit:  $319.05 (80%)
  Availability: Can be downloaded INSTANTLY.
  http://ot.rezinkaoem.com/2480.php?0B85330BA896A9992D0561E08037493852CE6E1FAE&t3
  Best choice for home and professional. (37268 reviews)

  Microsoft Office 2007 Enterprise Edition
  Regular price:  $899.00
  Our offer:  $79.95
  You save:  $819.95 (89%)
  Availability: Pay and download instantly.
  http://ot.rezinkaoem.com/2442.php?0B85330BA896A9992D0561E08037493852CE6E1FAE&t1
  Sales Rank: #1 (121329 reviews)

  Adobe Acrobat 8.0 Professional
  Market price:  $449.00
  We propose:  $79.95
  Your profit:  $369.05 (80%)
  Availability: Available for INSTANT download.
  http://ot.rezinkaoem.com/2441.php?0B85330BA896A9992D0561E08037493852CE6E1FAE&t2
  Top-ranked item. (31949 reviews)

Further work

Things that would be nice:

  • It’d be nice to extend this to support /.*/ and /.{0,10}/ — matching “anys”, also known as “gapped alignment” searches in bioinformatics, using algorithms like the Smith-Waterman or Needleman-Wunsch algorithms. (Update: this has been implemented.)
  • A way to detect and reverse-engineer templates, e.g. “this is foo”, “this is bar”, “this is baz” => “this is (foo|bar|baz)”, would be great.
  • Finally, heuristics to detect and discard likely-poor patterns are probably the biggest wishlist item.

Tuits are the problem, of course, since $dayjob is the one that pays the bills, not this work. :(

The code is being developed here, in SpamAssassin SVN. Feel free to comment/mail if you’re interested, have improvement ideas, or want more info on how to use it… I’d love to see more people trying it out!

Some credit: I should note that IBM’s Chung-Kwei system, presented at CEAS 2004, was the first time I’d heard of a pattern-discovery algorithm (namely, their proprietary Teiresias algorithm) being applied to spam.

Irish Blog Awards 2007

Published March 5, 2007

Well, that was fun! Taint.org didn’t make the shortlists, but I went along anyway just to hang out — and lots of chat was had accordingly. Got to finally meet up with a few people I’d chatted with online, like Nialler9 — and with a few old friends I don’t get to see often enough: Antoin, Elana, Brendan, Clare Dillon (ex-Iona!), and another ex-Ionian, Aisling Mackey. A good laugh.

Have to say though, it seems a vote from me was the kiss of death in many of the categories: Sarah Carey, Blogorrah, Ireland from a Polish perspective, and (the late lamented) TCAL all got my thumbs-up in the shortlist voting, and all wound up missing out on the chunk’o’lucite. Sorry about that guys. ;)

Thanks again to Damien for organising the whole do! It’s great to have an event like this to bring each of our disparate blogs physically together for a bit of community.

By the way I’d like to point out that, in contrast to the Blogorrah Bock the Robber mafiosi, I had a real moustache… ;)

BT’s daily disconnects, revisited

Published March 2, 2007

As I noted last year, BT, the ISP I use here in Ireland, disconnects broadband sessions on a daily basis, assigning a new IP address; this is really aggravating to anyone who uses a VPN, such as most telecommuters. Reportedly, this is done to work around deficiencies in their billing system.

A comment from Jeremy on that post suggested something interesting, though:

Just had a very helpful tech support guy on from BT. [… he] told me to restart the modem sometime that will make it convenient for the 24 hour IP change – i.e. restart it at 6am, and then it’ll change IP every day at 6am.

I’ve tested this, and it works. Much more convenient! Now the renumbering and VPN breakage can take place when I want it to — at the start of the workday, instead of some random point chosen by BT’s billing system. Quite an improvement.

To make this useful, here’s a script, “reboot-zyxel”, which will reboot your Zyxel P-660RU router remotely over the LAN. (It requires perl and curl.)

MailScanner developer in hospital

Published February 28, 2007

According to this message, Julian Field, the main developer of MailScanner, was found collapsed at his home last Friday. More details via the SA list:

He is in ICU though stable condition. I’ll not go into any details, anyone interested and not on the MS list can read the thread on the MS archive.

Currently any plans for cards and such as are on hold until further instructions are given to the MS list. However Matt Hampton has setup a clustermap at this address.

Matt will also forward any well wishes left on the website along with the map. Visiting the page will show Julian and his family just how far reaching his software is and how many people appreciate his efforts.

Get well, Julian! :(

Script: mythsshimport

Published February 27, 2007

Here’s a useful script for users of a MythTV box equipped with a PVR-350 MPEG capture/playback card — mythsshimport:

NAME

mythsshimport – transcode and install video files onto a MythTV box

SYNOPSIS

mythsshimport file1 [file2 …]

DESCRIPTION

Transcodes video files (AVI, MPEG, MOV, WMV etc.) into MythTV-compatible and PVR-350-optimised MPEG-2 .nuv files, suitable for viewing on a 4/3 screen, then transfers them to the MythTV backend, inserts them into the “recorded programs” listings, and builds seek tables.

All this happens on-the-fly, at faster-than-real-time rates; with a recent CPU in the transcoding box, and over an 802.11b wifi home network, you can start the process and start watching the video within 20 seconds, while it is transcoded and transferred in the background.

SSH is used as the network transport. If you have the CPU power available on the MythTV backend itself, you can run this script there (as the mythtv user) and it will skip the SSH parts entirely.

REQUIREMENTS

  • ssh password-less key access from transcode box into mythtv@mythbox (this could be localhost, if you’re transcoding on the mythbox). Test using: “ssh mythtv@mythbox echo hi”. If you run this script on the mythbox as the mythtv user, this is not required.

  • mencoder. Tested with 2:0.99+1.0pre7try2+cvs20060117-0ubuntu8 (I swear that’s a version string and not just me rolling my head around the keyboard)

  • MythTV. Tested with MythTV 0.20.

  • The “contrib/myth.rebuilddatabase.pl” script from the MythTV source tarball, installed on the mythbox in $PATH: download from svn.mythtv.org.

  • screen(1) installed on the transcoding box, used to keep the mencoder output readable

Download here.

Masonic spam

Published February 22, 2007

Wow, here’s a new one — and kind of appropriate, given my surname ;) Masonic spam!

To: xxxxxx at taint.org

Subject: Dear Benefactor Of 2007 Masory Grant,

From: Dr.Lavine Ferdon Ferdon

Date: Wed, 21 Feb 2007 15:40:26 +0100 (CET)

Dear Benefactor Of 2007 Masory Grant, The Freemason society of Bournemout under the jurisdiction of the all Seeing Eye, Master Nicholas Brenner has after series of secret deliberations selected you to be a beneficiary of our 2007 foundation laying grants and also an optional opening at the round table of the Freemason society. These grants are issued every year around the world in accordance with the objective of theFreemasons as stated by Thomas Paine in 1808 which is to ensure the continuous freedom of man and toenhance mans living conditions. We will also advice that these funds which amount to USD2.5million be used to better the lot of man through your own initiative and also we will go further to inform that the open slot to become a Freemason is optional, you can decline the offer. In order to claim your grant, contact the Grand Lodge Office co-secretary Dr.Lavine Ferdon Ferdon Grand Lodge Office Co-Secretary’s email: (lavin_ferd_law at excite.com)

Dr.Lavine Ferdon Ferdon,

Co-Secretary Freemason Society of Holdenhurst Road,

Bournemouth.

Sir David Hurley,

Secretary Freemason Society of Holdenhurst Road,

Brilliant. But why Bournemouth?

HOWTO block editing of pages in Moin Moin

Published February 20, 2007

A useful Moin Moin anti-spam tip, via Upayavira at the ASF: adding ACLs to pages so that only certain users can edit them. This is an easy way to interfere with the wiki spammers who get past the existing (quite good) Moin Moin anti-spam subsystems. They tend to aim for the common Wiki pages, such as WikiSandBox, RecentChanges, and FrontPage, so if you make those pages uneditable, that’ll cause them more trouble — and hopefully cause them to move on to easier targets, instead of defacing your wiki. Here’s how to do it (at least for Moin Moin >= 1.5.1).

Open a shell on the machine where the Moin Moin software is installed. Edit your “wikiconfig.py” file (in my case this is at /home/moinmoin/moin-1.5.1/share/moin/jmwiki/wikiconfig.py), and change the “acl_rights_before” line to read:

    acl_rights_before = u"JustinMason:read,write,delete,revert,admin"

Replace “JustinMason” with your wiki login name, of course.

Create an administrative group of trusted users. Do this by creating a page called “AdminGroup” containing

#acl All:read
These are the members of this group, who can edit certain restricted pages:
 * JustinMason

Now, for the sensitive pages (like FrontPage etc.), edit each one and add an access-control list line at the top of each page containing:

#acl AdminGroup:read,write All:read

That’s it. Users who are not in the AdminGroup will no longer be able to edit those pages. That should help… at least for a while ;)

Update: you should also use this in wikiconfig.py:

    acl_rights_default = u'Known:read,write,revert All:read'

This blocks non-logged-in users from writing to pages.

Irish Blog Awards

Published February 16, 2007

A quick note; the Irish Blog Awards shortlisting votes are about to end later today. I’ve been nominated in the long list (thanks!), for best technology blog — feel free to vote for me if you like ;)

Update: boo, no shortlisting. Still, probably my own fault, I was a bit too wishy-washy with the vote hustling! Maybe next year…

Odd legal mail

Published February 13, 2007

Last week, I received an odd-looking mail from “Claims Administration Center” ClaimsAdministrationCenter /at/ enotice.info, sent to my private email address — the one listed in an image on http://jmason.org/ (it never gets spam).

The mail reads:

Mittlholtz v . International Medical Research, Inc., Sophie Chen, John Chen, and Allan Wang (“IMR Defendants”), aka Meco, et al. v. IMR, et al., case No. GIC846200.

We are requesting by order of the Court filed with the Superior Court for the County of San Diego, CA, that you post the attached Summary notice as a Public Service Announcement on your web-site.

Below is a link to the PDF Summary Notice (Note: The document is in the .PDF format. To view the documents you will need the Adobe Acrobat Reader)

http://echo.bluehornet.com/ct/ct.php?t=….

This message was intended for: webaddress@jmason.org You were added to the system January 17, 2007. For more information please follow the URL below: http://echo.bluehornet.com/subscribe/source.htm?c=…

Follow the URL below to update your preferences or opt-out: http://echo.bluehornet.com/phase2/survey1/survey.htm?CID=…

Googling for GIC846200, I find it on a cached “civil new filed cases index” page at sandiego.courts.ca.gov:

CASE NUMBER FILE DATE CATEGORY LOCATION

GIC846200 04/21/2005 A72120 – Personal Injury (Other) San Diego MECO vs INTERNATIONAL MEDICAL RESEARCH INCORPORATED

So the case exists. I have no idea who either of the parties are, however.

The URLs in the message were all web-bugged; but bluehornet seem legit in general.

The URL http://www.enotice.info/ times out. Seems to have no spam-related Google Groups hits, although there are a lot of discussions about some iffy-looking class-action suit about Google Adsense.

After quite a bit of discomfort and asking around about the reputation of both bluehornet.com and enotice.info, I eventually succumbed and clicked through. The Summary URL above, after logging my click, redirects to this PDF file, which reads:

This case, called Mittleholtz v . International Medical Research, Inc., Sophie Chen, John Chen, and Allan Wang (‘IMR Defendants’), et al., case No. GIC846200, is a class action lawsuit that alleges that the IMR Defendants unlawfully distributed a product containing synthetic chemicals, the presence of which was also concealed from the public as a result of the IMR Defendants’ alleged failure to conduct any testing for adulteration by synthetic chemicals, including but not limited to diethylstilbestrol (DES) and warfarin (or coumadin), which is the active chemical in bloodthinners. Defendants deny the allegations. The Court has not formed any opinions concerning the merits of the lawsuit nor has it ruled for or against the Plaintiffs as to any of their claims. The sole purpose of this notice is to inform you of the lawsuit so that you may make an informed decision as to whether you wish to remain in or opt out of this class action.

You have legal rights and choices in this case. You can:

  • Join the case. You do not have to do or pay anything to be part of this case. And, you have to accept the final result in the case.

  • Exclude yourself and file your own lawsuit. If you want your own lawyer, you will have to exclude yourself as set forth below and pay your lawyer’s fees and costs.

  • Exclude yourself and not sue. If you do not wish to be part of this case and do not want to bring your own lawsuit, please mail a first class letter stating that you want to be excluded from the Mittleholtz v IMR class action (Case No. GIC846200), or you may fill out the letter available at www.gilardi.com/mittleholtzsettlement. Make sure the letter has your full name, address and signature. Mail it to: PC-SPES Litigation, Class Administrator, c/o Gilardi & Co. LLC, P O Box 8060 San Rafael, CA 94912-8060 by March 23, 2007.

    *This is only a summary. For complete notice and further information go to: www.gilardi.com/mittleholtzsettlement or call the toll-free number 1-877-800-7853.

So in other words, it’s hand-targeted unsolicited, but probably not bulk, email, flogging a class-action suit about ‘synthetic chemicals’ (presumably as opposed to the ‘organic’ variety). I suspect, given the phrasing in the initial mail, they probably googled for a keyword or company name, and found a hit somewhere in taint.org’s 5 years of archives — hence the PSA request.

In fact, I bet this forwarded story is what they found through Googling. Pity they didn’t include a URL for that!

Does sending legal notices like this through email not seem particularly risky, given the lack of reliability of the medium?

An odd situation, all told…

More ‘Small Engine Repair’

Published February 13, 2007

Plug plug plug: next week is the 2007 Jameson Dublin International Film Festival — some great movies being shown, I’m looking forward to it. Most of all, though, I want to recommend Small Engine Repair, which I’ve written about before. It’s being shown in the festival at 6:20 PM on Wed 21st Feb in IFI 1 — tickets can be booked online here, at EUR 9 apiece.

Writer and director, Niall Heery, won the Breakthrough Talent Award at this year’s Irish Film and Television Awards at the weekend. Nice one Niall!

Go see it if you get a chance — it’s a fantastic movie, in my opinion. And be sure to vote for it for the festival’s Audience Award…

Wikipedia and rel=”nofollow”

Published January 24, 2007

Apparently, Wikipedia has (possibly temporarily) decided to re-add the rel=”nofollow” attribute to outbound links from their encyclopedia pages.

There’s been a lot of heat and light generated about this, most missing one thing: there’s no reason why Google needs to pay attention.

Google, or any other search engine, can treat links in the Wikipedia pages any way they like — including ignoring ‘nofollow’, applying extra anti-spam heuristics of their own, or even trusting the links more highly.

‘Nofollow’ has had pretty much no effect on web-spam, and now is generally festooned all over weblog posts across the internet, both spammed and non-spammed posts, at that. It’d be interesting to see if it’s yet flipped to mean a higher correlation with nonspam than spam content…

Update: It appears Wikipedia used ‘nofollow’ before, so this is not exactly new, either.