Skip to content

Category: Uncategorized

DVD pirate’s pitch ends in arrest

Funny: BBC: DVD pirate's pitch ends in arrest:

A man has been arrested after trying to sell counterfeit DVDs - at a Trading Standards Office.

The man had apparently missed the sign on the office in Beehive Lane, Chelmsford, Essex, and asked if anyone would like to buy pirated films. Staff said they were very interested indeed in what he had to sell, but when he realised where he was he ran off, leaving his wares and £210 in cash.

Police later arrested the man in a supermarket in Chelmsford.

Hacking Netflix

Movies: Hacking Netflix, via torrez.

Jason Kottke points out a great quote on a Friendster cross-site scripting attack -- this great quote: 'We have a policy that we are not being hacked.'

He also speculates that Google used the GMail invite-network data for whitelisting -- but whitelisting based on email address alone is trivially exploitable, so I'd doubt it.

I'm just back from a trip over to Cape Cod to meet family (halfway between here and Ireland, y'see ;) -- lots and lots of luvverly lobster and sundry shellfish -- and after a 6 day trip, had 5000 spams and a couple of thousand nonspam mails to deal with. Thankfully SpamAssassin dealt with the spams (only about 5 false negatives, no false positives I could spot) -- but I'm going to have to do something about that volume of mail. drowning in the stuff. argh.

Microsoft 0wnz ‘http’

Web: Back in 2002, it occurred to someone to check the Google search results for 'http', to figure out what the most popular sites were.

Looks like it's changed -- here's the top five results from a Google search for 'http' now:

  • 1: Microsoft
  • 2: AltaVista (!!)
  • 3: Yahoo!
  • 4: My Excite
  • 5: Google

My guess: older links are getting good PageRank, using whatever new tweaked algorithm they're using. But AltaVista beating Google? ;)

RTE’s Bush Interview

TV: RTE's 'Prime Time' secured a fantastic interview with GWB, with Carole Coleman asking a few very pointed questions. Watch it with RealPlayer, or listen to the audio in MP3 (2.7Mb).

There's a pretty accurate transcript here:

Let me finish! How many times do I have to tell you how to do your job? See, I gotta insult France at least once. Then I gotta claim 'merica to be the most generous nation in the whole wide world, even though it's not true. And listen, let me mention that democracy in Pakistan, too. And guess what? I'm the first president to ever call for a Palestinian state and I'm damn proud of it - just look at the size of my smirk now. Listen, as long as I keep repeating myself and mouthing empty platitudes, you won't have a chance to call me on any of the bullshit coming out of my mouth.

OK, the official one is here.

It appears that the White House just dropped the ball on this one; reportedly, they had her list of questions three days in advance, but given that they suggested that she 'ask him a question on the outfit that Taoiseach Bertie Ahern wore to the G8 summit' (!!!), they weren't paying attention, and expected some kind of giggling moronic schoolgirl, or something.

Hilariously, the White House has since complained to RTE, the Irish Embassy, the Irish Government, and the reporter herself. Probably God, too. I doubt Prime Time will ever get a White House interview again, but given what they clearly expect from the poodles in the White House press corps, that's hardly much of a loss.

(I'd love to see what'd happen if he had to deal with Paxman ;)

Also, went to see Fahrenheit 9/11. Fantastic movie, and best of all, incredibly well-attended.

My favourite moment: the reminder of just how easily the US news media sold itself out during the war. Seeing Katie Couric blurting 'Navy Seals rock!!' like some kind of starstruck 5-year-old with an Action Man toy, was a classic. It's good to see that this will be immortalized in celluloid, as it was truly shocking at the time. (Not much has changed; Judith Miller is still writing for the NYT.)

Samuel L. Jackson’s ‘Irish’ comment

Ireland: Here's a hot UL that's floating around the irish web right now --

In a British program about Samuel L Jackson and Colin Farrell's lastest movie SWAT presented by British presenter, Kate Thornton, the following exchange occured:
  • Thornton: What was it like working with Colin (Farrell), cos he
    • is just so hot in the U.K. right now?
  • Jackson: He's pretty hot in the U.S. too.
  • Thornton: Yeah, but he is one of our own.
  • Jackson: Isn't he from Ireland?
  • Thornton: Yeah, but we can claim him cos Ireland is beside us.
  • Jackson: You see that's your problem right there. You British keep claiming people that don't belong to you. We had that problem here in America too, it was called slavery.

... yeah, right. ;)

(Update: Actually, believe it or not, that's more or less how it really went. Here's the transcript.)

Some commentary at
TheReggaeBoyz.com (quote: 'I NEARLY DEAD TO RASS!!!!') and Kuro5hin.

It looks like the TV programme does exist; no scripts online, unfortunately, so we'll never figure out if this one really happened, I think.

IMO, it's made up for sure. That last line is just a little too harsh for a primetime schmooze-a-gram, at the very least. Plus, it's the kind of thing only an Irishman would give a shit about -- the perpetual adoption of Irish celebs and worthies by the UK media is a continual source of irritation for the Irish -- as Dervala puts it:

'No, Oscar Wilde was ours. You put him in jail, though. And Shaw was ours. And Yeats. And Johnny Rotten.'

Announcing a new script

Web: Minor software announcement -- after some time using HTMLThumbnail, album, and even WebMake to build photo galleries, I finally got peeved enough, and gave in to the temptation of 'not invented here'. ;)

Presenting Uffizi, a CSS- and template-driven, themable perl script to generate photo galleries. Quoting the POD:

  • it's very self-contained, apart from dependencies on Image::Size and the ImageMagick convert command
  • fast, efficient incremental rebuilding
  • generates full CSS-styled, templated and valid HTML
  • every part of the generated HTML can be modified through the templates
  • generates reasonably-sized images as well as thumbnails, with a link to the full-sized image
  • secure -- all pages are static HTML, so your webserver won't get r00ted through a silly photo album script

I am, of course, using it on my own photo pages, and I'm very happy with it; it's been a while since I had to hack it. (I need to get it to thumbnail MPEGs as well, but apart from that it's teh nifty IMO.)

SpamAssassin now an Apache TLP!

Spam: SpamAssassin is now officially an Apache top-level project! InternetNews.com coverage:

The Apache Software Foundation is taking the spam fight to a new level -- literally -- with the promotion of its Spam Assassin project to top-level status.

Hooray ;)

The ‘humans are 99.84% accurate’ figure

Spam: 'The spam-classifying accuracy of a human being is 99.84%'. This statement has passed into SlashDot lore as the gospel truth, so time for some debunking.

First off, that's not what Bill Yerazunis said in the CRM-114 Sparse Binary Polynomial Hashing and the CRM114 Discriminator paper. Here's the real quote:

the human author's measured accuracy as an antispam filter is only 99.84% on the first pass

Here's a copy of the original mail:

I manually classified the same set of 1900 messages twice, and found three errors in my own classifications, hence I have a 99.84% success rate.

(my emphasis). In other words, the author sat down and ran through 1900 messages manually, then ran through them again, and checked to see how many messages in the first batch disagreed with the second.

Let's consider an alternative situation, where a user is presented with one message, and asked to take their time, give it a full examination and some thought, and then classify the message. I would consider that more likely to be classified correctly, since fatigue will not be an issue (after 1900 messages, I'm pretty tired of eyeballing), and neither will time pressure (taking 20 seconds on each of 1900 mails would require 10.5 hours, and would be excruciatingly boring to boot).

In addition, the study wasn't clear on exactly how much information from each mail was presented. Too little (just the subject line) or too much (every header and raw HTML), and a human will be more likely to make mistakes than if the mail is rendered fully, and the extraneous header info hidden. In my experience, I've never hand-classified 1900 messages purely through either method, because it's just too tiring, and I know I'll make quite a few mistakes. The UI for this work is important.

And finally, the figure is derived from a study with one user performing a task once. There's no way you could use that figure in a serious setting -- it's not valid statistical science. Here's Henry's comment:

Yerazunis' study of "human classification performance" is fundamentally flawed. He did a "user study" where he sat down and re-classified a few thousand of his personal e-mails and wrote down how many mistakes he made. He repeats this experiment once and calls his results "conclusive." There are several reasons why this is not a sound methodology:
  • a) He has only one test subject (himself). You cannot infer much about the population from a sample size of 1.
  • b) He has already seen the messages before. We have very good associative memory. You will also notice that he makes fewer mistakes on the second run which indicates that a human's classification accuracy (on the same messages) increases with experience. For this very reason, it is of the utmost importance to test classification performance on unseen data. After all, the problem tends towards "duplicate detection" when you've seen the data before hand.
  • c) He evaluates his own performance. When someone's own ego is on the line, you would expect that it would be very difficult to remain objective.

So, to correct the statement:

'The spam-classifying accuracy of this one guy, when classifying nearly two thousand mails by hand, was 99.84%, once.'

Cormack and Lynam’s study on supervised spam detection

Spam: or, 'SlashDot spam drama'. So, a few days ago, I forwarded a link to a paper I'd been sent -- it's a great paper, and I'm not just saying that because SpamAssassin did well -- it really tests some of the popular open-source spam filters comprehensively, and correctly. (The authors have 24 years of information retrieval research between them.)

The results have been pretty incendiary. ;) Here's a timeline with links, in case you were wondering where we are right now:

A UNIX shell tip

UNIX: I've just made the first change to my core bash configuration in years, to add -b to the set command-line. It triggered some thinking about when the last one was.

It turns out, that apart from writing scripts and aliases frequently, I haven't changed my commandline UI in any respect, since about 2 years ago. By contrast, I've been hacking about with GUI settings continually, new desktop backgrounds, themes, colours, etc. Odd!

Anyway, here's the tip -- it's very handy, I find.

I changed to using a 2-line prompt, with the first line containing the time and the full working directory, in a 'magic' cut-and-pasteable format:

        : exit=0 Thu Jun 24 17:55:29 PDT 2004; cd /home/jm/DL
        : jm 1203...; 

Note that the prompt starts with ":", which means that bash/sh will ignore the line until it hits ";". The end result is that the entire line evaluates to "cd /home/jm/DL" when pasted. Hey presto, cd'ing several terminals to the same dir just involves triple-clicking in one, and middle-button-pasting into the others. nifty! Similarly, the second line has a little bit of prompt, but that snippet will be ignored when cut and pasted.

Having the exit status of the last command (bash var: $?) is useful too. The code:

  do_prompt () {
    echo ": exit=$? `date`; cd $PWD"
  }
  PROMPT_COMMAND='do_prompt $?'   # executed before every prompt
  do_prompt 0                     # set up first prompt
  PS1=": `whoami` \!"
  PS2="... >>; "            # continuation prompt
  PS1="$PS1...; "

The Web-App generation

Software: Mark Twomey, in response to all the Win32 API stuff recently:

We now have a generation of computer users ... who have never received or sent email from a so called 'rich client', never had to send a postal order off to order something from some distant vendor, and are not amazed by something like a search engine. ....

Those ('rich client') people remind me of minicomputer users who crapped on the 'crummy little operating systems' used on 'crummy little desktop computers.'

He's right, you know -- for de yoot, Windows is generally just a way to access Hotmail.

Ahmed Chalabi and Iran’s encryption

Security: some crypto drama.

Ahmad Chalabi apparently told the Iranian government that the NSA had broken their secret code, according to 'US intelligence officials': NYTimes: Chalabi Reportedly Told Iran That U.S. Had Code. This story is still running -- Bruce Schneier has just posted his expert opinion, as has Ross Anderson. As I noted on Eric Rescorla's weblog, here's my (non-expert) theory ;)

It's known that the Iranians used Crypto AG equipment up until about 1992, and it's been widely reported that Crypto AG's systems were backdoored by the NSA and traffic routinely decrypted. (also, Baltimore Sun story, 1995)

Reportedly, the Anglo-Irish discussions of the 1985 were a rather one-sided affair, because the Irish government used Crypto AG machines to communicate between their Embassy in London and Dublin, and intercepts of their reports were fed back to the UK government.

In addition, according to this article (backup), the NSA also provided Iraq with intercepts of Iranian secret traffic, while Iraq was a US ally -- which could explain why Chalabi would have known about it.

It also speculates as to how it was done:

'Knowledgeable sources indicate that the Crypto AG enciphering process, developed in cooperation with the NSA and the German company Siemans, involved secretly embedding the decryption key in the cipher text. Those who knew where to look could monitor the encrypted communication, then extract the decryption key that was also part of the transmission, and recover the plain text message. Decryption of a message by a knowledgeable third party was not any more difficult than it was for the intended receiver. (More than one method was used. Sometimes the algorithm was simply deficient, with built-in exploitable weaknesses.)'

So my opinion is that Chalabi's claim was very old news from the 80's and early 90's -- which pretty much fits in with the rest of his tip-offs to everyone else ;)

“Vice-President Hunter Thompson”

Politics: Kerry in Colorado:

"Just to put your minds all at ease, I have four words for you that I know will relieve you greatly," Kerry told the fund-raiser. "How does this sound? Vice President Hunter Thompson."

Travel: Great posting on culture shock and 'going native' at Yankee Fog.

Hacks: Dan Kaminsky's LayerOne presentation hits Slashdot. Definitely one of the highlights of that conference.

Spam: confession for two: a spammer spills it all. Interesting -- especially since the spammer winds up earning less than he would have working for Starbucks.

It's also worth noting this posting from Gary Smith on the sa-users list, in which Gary filled out a spam form with some not-entirely-valid info -- with hilarious results!

So I did talk to some of these lenders. Apparently they buy leads from www.lendergateway.com . One guy that I talked to was irritated because it costs him $100 per lead they sell him and it's supposed to only be sold to him. He apologized quite a bit and was nice enough to give me the information on who sold him the names. The number he game me goes to voicemail which I'm going to try later. A couple other people told me what I can do with myself and one lady kept saying that she couldn't give me information on who provided her with my information.

The stupid thing is each time I talk to them I tell them I'm on a cell and that I need their name and number and I'll call them right back. They give it to me... So when they hang up I start calling again and again. I've been irritating the hell out of them...

Anyways, that's the fun storing of what happens when these forms are filled out.

$100 per spurious 'lead' would make a serious dent, if enough spurious leads showed up... ;)

WINW

Net: WINW Is Not WASTE: 'WINW is a small worlds networking utility. It was inspired by WASTE ... (WINW) has diverged from its original mission to create a clean-room WASTE clone. Today, the WINW feature set is different from that of WASTE, and its protocol is incompatible with WASTE's protocol. However, WINW and WASTE achieve similar goals: they allow people who trust each other to communicate securely.'

Not quite there yet -- just a Windows version with no sharing -- but actively under development. One to keep an eye on...

Great Economist article on UNIX

Software: Economist: Unix's founding fathers (via sourcefrog.net). A very good article on Thompson, Kernighan and Ritchie's amazing achievement, with some new details I hadn't heard before:

AT&T was required under the terms of a 1958 court order in an antitrust case to license its non-telephone-related technology to anyone who asked. And so Unix and C were distributed, mostly to universities, for only a nominal fee. When one considers the ineptness of AT&T's later attempts to commercialise Unix -- after the court order ceased to be applicable because of another antitrust case which broke up AT&T in 1984 -- this restriction, an accidental boost to what would later become known as the open-source movement, becomes even more crucial.

So that's how that happened. Just think -- if it wasn't for that court case, we'd probably all be hacking on VMS. ;)

Also at sourcefrog, mbp points out that the Sulston reverse-engineering story is 'remarkably similar to that of Richard Stallman several years earlier, when the frustration of closed-source printer software helped motivate him to start the GNU project'.

Patents: yet another sourcefrog link, this time to a CNet story with a hilarious quote regarding software patents and the GIF/PNG debacle:

But Unisys credited its exertion of the LZW patent with the creation of the PNG format, and whatever improvements the newer technology brought to bear.

'We haven't evaluated the new recommendation for PNG, and it remains to be seen whether the new version will have an effect on the use of GIF images,' said Unisys representative Kristine Grow. 'If so, the patent situation will have achieved its purpose, which is to advance technological innovation. So we applaud that.'

Wow. Presumably by the same logic, they applaud al-Qaeda for improving airline security innovation, too...

What’s wrong with DRM, and ‘better support’

Copyright: Cory Doctorow's DRM talk presented to MS research yesterday. This is a fantastic introduction to the issues regarding DRM; if you know someone who isn't convinced that DRM is A Bad Thing, this is the argument they need to read.

OSes: /.: France Considers Open Source. The usual arguments are going on in the comments, but some people still insist that they get better support from MS than from Linux vendors.

What planet are they on? Because it would have been handy for me to live there, on the occasions in the past where I've had to develop code on MS platforms, and administer networks of Windows PCs. In my experience, you do not get support from Microsoft. Instead, you do what you do with Linux -- go searching on Google, read MSDN, or post in the MSDN forums.

As far as I can see, there's zero difference between doing that with Windows, and doing exactly the same thing with Red Hat -- except in the latter case, you can turn up debug logging through a documented API or switch, use the source and fix it yourself, find the original developers and post a message to their core -dev list, or even ask them personally.

Where's this amazing support? Maybe the companies I've worked for just weren't paying enough, and therefore weren't significant blue-chip customers. Or maybe it's because we weren't based in the US, and so got support from less-skilled, less high-priority staff in a regional office. But I've certainly never experienced the support these advocates claim MS offers, which makes me think it's FUD as usual.

Bloomsday!

Literature: Happy Bloomsday Centenary! Google agrees:

Google Bloomsday logo

You can have a read of Joyce's masterpiece online at online-literature.com, although this is certainly one text that works better on paper, to be pored over and parsed slowly. But regardless of whether it's readable on-screen or not, the legality of that copy is dubious, anyway.

As this Telegraph article notes, the copyright situation on Ulysses is, sadly, a total mess. Even 84 years after it was written, and promptly banned in the US, UK and Ireland for 'obscenity', Ulysses remains a thorny legal subject.

The novel was first published in 1922, and as such, fell into public domain in the UK in 1992, but was apparently 'pulled back' in 1996. According to this mail, due to recent copyright term extensions, the 1922 text will now remain in copyright in the EU until the end of 2011, and may not expire until 2032 in the US. And this Irish Times article notes that in Ireland, 'copyright on Joyce's works ran out on December 31st, 1991, 50 years after his death. However, EU regulations revived copyright from July 1995 when it extended the lifetime of copyright to 70 years.'

Reportedly, the Dail even had to pass emergency legislation last week to prevent an exhibition at Dublin's National Library from being sued by the Joyce Estate:

The threat to the exhibition has been caused by the 2000 Copyright Act which creates a doubt about its ability to display manuscripts bought by the State because the Joyce estate still holds copyright.

Hilarious. Recent overzealous copyright extension legislation snares governments too! But they get to rewrite the laws in emergency session to fix it ;)

All very ironic, considering Ulysses' structure was deliberately derived from The Odyssey in the first place.

Making a Bootable CD from a Floppy Image

Tech: Troubleshooters: Making a bootable CD from a bootable floppy image.
Making a note of this for future reference -- it should be handy next time I need to do a BIOS or firmware upgrade on my Thinkpad.

I ran into the need for this recently when trying to upgrade the BIOS on my Thinkpad running Linux, so hibernation would work. IBM don't provide BIOS upgrade tools for Linux, so you have to keep a Windows partition around. (Yes, I pay the Windows Tax -- I've been bitten by proprietary firmware upgrades requiring it in the past, as in this case.)

Amazingly, however, even after paying the Tax, the 'non-diskette' BIOS upgrade (ie. the standalone Windows app) doesn't work from Windows XP! Instead, you get a hard hang when it tries to bring the machine down from XP to a single-app mode to perform the upgrade. Running from DOS similarly fails, because the BIOS upgrade app is a WIN32 application. Clever.

Eventually, I wound up reformatting my Windows partition, installing Windows 98 (!), and running the BIOS upgrade app from that worked fine. But next time around, I should be able to save myself a few hours of MCSE imitation by using this floppy-to-CD trick... here's hoping. ;) PCs Are Hard.

‘Precision’ bombing, and iTMS Europe

War: A couple of war links, I'll keep it short. ;)

High-profile air strikes 'killed only civilians'. 'The American military launched some 50 air strikes designed to kill specific targets during the Iraq war, it emerged yesterday, but none of them found its mark. Instead the air strikes had a high civilian toll, according to military officials serving at the time.' Still, it sounded good, like as if CSI were doing all the war strategification and stuff ;)

And: the
Pentagon 'Torture Memos' took some tips
from the torture techniques used in Northern Ireland in the 1970s.

Music: Licensing row mars iTunes launch. UK indie labels report that 'where Apple has spoken to labels the terms on offer have been commercial suicide', and as a result, they won't be selling their tunes via iTMS Europe.

I agree with Mark Twomey on this one -- bad move. This (and the prices!) reduce the Euro-iTunes offering to about the usefulness of whatever that one is that Real.com have (you know, the one you can't even remember the name of) -- and nobody in Europe buys major-label music online anyway.

First Nobel Prizewinner forced to reverse-engineer?

Software: This mail contains a fantastic anecdote from The Common Thread: Science, Politics, Ethics and the Human Genome, by John Sulston, head of the Sanger Centre, and a joint winner of the Nobel Prize for Medicine. I'll reproduce some bits here:

Once the first fluorescence sequencing machines arrived, it became clear that we had to take control of the software. The machines worked well, but ABI (jm: the vendor) wanted to keep control of the data analysis end by forcing their customers to use their proprietary software. ...

I could not accept that we should be dependent on a commercial company for the handling and assembly of the data we were producing. The company even had ambition to take control of the analysis of the sequence, which was ridiculous. ...

So, one hot summer Sunday afternoon, I sat on the lawn at home with printouts spread all around me and decrypted the ABI file that stored the trace data. ... Within a very few days, Rodger and his group had written display software that showed the traces - and there we were. The St Louis team joined in, and they all went to decrypt more of the ABI files, so that we had complete freedom to design our own display and analysis systems. It transformed our productivity. Previously we'd only been able to get the traces as printouts, which we bound together in fat notebooks ....

I certainly feel that between us we did push ABI back a bit and denied to them complete control of this downstream software. It was the first experience of the kind of battle for control of information that I seem to have been fighting with commercial companies ever since: a foretaste of the much larger battles that would later surround the human genome.

Amazing. Was John Sulston the first Nobel Prize-winner to have to reverse-engineer a proprietary file format in the course of his research?

And would his actions be legal in the UK in a few years, once the IPR Enforcement Directive is transposed into law there?

LayerOne

Conferences: LayerOne was seriously great! Got to meet up with some really interesting people; discuss some nifty stuff; and get some new angles on the whole hacking scene.

Seriously, that was well worthwhile, especially in terms of potential new ways to deal with spam, and issues to watch out for in terms of spammer techniques in future. A great techie conf, and the boozing^Wsocialising was pretty good too ;)

I'm actually giving some thought to going to Defcon after that...

German neo-nazi UBE, and CAN-SPAM

Spam: Reg: German hate mail spam attack stuns experts: 'Mailboxes in Germany and the Netherlands were flooded yesterday with spam containing German right-wing propaganda. Spammers used the Sober.G virus - a mass mailing worm that sends itself to email addresses harvested from infected computers - to spread their messages as widely as possible.'

The one good thing about this is that it might help some people realise that spam isn't all about porn and commercial email; any kind of mail can be spam, including political speech.

However, this may be a bit late for the US, since CAN-SPAM explicitly does not regulate political spam. ah well, you live and learn, I suppose. ;)

Updating European Election voting guide for Ireland

Patents: Ciaran O'Riordan just posted a message to ILUG, regarding how concerned voters in Ireland can use their votes in tomorrow's European elections to prevent legalising patenting of software ideas in Europe. Here's the scoop:

Area Vote #1 and #2
East Avril Doyle Eoin Dubsky
South Brian Crowley Gerard Collins
North WestSean O'Neactain
Dublin Patrica McKenna Ivana Bacik

Note the main thing I got wrong -- some sitting MEPs from Fianna Fail and FG actually voted the right way! So a vote for FF in this case, is a vote against software patents. (I never thought I'd be saying that, but there you go ;)

TaintBochs, and oil

Security: A very interesting security paper -- Understanding Data Lifetime via Whole System Simulation. It combines virtual machines with data-flow tracking (a la perl's 'taint' mechanism, after which this site is named ;)

By modifying the Bochs VM to support tracking 'tainted' data, they found several cases in popular apps (Mozilla, emacs, and MSIE) where passwords entered from the keyboard are retained in memory, and thereby wind up on disk due to swapping.

This has been a known issue for a long time -- see the source for passwd.c from the 'shadow' package -- but aside from security-naive developers, several other factors have made it more complex recently:

  • recent too-smart compilers will optimise away memset()
    • buffer-zeroing unless you're careful (oops!)
      • Input buffers and event queues are a problem; password data from the keyboard will often persist in the kernel, window system, and application event queue buffers.
      • Abstractions cause many needless copies of tainted strings. Mozilla's abstraction layers even include a string-copy to the heap to perform a string comparison operation, ouch ;)

In general, they suggest more use of buffer zeroing, even for low-level buffers that might not seem to require it (such as the X server's event queue, and the kernel input buffers).

BTW, a similar system they didn't mention is the Sidewinder firewall appliance, which uses what they call 'Type Enforcement' -- effectively, tainting the data based on which network interface it arrived on.

Overall, a very nifty paper. I wonder if Tal Garfinkel is related to Simson? ;)

Oil: a MeFi gem: expert opinion on depletion of the oil reserves. 'Simmons, Campbell, even the Iranian Bakhtiari agreed that the real situation of Saudi reserves is very bad. ... Not a rosy picture, even for optimists.'

Patents: Transcript of the rms talk from a couple of weeks ago.

MS’ latest patent

Patents: Oh, come on. USPTO: task list window for use in an integrated development environment. Here's claim 1:

  1. A computer-implemented method for managing development-related tasks, the method comprising:

    during an interactive code development session, evaluating source code to determine whether a comment token is present;

    in response to determining that the source code contains a comment token, inserting a task into a task list; and

    in response to completion of a task, modifying the task list during the interactive code development session to indicate that the task has been completed.

There's 74 more claims that are about up to that standard, including the usual 'an input module connected to the knee-bone' mumbo-jumbo that means it 'isn't a software patent'.

This is just quite simply absurd. Are we really supposed to believe that nobody had thought of what is essentially a list of tickboxes, displaying the output of 'grep TODO *.c', before March 6, 2000? You have got to be kidding. This /. comment suggests that Delphi 5 (released 1999) did it.

(update: looks like there was a provisional patent application, so that may have to be Mar 5 1999.)

William Chiles, Anders Hejlsberg, Randy Kimmerly and Peter Loforte should be ashamed of themselves for filing this joke. And the USPTO examiner who granted it should be fired.

(PS: a factoid from the slashdot comments: IBM receives (note: not even 'files for') nearly 10 patents every day.)

Invasion of the spambots

Spam: Good Salon article on the new forms of spamming, such as Wiki and referrer-log spamming etc. Here's a good quote:

'The adult industry will likely be married to spam and its attendant distribution methods long past the evolution of man into beings of pure energy,' jokes Domenic Merenda, vice president of business development for Edge Productions, a company that operates adult-media properties.

There's a good deal of crossover -- I've seen both email and referrer-log spam advertising the same porn sites.

Nigritude Ultramarine

Web: the June part of the contest is over, but given that there's a July part still to go -- here's a 'Nigritude Ultramarine' link to Anil Dash.

I wasn't really bothered at all about this, until I came across this guy, whose technique involved spamming third-party Wiki sandboxes with backlinks. His excuse? 'A Sandbox (is) a part of a system in which everybody is urged to play around freely. Usually for testing purposes. You can post headings, paragraphs, lists and links here. The content in return will be indexed by Google.'

As this forum thread points out -- 'The SandBox page is there for a purpose: to allow users of the wiki to learn to use the software. It is
not meant to be "a place where anyone can create backlinks."'

Sorry, that's spam in my book.

GMail Invites

Mail: GMail users, check your mail; if mine was anything to go by, you should have three new invites to give out.

Irish Dating Site, and TheyWorkForYou.com

Web: Bernie Goldbach points to a site that's news to me: AnotherFriend.com. It's an Irish dating site.

I've had the odd discussion comparing dating culture in the US (organised 'dating') and Ireland and the UK (where it's a lot more casual), and I must say, I was really convinced that the Friendster/craigslist-style organised, web-mediated dating just wouldn't fly.

Seems I was wrong! Right now, there's 157 people online on the site, with a good half of those being logged-in, chatting users, and about 75% of those in turn being premium, paying members. Wow, not bad.

Politics: TheyWorkForYou.com is a triumph. The most incredibly detailed, and web-aware, hypertextual database of political activity I've seen yet. The web-awareness -- full of scraping, links, RSS and even community -- is what makes it amazing; the concept of being able to read news of your representative's latest speeches and voting record in your RSS aggregator is incredible. We need to get this out there for every country in the world.

It certainly beats Today in Parliament, that's for sure ;)

Aside: nice choice of username for the 'Site News' weblog:

Some sites linking to this entry

An error occurred: Connection error: Access denied for user: 'fawkesmt'@'localhost' (Using password: YES)

Wierd: Incredible footage (WMV stream) of a guy who went nuts, converted a caterpillar earthmover into what is essentially a tank, and went on a GTA-style rampage through the streets of Granby, 15 miles west of Denver, Colorado. In the process, he destroys the local bank, the newspaper, and several stores, seemingly working on the basis of (several) personal grudges.

Action Replay

Hacking: Amazing -- the Action Replay cartridge is still around!

To be honest, I'm quite surprised that the PS2 hardware platform allows any of this stuff without some mod-chip-style soldering... but then, it's pretty clear Datel have the technology to figure these things out. Impressive.

Aside: in my teens, I wrote demos on the Commodore 64 entirely in the Action
Replay's built-in monitor. I tried using compilers that supported such luxuries as symbolic labels, variable names, etc., but the ability to halt the entire machine and debug extensively, with a single button press, was just too nifty ;)

Irish MEP Candidates

Patents: lyranthe.org notes that the EU elections are coming up this Thursday, 11th June. Accordingly, here's a single-issue roundup of the candidates, from what I've heard:

  • The Labour Party, sadly, haven't yet come up with a concrete policy on the issue -- but the Dublin candidate, Ivana Bacik, has (verbally) stated her opposition.
  • The Greens, however, are actively campaigning against them, their candidates clearly understand and have communicated with voters on the issue in the past, and the cross-Europe party policy is clearly stated.
  • Eoin Dubsky is an independent candidate, standing on a primarily anti-war platform. He's stated his opposition to software patenting clearly and publically. He's also a total techie -- with RSS feeds and a Redbrick account! ;)
  • FG's position is totally unclear, as usual... ;)
  • And in the other corner: FF and the PDs are whole-heartedly supporting software patenting; in fact, they're the ones running the EU Council which just pushed through software patenting law despite the democratic mandate from the European Parliament. boo.

(PS: these are my opinions, not those of my employer. ;)

(updated: I'd left out Eoin Dubsky! my bad, now fixed.)

Easy-peasy web scraping: HTTP::Recorder

Perl: I've been writing a few convenience web-scrapers recently using WWW::Mechanize, with great success.

So the latest development, HTTP::Recorder, looks very nifty too:

HTTP::Recorder is a browser-independent recorder that records interactions with web sites and produces scripts for automated playback. Recorder produces WWW::Mechanize scripts by default (see WWW::Mechanize by Andy Lester), but provides functionality to use your own custom logger.

... Simply speaking, HTTP::Recorder removes a great deal of the tedium from writing scripts for web automation. If you're like me, you'd rather spend your time writing code that's interesting and challenging, rather than digging through HTML files, looking for the names of forms an fields, so that you can write your automation scripts. HTTP::Recorder records what you do as you do it, so that you can focus on the things you care about.

No SSL support yet, though, as far as I can see, but for simple scraping -- or as a good starting point for a more complex Mechanize script -- it looks like it'll work great.

ISPs, AUPS, and RIRs

Spam: Kasia raises a very interesting question. Here it is, in a nutshell:

Should the quality of an ISP's enforcement of its Acceptable Use Policy, be a condition of their contract with their Regional Internet Registry, and therefore affect whether they can be assigned new network address space?

  • Are there that many ISPs with lax or virtually nonexistent spam-related AUP enforcement? Yes, definitely.
  • Is spam that much of a problem? Speaking personally, I would say yes, but then, I would ;)
  • who would judge whether an ISP is doing enough, or too little?

Head on over to her weblog if you have a comment on this.

Don’t look for it, and you won’t find it

Health: USDA orders silence on mad cow in Texas: 'The U.S. Department of Agriculture has issued an order instructing its inspectors in Texas, where federal mad cow disease testing policies recently were violated, not to talk about the cattle disorder with outside parties ... The order ... was issued in the wake of the April 27 case at Lone Star Beef in San Angelo, in which a cow displaying signs of a brain disorder was not tested for mad cow disease despite a federal policy to screen all such animals.'

Great idea -- if you want to avoid finding mad cow cases, just don't bother looking for them! The beef rendering plant in question supplies beef to MacDonalds, reportedly.

Press: LWN: A look at SpamAssassin 3.0 (article is subscriber-only until next week).

OSes: Kernelthread.com: Making an Operating System Faster. Great article on some OS-level optimisations Apple used in MacOS X -- including a nifty boot-time read-ahead system which reportedly more than doubles the speed of OS X reboots. nice!

Wildlife: here's another critter we encountered last weekend -- a baby Western Diamondback rattlesnake, hiding in a crevice.

Spamometer

Spam: The Spamometer; a 1997-vintage spamfilter along the lines of filter.plx. Interestingly, I hadn't seen this before -- who knows, if I had, SpamAssassin could have used a (0.0, 1.0) scoring system instead of the '5 point threshold'. ;) (Thanks, Gary!)

Going to LayerOne

Conferences: I'm going to LayerOne; it looks interesting, and I've been hoping to bump into Danny O'Brien (who's there doing his Life Hacks talk) for a couple of drinks and a blather for quite a while. Other speakers look similarly interesting, in an 'offbeat hacker conference' way, so I think it'll be fun.

Conflicts with The Streets playing the Wiltern though, but c'est la vie ;)

Desert camping, and Dr. Strangelove’s all-zeroes password

Life: I've learned one thing this weekend -- humans are not designed to function in the desert. I went bush-camping in the Anza-Borrego Desert state park with a few mates, and we quite simply baked in the 45C/113F degree heat. Walking 3 miles in that heat was easily equivalent to 15 miles in normal temperatures.

We did manage to catch a good look at one of the endangered bighorn sheep that live there -- the poor sheep was clearly trying to get to some water, but those damn humans kept getting in the way!

On the way back, we passed the aftermath of a forest fire near Temecula. Scorched earth.

Security: via IP -- a very scary article at Bruce Blair's Nuclear Column -- apparently, the secret unlocking codes on the launch control mechanisms of Minuteman nuclear missiles were deliberately set to '00000000' throughout the height of the cold war, because the Strategic Air Command 'remained far less concerned about unauthorized launches than about the potential of these safeguards to interfere with the implementation of wartime launch orders.'

Green: A couple of good /. comments on renewable power sources: one from a wind farm designer, and some anti-FUD figures for solar panels.

Music: The full text of
The Timelords' The Manual (How To Have a Number One the Easy Way) is online:

        THE JUSTIFIED ANCIENTS OF MU MU
      REVEAL THEIR ZENARCHISTIC METHOD USED
        IN MAKING THE UNTHINKABLE HAPPEN.

                  KLF 009B
          1988 (YOU KNOW WHAT'S GONE) 

Snippets

Photos: the view out to sea from Seal Beach, just south of LA. (duh. thanks Ben, I'd b0rked the link earlier.)

Patents: via the FFII Kwiki, here's 2087 Microsoft USPTO software patents viewed roughly by subject matter. The 'Web' selection is particularly interesting.

Terror: The Atlantic: All you need is love -- how the terrorists stopped terrorism. Amazing -- marry them off!

Tourism: Pictures from Bangkok's new 'Sky Bar' -- open-air dining, 63 floors up, with no walls apart from 1.5-metre-high glass.

new terror indicators

Funny: NYPD alerts cops to 'terror indicators'.

The NYPD has ordered its patrol force to be more vigilant about spotting and reporting possible signs of terrorism, including individuals who "express hatred for America." .... The cards advise them to contact counterterrorism investigators when they have suspicions over anyone who is, among other things, carrying driver's licenses from different states, videotaping utilities and tunnels or wearing fake uniforms.

Sounds like the Village People won't be playing NYC any time soon, then ;)

Language registration: en-Spam-porn

Funny: via swhackit! -- Language registration: en-Spam-porn:

'One is very much tempted. It is certainly a unique orthography.'

Indeed. When I was offered "[t]ons of dwolnaoadble mvoies, pohtos and sotires", I quickly read past "mvoies" and "pohtos", but was stumped for a while by "sotires". Perhaps I was blocked by interference from "satires".

But I think that registration will fail, because there are no descriptive works provided for the Language Tag Reviewer to consult.

a new use for the ‘Terrorism Quotient’

Marketing: It appears that MATRIX (the Multistate Anti-TeRrorism Information EXchange) at one stage did -- and may still -- include a 'terrorism quotient' field, representing 'a statistical likelihood of (people) being terrorists'.

Seisint, the company providing the system, is a Boca Raton, FL company founded by Hank Asher, previously of Database Technologies, the company that 'stripped thousands of African Americans from the Florida voter rolls before the 2000 election, erroneously contending that they were felons'. Lovely.

Boca Raton, eh? Yep, there's a spam connection -- Hank Asher also, apparently, bought eDirect.com from noted Boca-based spammer Steve Hardigree (ROKSO record).

The email in the linked article goes on to note that Asher and Hardigree had 'disagreements' regarding 'how eDirect should position itself in the Direct Marketing Community', so I doubt Asher might have necessarily approved of spamming -- but it does appear he had interests in Direct Marketing.

Given that, I suggest a new spin-off strategy for Seisint's 'terrorism quotient' field, courtesy of my mate Luke: terrorist-targeted direct marketing!

Those turrists are in the market for lots of high-profit-margin goods:

  • AK-47s (OK, not a very big margin there)
  • chemical weapons instructions (just download from the internet! but don't tell them that)
  • weapons-of-mass-destruction-related-program-like-activities

All Seisint have to do is SELECT Name,EmailAddress FROM MATRIX WHERE TQ > 120, do a mail run, then watch those non-consecutively-numbered US dollars roll in. Easy!

Caesar’s Palace open wifi

Tech: I should note this here just in case anyone finds it useful. A handy tip for anyone visiting Caesar's Palace; their 'Business Center' doesn't have wifi yet, but (cough) one of their neighbours certainly does ;)

‘Papers!’ and the Hass

Travel: I've just spent a week in the UK; much culture was imbibed, I got to see Michael Landy's Semi-detached at the Tate, met up with some good mates including the pregnant Lean, and was a happy camper overall.

Then I had an 11-hour transatlantic flight, stuck in the middle of a 5-seat row with pointy elbows on both sides; then, best of all, arrived at US Immigration and found myself fingerprinted and had my photo taken, in accordance with their new policies under the US-VISIT program.

Apparently the biometrics equipment providers are a company called Cross Match Technologies. Fingers crossed (arf!) they have better false positive rates than their competitor, Identix.

I'm looking forward to seeing similar false-positive-prone usage of biometric data, for US visitors to other countries in response. (With hilarious results!)

Aside: I wonder how href="http://use.perl.org/%7eMatts/journal/18915">Matt's cooking-related-program-activities injury will affect his biometric profile?

Also of relevance -- apparently Boston are introducing random spot-checks of passenger's papers on their metro transport.

It's interesting that travel by train requires a passport, driver's license, or similar heavyweight documentation -- but one can zip around the country unimpeded by road. Of course, all of this is moot, seeing as the 9/11 hijackers had perfectly-in-order documentation, including driver's licenses, and travelled extensively under their real names and passports. One wonders what exactly all this has to do with the War Against Terror, given that.

Funny: Knight Foundation, featuring a downloadable David Hasselhoff Paper Plane! Don't forget, the song 'Hot Shot City' is particularly good.

The ‘as such’ loophole

Patents: According to Ciaran O'Riordan of IFSO, one key aspect of the EU Council's meeting on the software patent legalisation proposal hinged on the use of the phrase 'as such', to effectively sneak a loophole past the Council members:

I recommend that everyone listen to the recordings of the Council's meeting. Transcripts are also linked from there, but the tone of voice etc. is interesting.

Anyway, basically, the people in the room didn't understand the implications of the text (that's our fault).

Bolkenstein added an amendment: "computer programs will not be patentable as such" - this (rightly) fooled most people into thinking that software would not be patentable. Really, it just means you can't patent software as software, you have to patent "software running on a computer". I think the rejected part of the German amendment would have closed this loophole. .....

Anyway, the point is that the Council members were on our side, we just hadn't told them precisely what we want .... We told them "no to software patents", and they think they've done that. We should have said "no to 'as such"', and similar textual lobbying rather then implication lobbying.

Yahoo! release DomainKeys

Spam: Yahoo!'s DomainKeys proposal for sender auth.

I'm in the UK this week, so commenting in detail isn't too easy right now. But briefly, the big problem I foresee for DK is dealing with mailing lists and forwarders.

I did spot this oddity in the patent license, though:

Yahoo! will grant a royalty-free, worldwide, non-exclusive license under any Yahoo! patent claims that are essential to implement or use any Implementations so that licensees can make, use, sell, offer for sale, import, or yodel Implementations; provided that the licensee agrees not to assert against Yahoo!, or any other Yahoo! licensees of Implementations, any patent claims of licensee that are essential to implement or use any Implementations.

My emphasis. "Yodel"? ;)

But seriously -- patents will make implementation of this tricky for open-source projects, unless those terms are extended to allow the license to be transferable and usable indefinitely.

Patents: argh. That's all I can say for now. :(

IFSO talk update

Ireland: Update update! The Stallman talk is now free (-as-in-beer), apparently. No more updates, any further news will just be on their site. ;)

Compare and Contrast

Compare this recent statement from Minister Mary Hanafin, Minister of State with Responsibility for the Information Society, and this extract from 'Why Microsoft Wins' advertorial written by a Microsoft product manager, Sunday Business Post, 2004-05-02:

ILUG have already written an article in response to this pretty obvious prompting of a government minister by a commercial interest.

(thanks to ompaul at lwn.net for pointing that out.)

Stallman Speaking in Dublin

GNU: Hey, Dublin-based people! Richard Stallman will be giving a talk titled 'The Dangers of Software Patents' in Dublin on May 24, at 19:30. It'll be in the TCD Hamilton building, right beside Pearse St. DART station. I've never seen him speak, but I hear it's definitely worth attending, and his message needs to get out there, further into the Irish software industry and political circles.

Also on patents: good news via groklaw.net -- Germany has stated they plan to vote against the Irish software patent legalisation plan, and some French ISVs are asking Chirac to do likewise.

Newseum link fixed

News: Oops -- I've just realised, that Newseum site I linked to a few days ago actually does change the URLs frequently for those front-page PDFs. However, the changing is limited to using the day of the month in part of the URL, as far as I can see.

So here's bookmarklets that'll do that:

Also -- Breedster explained: Frequently Asked Questions On Viral Marketing. 'Viral', geddit?

Sky News Ireland needs a guidebook

Doh: Garret Collins on the IE-rant mailing list points out a notable 'oops' moment in Sky News Ireland's new promo:

(Original here.)

Debunking the ‘make the patent examiners work harder’ myth

Patents: There's a good discussion over at Joi Ito's weblog on software patents.

Unfortunately, there's a persistent, and popular, fallacy that crops up quite frequently in these discussions, and does so here in the comments:

'much of the processing of patents has been, to use understatement, deficient. An invention that is 'silly or obvious' will likely not pass the approrpiate legal test - if this test is applied by people who understand the inventive technology .... while I agree with most of your observations about deficiencies, I fail to see the logic in your solution (to simply outlaw these kinds of inventions).'

So, what the commenter is saying is that the patenting of software and business methods would be acceptable, if only the 'inventive bar' was raised so that trivial patents were not granted.

The problem with this is that:

  • it ignores the fundamental problem with these kinds of patents, which is
    • that they patent ideas instead of physical inventions.

      A parallel would be to allow the patenting of plot-lines in fiction, meter in poetry, or combinations of ingredients and cooking methods in recipes. These are all ideas, transformed into output 'products' by performing them as input on a set of hardware (books, cooking equipment), in the same way as software patents and business method patents are abstract ideas that operate on input, generating output, when implemented on a CPU. So, should they be patentable, too?

      Patenting of physical designs is fundamentally different from patenting of abstract ideas in one key way. Physical designs must function correctly under real-world physics, and this requires extensive up-front design and prototyping, before they can be turned into mass-produced products.

      Abstract ideas can be developed mentally, and the up-front work required before the idea can be put down on paper is trivial by comparison.

      Consider these EPO patents: EP0807891 (Sun's 'shopping cart' patent) or EP0689133 (Adobe's 'tabbed palette window' patent). The up-front work required to devise these applications is trivial to anyone with a rudimentary knowledge of UI design; the hard part appears to be writing the legalese, and I understand the patent lawyers take care of that part. ;)

      Compare with US patent D0450164, a design patent for a Dyson washing machine. The level of detail, and extensive specifications, is massive, and it's clear a lot of work had gone into the process before the patent application was filed.

      • In addition, the commenter assumes that extensive prior art searches really do take place. From what I've heard from patent applicants, and from what I've observed in the range of granted software patents, this is cursory at best, and generally performed by the patent lawyer and the examiner, not the applicant themselves.

        I've even observed a few patents where prior art, cited in the patent, implemented exactly what was claimed!

Breedster, and a Joyce domain

Toys: My Breedster profile Argh, I've been infected by the Breedster STD!

Apparently, though, there's a way around it through reincarnation, or -- rumour has it -- through touching Asriel, the bug with the power to heal.

In the meantime, paranoia reigns, and this time of crisis has brought out the worst in some bugs:

It's an interesting piece of emergent net-art, if you ask me, but the STD is pissing me off. (it's itchy!)

Literature: Ulysses:

The cat walked stiffly round a leg of the table with tail on high.

-- Mkgnao!

-- O, there you are, Mr Bloom said, turning from the fire.

The cat mewed in answer and stalked again stiffly round a leg of the table, mewing. Just how she stalks over my writingtable. Prr. Scratch my head. Prr.

Mr Bloom watched curiously, kindly the lithe black form. Clean to see: the gloss of her sleek hide, the white button under the butt of her tail, the green flashing eyes. He bent down to her, his hands on his knees.

-- Milk for the pussens, he said.

-- Mrkgnao! the cat cried.

They call them stupid. They understand what we say better than we understand them. She understands all she wants to. Vindictive too. Cruel. Her nature. Curious mice never squeal. Seem to like it. Wonder what I look like to her. Height of a tower? No, she can jump me.

-- Afraid of the chickens she is, he said mockingly. Afraid of the chookchooks. I never saw such a stupid pussens as the pussens.

-- Mrkrgnao! the cat said loudly.

mrkrgnao.com is available ;)

Open Voting Consortium

EVoting: I didn't realise it, but the Open Voting Consortium's 'EVM2003' e-voting system looks excellent. Here's the key point: it produces printed ballots, unlike the DRE (Direct Recording Electronic) systems. Those are what's counted, and those are what the voter verifies. And it's open-source, too, so the source is available.

Here's a good intro from the Baltimore Sun:

Although it's far from a finished product, the system retains what's good about current electronic voting systems. It's voter-friendly, easier than older systems to administer, and accessible to blind voters without assistance.

It also addresses the concerns of today's critics. First, it uses open-source software that's available for public inspection - eliminating the secrecy that outrages critics of today's proprietary "black box" systems.

Second, the software is free and can run on a variety of computer platforms, which makes the system cheaper to acquire and maintain. Third, it creates a paper trail of printed ballots that can be counted by hand or machine in case of disputed elections - without compromising privacy for the blind.

Instead of printing a "receipt" that confirms a ballot cast electronically, it's based on the quaint notion that the best ballot is still a paper ballot. "We didn't see any reason to reinvent the wheel," said Fred McLain, the project's lead software developer.

The ‘Human Shredder’ Rumour

Iraq: OK, I've been keeping quiet on the whole Iraq thing -- so far, it's pretty much turned into what I was suspecting would happen once GWB declared 'Mission Accomplished', and now there's lots more people saying what I previously felt wasn't being said. However, I've just heard something that really winds me up.

Richard Perle was being interviewed on BBC Radio 4's PM show about the torture at Abu Ghraib. He made a comment to the effect that 'at least the Abu Ghraib incidents weren't as bad as Saddam's use of the human shredder'.

First off, two wrongs do not make a right, and the neocons needs to stop assuming that this is an excuse.

Secondly, the human shredder story is uncorroborated rumour from a single person in Northern Iraq, and no evidence has ever found to support it. All evidence points to the opposite.

But if we let it pass without debunking, this one's going to go down as 'history', alongside the 'babies thrown out of incubators' story from Gulf War I, and the 'bayoneted babies' story from 1914.

good interview with Philip Greenspun

Open Source: ITConversations: Doug Kaye and Philip Greenspun (via Tony Bowden).

Very interesting interview overall. Philip notes that he didn't see weblogs coming because 'it never occurred to me that relatively minor changes in how you allow people to author would cause such a revolution'. I must admit, I was the same. As far as I could see, it was just another HTML page, being updated frequently -- it took me quite a while before I realised the social aspects, of conversations taking places in a group of weblogs, was making a whole new thing.

Also, there's a great few paragraphs where he discusses how sensitive to supply-side economics the whole 'building a business on open source' thing is. Search for 'a dollar cheaper and a day faster' to find it.

‘The EU is a democracy only on paper’

Patents: The Irish EU Presidency keeps on rolling.

FFII notes that 'this Wednesday, the Irish Presidency managed to secure a qualified majority for a counter-proposal to the software patents directive, with only a few countries - including Belgium and Germany - showing resistance. (This 'compromise' is the most pro-patent text yet,) discarding all the amendments from the European which would limit patentability. Instead the lax language of the original Commission proposal is to be reinstated in its entirety, with direct patentability of program text fragments added as icing on the cake.'

'The proposal is now scheduled to be confirmed without discussion at a meeting of ministers on 17-18 May, unless one of the Member States changes its vote. In a remarkable sign of unity in times of imminent elections, members of the European Parliament from all groups across the political spectrum are condemning this blatant disrespect for democracy in Europe.'

Some quotes from MEPs about this behaviour:

  • Daniel Cohn-Bendit, chairman of the Greens/EFA Group: 'The national patent officials in the Council do not want "harmonisation" or "clarification". They merely want to secure the interests of the patent establishment. If they don't get what they want, they simply bury the directive project and try to find other ways to get around the existing law.'
  • Anne Van Lancker, a Belgian MEP of the Socialist group: 'the current Council proposal was written behind closed doors by patent office administrators.'
  • Piia-Noora Kauppi, Finnish MEP of the European People's Party: 'the Council is not taking the will of Europe's elected legislators into account.'
  • Pernille Frahm, Danish member and Vice-Chairwoman of the GUE/NGL group: 'The patent administrators in the Commission and Council are abusing the legislative process of the EU.'
  • Bent Hindrup Andersen (MEP, DK, EDD): 'The approach of the Commission and Council in this directive is shocking. They are making full use of all the possibilities of evading democracy that the current Community Law provides.'
  • Johanna Boogerd-Quaak (MEP, NL, ELDR): 'the Irish Presidency has buckled under the interests of American Companies. A handful of big American Companies may actually profit from software patents, but it is a very bad deal for innovation in European SMEs. Additionally, the Council is showing contempt for parliamentary democracy. We must make sure that after the elections there will again be a majority in the European Parliament that is willing to show its teeth.'

Amazingly, the Council proposal documents aren't even being released to the public, 'due to the sensitive nature of the negotiations and the absence of an overriding public interest'; the FFII got hold of them via a leak.

There's still a chance that this can be reversed; this still needs to be confirmed at the Competitiveness Council of Ministers on 17-18 May. This isn't a dead cert just yet. As a result, FFII are proposing more demonstrations and another 'net strike'.

It's unclear whether writing to anyone will make a difference, at least for people in Ireland, however; everything I've read seems to indicate that our representatives on the EU Competitivity Council are not on our side.

Specifically, the only names I can find regarding this Council are Mary Harney, pro-business, anti-regulation right-wing leader of the Progressive Democrats and 'President-in-Office' of this committee; and the staff of the Department of Enterprise, Trade and Employment's Intellectual Property Unit.

(Of course, Harney at least can always be voted out at the next elections, and I'd strongly suggest anyone working in the field bear that in mind if this gets passed!)

Newspaper front pages from Around the world, as PDFs

News: Newseum: Today's Front Pages (Flash map view). A great site;
the best thing about it is, a double-click on each newspaper's 'dot' will pop up their front page as a larger image in a new window, and give you a URL for a full-page PDF file.

Best of all, those full-page PDF links update every day with that day's front page... for example, these are eminently bookmarkable:

Excellent!

A bit like The Guardian's Digital Edition, but a whole lot cheaper and simpler.

E-Voting debacle gives us the first F-word in the Dail

EVoting: No ducking the f*ing question . . . did he say it? (Irish Independent) (reg req'd, see bugmenot):

A direct transcription of Mr (Michael) Smith's comments reads: "Let them, f* it, we'll say no more - we'll say no more."

Given the barrage of taunts he was facing in the Dail at the time, it is quite plausible - and in context - if the 'eff it' is replaced in the sentence by 'duck it'.

The Opposition was continually interrupting Mr Smith when he was trying to put a brave face on the Government's squandering of EUR 52m on e-voting. Ill at ease and clearly keen to avoid the onslaught from the Opposition, the minister seemed to know he was on a hiding to nothing.

Labour leader Pat Rabbitte, who has made baiting Michael Smith a career work-in-progress, was pursuing his quarry with noticeable effect. Mr Smith's eyes narrowed as his mouth tightened in frustration, he turned to address his frontbench colleagues, and uttered the sentence that has turned him from Tipperary choirboy to bad-boy rapper.

It seems that the F-word isn't specifically prohibited in the Dail -- "the 'Salient Rulings of the Chair, Second Edition', the book which governs behaviour in the Dail, doesn't specifically forbid the use" of the word. It does, however, apparently prohibit the words "brat, buffoon, chancer, communist, corner boy, fascist, gurrier, guttersnipe, hypocrite, rat, (and) scumbag." ('corner boy'?)

Some history: Unisys and the GIF patent

Patents: I've just come across Tim Oren's page on the Unisys GIF patent furore of 1994-5. Tim used to be VP of 'Future Technology' at CompuServe.

The GIF furore, in case you missed it, was one of the most far-ranging software patent debacles to date. Here's what happened...

Compuserve was one of the biggest online services at the time. In 1987 they'd created GIF, an efficient image file format, for public use, with a very liberal license. As a result, everyone and their dog wrote software to read and write GIF files (including myself ;).

GIF, like many other tools of the time, used the LZW (Lempel-Ziv-Welch) file compression scheme, which had been widely published without any indication that it was considered proprietary. LZW was pretty much the de-facto standard for file compression in the early 90s, in the same way that 'gzip' is nowadays.

However -- 7 years later, in 1994, Unisys suddenly announced that they had filed for, and eventually received, a patent on the LZW algorithm. As Tim wrote at the time, this was a 'submarine' patent. (Unisys had owned that patent since 1985, and pursued hardware licenses -- but all and sundry believed that the patent didn't cover software-only implementations.)

Unisys shook downbrought an infringement suit against Compuserve, who had published the GIF standard and implemented it widely in their software. Compuserve had 'no recourse but to settle'.

(Interestingly, it appears that at the time, Unisys seemed to think that GIF decoders needed licenses as well -- popular thinking nowadays is that only GIF encoders need licensing, but Unisys didn't think so at that stage at least.)

There is a happy ending -- thankfully, free software saved the day. ;)

As Tim writes, Thomas Boutell, Jean-loup Gailly and others came up with PNG; Jean-loup and Mark Adler wrote GZIP; and LZW was consigned to the dustbin of unusable technology for most new projects. Old projects, of course, had to go through some redesign pains to achieve the same goal.

BTW, it's worth noting that, even though the Unisys patent has expired, it's still not safe to dust off LZW. GNU (and others) believe that there's another patent filed on the same algorithm independently by -- guess who -- IBM, which doesn't expire until 11 August

  1. The thoroughly-competent USPTO strikes again ;)

The lesson: be careful when implementing published standards. Nowadays, the IETF requires that contributors disclose 'the existence of any proprietary or intellectual property rights in the contribution that are reasonably and personally known to the contributor'. But in this case, the patent was owned by another body, Unisys, and the contributor (CIS) didn't know that, so that wouldn't have helped.

So, the real lesson: Just Say No to software patents ;)

BBCtorrents and some bits

Television: Tony Bowden: BBCtorrent? 'Later this month, the BBC will launch a pilot project that could lead to all television programmes being made available on the internet.' I have my fingers firmly crossed here. This could be really excellent news. Of course, not being located in the UK could make it not-so-easy to actually watch them from here, but the underlying thinking is really cool.

Tech: LayerOne. Weekend conf in LA, with Danny O'Brien -- think I might just tag along!

Patents: Posting this here so I can find it in future. Here's a /. comment saying 'if it becomes impossible to safely develop software in the US and EU due to patents, innovation will move to India and China'. This isn't quite true anymore -- my response, noting the Brazil/Glaxo/AZT case.

MS sponsoring the Irish EU Presidency

Europe: Given the Irish EU Presidency's recent passing of the IP Enforcement Directive and the second attempt to get the Software Patents directive through using the EU Council of Ministers, is it really appropriate for Microsoft to "contribute" to the Irish EU Presidency?

MS reportedly see software patents as a very important part of their strategy to deal with open source, as they noted way back in 1998 in the leaked Halloween I document.

MS is reportedly applying for 10 new patents a day (or is it per week? eWeek can't decide. anyway.)

It's pretty clear that MS want to 'de-commoditize' open standards, using software patents; they said so in the Halloween doc. Their XML Word-processing patent, which claims to patent the use of two open standards (XML and XSD) in a word-processing file format, is a great example of locking up an open standard as a patented, proprietary format.

As a result, they'd have a vested interest in helping the EU Presidency to decide that software patents should be legalised in the EU. A more conspiracy-minded type than myself might read something into their 'contributions' accordingly ;)

Now, it could be all touchy-feely niceness from MS. This eWeek article quotes David Kaefer, Microsoft's director of business development for intellectual property:

According to ... Kaefer, "We'll make our IP available to all comers, open-source or not." Kaefer added that Microsoft isn't focused on what garage-shop developers are doing ...

Sounds lovely, except it didn't happen in this case, where MS threatened an open-source developer with patent litigation:

Today I received a polite phone call from a fellow at Microsoft who works in the Windows Media group. He informed me that Microsoft has intellectual property rights on the ASF format and told me that, although I had reverse engineered it, the implementation was still illegal since it infringed on Microsoft patents. ... At his request, and much to my own sadness, I have removed support for ASF in VirtualDub 1.3d, since I cannot risk a legal confrontation.

E-Voting nobbled in Ireland

eVoting: Success! The use of e-voting systems for the June elections in Ireland has been abandoned, after a severely critical report from the Commission on Electronic Voting. Take a look at the report here. Some bits:

  • They particularly do not like the continual revision of the software, noting the 'large number of new versions of the software since the original ... review' and 'the fact that new versions of the software continue to be issued in the run-up to the June elections'.
  • 'as the software version proposed for use at the forthcoming elections is not as yet finalised, it is impossible for anyone to certify its accuracy'. (my emphasis)
  • They were not given access to 'the full source code'.
  • They found a bug! 'certain of the tests performed at the request of the Commission identified an error in the count software which could lead to incorrect distributions of surpluses'.
  • 'experts retained by the Commission found it very easy to bypass electronic security measures and gain complete control of the hardened PC, overwrite the software, and thereby in theory to gain complete control over the count in a given constituency'.
  • And they raised the pre-arranged-transfer-pattern hack: 'publication of ballot results in full is a valuable aid in checking the accuracy of the results but this can in theory reveal deliberate voter signatures of low-preference votes which could allow voters to identify themselves in a context of corruption or intimidation'.

The use of VVAT, and changes to the counting procedures to remove randomisation, was outside the terms of reference, unfortunately, so it's not totally over yet. But I can't see the government getting away with re-introducing e-voting without VVAT now.

Finally, the opposition political parties are calling on the Minister to resign.

I've got to say -- nice work to all the concerned citizens who've achieved this, despite the government's continual stonewalling and secrecy.

CAN-SPAM’s first prosecution

Spam: CNN: First four charged under 'can spam' law:

Court documents in the landmark case in Detroit describe a nearly inscrutable puzzle of corporate identities, bank accounts and electronic storefronts in one alleged spam operation.

At one point, investigators said, packages were sometimes delivered to a restaurant, where a greeter accepted them and passed them along to one defendant.

Detroit Free Press: 4 Oakland men cited in 1st U.S. spam case:

The four are accused of secretly commandeering computers that forward e-mail for some of the nation's biggest corporations -- including Ford Motor Co. -- to send millions of junk messages advertising herbal supplements, diet patches and sexual enhancement pills and products.

Other unwitting companies and agencies whose computers were used include Unisys Corp., Amoco Corp., the Administrative Office of the United States Courts and the U.S. Army Information Center, according to a complaint filed in U.S. District Court in Detroit on Wednesday. .....

Unraveling the trail of spam took four months. Berg said that because of the use of proxy servers, trying to trace the spam back to the original sender was difficult. .....

In Karlsruhe, Germany, an Internet security expert and activist named Anders Henke runs what he calls a "proxy pot," a system that simulates a mail proxy but doesn't actually forward mail. It sits on the Internet, looking vulnerable to the sophisticated scanning software used by spammers to sniff out open proxies.

Starting in early January, the complaint says, Henke's proxy pot intercepted 5 million attempts from computer accounts linked to the Michigan men.

More Thoughts on GMail

Mail: I've been playing around with GMail a bit more recently. They've fixed the issues they had with Firefox and keyboard control, and it is nice.

Threading: since I plan to bother a few open-source MUA developers ;), I've written up a thorough analysis of their 'conversation' model, with its 'collapsable history', archive-not-delete approach, etc. Take a look, if you're curious.

HTML: one feature that no-one's commented on, is that GMail does not create HTML mail -- all mail composed through their composer is sent as text/plain only.

This is very interesting, because it suits me just fine. HTML mail causes so many more problems than it solves, especially when full-featured web browser components are used to display it, IMO. I get to see the security exploits this enables, every day in my anti-spam work.

But it's also very significant that nobody else has commented on it -- nobody misses it!

Phantom Labels: another interesting thing I've noted: sometimes a mail will appear in your Inbox with a 'spam' label, even though you've never defined one. It's not in the 'Spam' folder; it's in your inbox.

Aaron has a good theory on what this is, and I think he's right -- he suggests it's when ' the two emails are in a conversation (same subject); one is marked as spam, one isn't. So the conversation (which is what appears in your inbox) gets two tags: Spam, and Inbox. So when viewing the list it looks like it gets the Spam tag.'

Also, while I'm here -- details on LiveJournal's distributed filesystem, MogileFS, which apparently 'will be open source'. Link via acme.

EU IPR Enforcement Directive Approved By Council

Politics: FFII reports that the 'IPR Enforcement Directive', the law proposed to deal with 'IPR infringement' by the wife of the CEO of Vivendi Universal, has just been approved by the EU Council.

Another glorious moment of digital cluelessness by the Irish presidency. But then, it had already been passed by the parliament. Reminder: that page lists the Irish MEPs and how they voted on a key amendment, which would have inserted safeguards so that 'surprise raids ... in the middle of the night by private security firms, on the flimsiest evidence' would not be possible.

It's now done in Europe. Next step is to deal with it when the member state governments implement it (which has to happen by June 2006).

Neologism Watch: ‘Neverendum’

Language: So, here's a word worth noting -- 'Neverendum'. This Guardian article notes:

(Quebecois politician Mario Dumont's) meteoric ascent is a sign of how weary voters in the French-speaking province have become about what has been dubbed the 'neverendum referendum', the debate over whether Quebec should become a country. It has dominated Quebec politics for three decades.

It looks like Ireland's ever-recurring referenda (motto: 'if at first the Government fails to get their desired result, try, try again') have driven the word into usage over there too, judging by this Irish Family Planning Association press release:

'The idea of holding another pro-life neverendum is clearly ludicrous and serves only to distract from the daily reality of Irish Abortion.'

And there's even a song, referring to the Nice referendum:

'The Government should not patronise us but should respect the views of the people,' he said. Or, as he puts it in verse, 'What part of our No don?t they understand?'

Pat Kenny tangles with Aileen

Ireland: So on Saturday last, Pat Kenny, the host of the Late Late Show (Ireland's longest-running chat show) had Aileen O'Carroll on to talk about the Dublin Grassroots Network's planned May Day march.

The Gardai have been doing their damnedest to block the march, gaining power to deploy armed police, and in turn, the PR big guns have been deployed in force to get scare stories printed, with the tabloid journos utilizing their considerable wiles in the process.

So, it's culminated in an appearance on the Late Late for Aileen. By all accounts, it went very well.

Apparently, another great moment of reported hilarity was a lengthy discussion between Pat Kenny, the tabloid journalist, and a 'security expert' as to whether there would be 'agent provocateurs' present. It seems all agreed there might just be. One wonders if they thought to look up the word beforehand:

Agents provocateurs are also used in the investigation of political crimes. Here, it has been claimed that the provocateurs deliberately seek to incite ineffective radical acts, in order to foster public disdain for the political group being investigated; and to worsen the punishments its members are liable for. Within the United States the COINTELPRO program of the Federal Bureau of Investigation had FBI agents posing as political radicals in order to disrupt the activities of political groups the U.S. government found unacceptably radical. The activities of agents provocateurs against political dissidents in Imperial Russia was one of the grievances that led to the Russian Revolution.

TRIPS, WIPO and the WTO doing the right thing on software patents?

Patents: The pro-software-patent lobby has frequently stated that TRIPS -- the Treaty on Trade Related Aspects of Intellectual Property Rights (TRIPs), signed on 1993-12-15 as a constituting document of the World Trade Organisation (WTO) -- requires that software be patentable. For example, here's one from the International Chamber of Commerce:

ICC believes that the directive should follow current practice in the EPO and a number of EU member states and make it clear that computer program products can be claimed. To disallow such claims in the directive would create great legal uncertainty for holders of such patents already granted. Prohibiting product claims would also render enforcement of patents difficult and raise questions with respect to TRIPS compliance. TRIPS requires patents not only to be available, but also to be 'enjoyable' in all areas of technology.

Well, it actually appears that the treaty may state exactly the opposite! Christian Beauprez, a UK-based consultant, has taken a closer look at the details, and come up with this:

TRIPS Article 10.1, 'Computer programs, whether in source or object code, shall be protected as literary works under the Berne Convention (1971).'

WIPO Copyright Treaty Article 4, 'Computer programs are protected as literary works within the meaning of Article 2 of the Berne Convention. Such protection applies to computer programs, whatever may be the mode or form of their expression'.

This includes the execution or processing of a program, as demonstrated in the EEC software copyright Directive 1991, 'the permanent or temporary reproduction of a computer program by any means and in any form, in part or in whole. Insofar as loading, displaying, running, transmission or storage'

They also stipulate that exceptions to exclusive rights of authors are to be limited to 'special cases' which do not conflict with a normal exploitation of the work and cannot be prejudicial to the author's rights. (e.g. the rights to sell,rent,broadcast,give away,translate, and generally enjoy.).

... Authors cannot own underlying ideas, but inventors can as part of their 'invention'. When the field of software (aka data processing) is opened up to 'inventors', they can block authors from exploiting their works on the grounds that they own the 'underlying ideas'. Therefore this is prejudicial to the rights of authors and illegal under all these Treaties.

There's lots more at Christian's site. FFII, one of the main anti-software-patenting players in Europe, have agreed that this is a key point in their TRIPS analysis:

In summary it can be said that the European patent establishment is 1. refusing to clarify and concretise the meaning of the TRIPs treaty; 2. wrongly equating the TRIPs treaty with 'US practise', using threats of alleged TRIPs-incompatibility for purposes of fostering Fear, Uncertainty and Distrust (FUD); 3. trying to impose a sui generis software patent regime on Europe which is incompatible with the TRIPs treaty.

GMail and Anne

Spam: Anne Mitchell on GMail's spam filtering -- sounds like her results are actually worse than mine were. But the ads worked well:

... just today, in an email from Mrs. Nwakama Ani, the wife of the late James Ani, a farmer in ZImbabwe, asking me to please help her to export $50million dollars which her late husband amassed, Gmail's Adsense very thoughtfully offered me 'Cheap airline tickets from the USA to Zimbabwe'. You know, just in case I want to go over there and help her personally.

Anne's spam weblog looks like good stuff -- I've added it to the blogroll...

Machine Molle

Art: Machine Molle bill themselves as 'post-production', but I suspect that's understating their work -- their site has Flash-playable copies of their videos for Royksopp's 'Remind Me', Air's 'Electronic Performers', and a recent ad for Areva, a Canadian power company. All are simply amazing. Go take a look. (link via Joe)

Closed-group Filesharing

Net: So, it looks like closed-group filesharing will be appearing in several more implementations soon. NTK writes this week, 'the big new (yet old) killer app this year is going to be a some dinky little program that lets you easily and selectively share individual files with groups and sub-groups of your friends.'

It's interesting to see this -- it's been several years in the offing. So far, there seems to be two main angles: secure collaboration in a private workgroup, and private filesharing in a closed group, defined socially (I've taken to calling this the 'playgroup' ;).

Groove is an example of the 'workgroup' idea. However, to my mind it's been crippled by a strict one-platform policy, and possibly because it's proprietary, commercial software. Still, nice idea.

Several MS researchers helped kickstart the 'playgroup' idea with this paper: The Darknet and
the Future of Content Distribution
. Clay Shirky's thoughts.

WASTE is the classic implementation of a 'playgroup' darknet, sadly killed off due to ownership issues. NTK state that it 'was too crypto-tastic to succeed', but I don't see that -- it was actually excellent software; in particular, its entirely-decentralised and public-key-crypto-based architecture worked surprisingly well in practice, even with NAT, firewalls and all that problematic stuff.

More of the up-and-coming projects -- at least the ones that intend to take heed of 'playgroup' needs -- need to take cues from this app. The only negative in their approach is that the 'gating' of new members is too relaxed; all it takes is for one existing member to accept them into the group, their public key is flooded out to all, and pretty much everyone is set to accept the new key by default.

Robert Kaye has written about his thoughts on how this all should work in this ETCON presentation and this O'Reilly Network article. I'm not sure that a loosely-coupled SSH-based system is easily deployable, though; IMO an 'all-in-one' app is easier to get installed and deployed.

iFolder is Novell's new tool in development. This sounds pretty interesting, although it seems very strongly workgroup-oriented, as does Foldershare, a new Windows-only app from some 'ex-AudioGalaxy staffers', apparently.

Both operate by using some kind of file-sync algorithm, along the lines of rsync or Unison, to synchronise multiple copies of a dir across a network. (Here's hoping it's up to the standard of Unison.) So very large collections will be duplicated throughout the net -- which may actually be quite cool for backups, but strikes me as bad news for users on slow links.

And finally, there's Clevercactus Share -- this sounds interesting, is cross-platform, and is now in beta, apparently. Haven't seen it, though ;)

So far, techie details on the internals of the latter three systems are scant; it'll be interesting to see how heavily they tilt towards the 'workgroup', how well they deal with firewalls and NAT, the extent of crypto use, etc. But nice to see more software entering the field...

Some stats on GMail’s spam filter

Update: greetings, visitors from 2006! Please pay no attention to these figures, they're from 2004, and both GMail and SpamAssassin have undergone major changes since those days. Historical interests only.

So, I set up a .forward to forward all my personal mail to GMail to see how it coped with my spam load, and compared it against the personal SpamAssassin install I'm running these days. Here's the results:

  • test start: Mon Apr 12 15:50:39 PDT 2004
  • test end: Tue Apr 13 18:26:45 PDT 2004
  • total spam messages received by both during the test: 210
  • total ham messages received by both during the test: 528

The SpamAssassin results:

  • true positives: 189
  • false positives: 0
  • false negatives: 21
  • true negatives: 528
  • FP%: 0.00%
  • FN%: 10.00%

The GMail results:

  • true positives: 144
  • false positives: 7
  • false negatives: 66
  • true negatives: 521
  • FP%: 1.32%
  • FN%: 31.42%

So, not too hot. But there are extenuating circumstances! ;)

  • The GMail false positives were not 'typical' mail, whatever that is -- all of them were Mailman 'administration required' messages regarding spam in Mailman mailing list queues. I'd only be annoyed if I was a GMail user administrating Mailman lists. And it turns out there's a bug in current dev SpamAssassin that now does the same thing...
  • presumably, GMail allows some element of per-user probabilistic classifier training -- if so, some 'move to Inbox' might also sort those out quite quickly, I'd guess.
  • GMail seems to be a four-phase classification system. Messages can either go into: 1. the inbox, 2. the spam box, 3. the inbox with a little green 'Spam' indicator, or 4. the spam box with a little green 'Inbox' indicator. Not sure what the latter two do, but they may indicate some level of 'unsure' as per spambayes; worth noting that most of the FNs in the Inbox did not get the green 'Spam' indicator beside them, though.
  • I used a .forward to bounce the traffic over. So if GMail includes spam-evasion at the SMTP level, along with whatever content-filtering and probabilistic classification they're using, they wouldn't get the benefits of that.
  • SpamAssassin has the benefit of some user configuration; I'd got a couple of my spamtrap addresses blacklisted in the SpamAssassin config, and my Bayes databases have been trained using SpamAssassin's autolearning.
  • this is all really unscientific, and it's a really small sample ;)

Surprisingly, all the SpamAssassin mailing list traffic discussing spam, throwing around spammy URLs and phrases, didn't get caught, however; probably because the volume of spammy phrases in those is less than in the Mailman admin stuff.

Blocking mail with no Message-ID

Spam: Bram shares a spam-filtering tip -- 'most of the viruses I get have a Message-Id tacked on by the local mailserver. A little bit of messing with procmail and suddenly my junk mail level is under control.'

This is what the SpamAssassin rule MSGID_FROM_MTA_SHORT does. It gets:

  4.432   6.7680   0.0560    0.992   0.94    3.67  MSGID_FROM_MTA_SHORT

6.7680% of spam is hit, but so is 0.0560% of ham mail -- which makes it 99.2% accurate. By default in 2.6x, it gets a score of 3.67 points.

There's a lot of divergence between people's corpora -- for instance, I currently have no ham mails that hit this, so it's 100% accurate for my current mail collection; but some other people have an 80% hit-rate.

This is because some large-scale legitimate mass-mailers -- for no apparent reason -- also omit the Message-ID when they send the message across the internet. This isn't quite a contravention of RFC 2822, but that RFC strongly recommends using the header:

Though optional, every message SHOULD have a 'Message-ID:' field.

(see RFC 2119 for what 'SHOULD' means -- it's a strong recommendation.)

The moral for legit senders: make sure you read the RFCs before you start sending SMTP; otherwise you'll look like a spammer.

The moral for spamfilter developers: watch out for the legit bulk mail senders; some of them do really bizarre things with SMTP. ;)

Wildfeeds

TV: from the #tvtorrents FAQ: 'Wildfeeds' are 'a transmission by the network to distribute the episode before it airs around to the tv relay stations. You need to be in the correct location and have a large satellite dish in order to receive them.'

Word for the day!

Good Guardian article on Spam

Spam: Guardian: Incredible Bulk, by Danny O'Brien. A great article from the
'Spam and the Law' conference. 'This is why people such as Richter are appearing from the shadows. They have a choice: turn legit, or risk an increasingly criminal lifestyle.'

Also spam-related: Code Fish Spam Watch, which lists and dissects phishing attacks, in great detail. Some of those trojans are exceptionally sophisticated -- such as this trojan targetting Barclays online banking, which actually takes screenshots of a CAPTCHA-style login protocol. Scary!

Ireland’s Disastrous EU Presidency

Patents: Disastrous for European software developers, that is.

It looks like Ireland's EU Council Presidency is pushing through some nasty stuff on behalf of the European Patent Office. FFII says:

On all points where substantial controversy exists, the Council Working Party has taken the most hardline pro-patent view of all parties. They make patentability hinge on the word 'technical' and yet refuse to explain what that word means. They have refused the interoperability exemption which even the Legal Affairs Committee had accepted. They have rejected the freedom of publication. They are insisting on making programs directly claimable, something which even Arlene McCarthy and the Commission did not advocate.

Nokia's Patent Department is leading the PR push:

The (Nokia call-for-support) letter calls on ministers to drop their objections, and to support a draft text issued by the Irish Presidency on March 17th: 'All of Europe's innovators, including individual inventors, small and medium size enterprises (SMEs), as well as large multinational companies, require patents to protect their inventions, provide incentives to undertake research and development in Europe, and to promote licensing and technology transfer', claims the letter.

'Nokia doesn't seem to be counting Opera among the European innovators', comments Håkon Wium Lie, CTO of Opera Software Inc, an innovation leader in the web browser market and producer of much of the software used in Nokia's mobile phones.

Note that it's the Patent Department of Nokia, not necessarily Nokia's top brass, pushing this -- here's a relevant anecdote from FFII:

The patent officials never see the CEOs themselves, and when they appear in public, their thinking on patent matters may surprise the audience. Last week Airbus CEO Peter Kleinschmidt was invited as a pro-patent speaker to a panel in Paris but then, during his speech, congratulated his co-panelist Michel Rocard for his important contributions to containing the expansion of the patent system, which, as he described in detail, was slowing down innovation at Airbus.

(The economic studies and the US' Federal Trade Commission both concur, incidentally. But it's pretty unlikely a patent lawyer will say the same thing in public ;)

On the other side, 15 MEPs have signed their own Call For Action which points out that 'patent professionals in various governments and organisations are now trying to use the EU Council of Ministers in order to sidestep parliamentary democracy in the European Union' and urges the Council to 'refrain from any counter-proposals to the European Parliament's version of the draft, unless such counter-proposals have been explicitely endorsed by a majority decision of the member's national parliament'.

Let's see if Ireland's presidency will do it the democratic way, or in a back-room deal, over all our heads...

Muff News

Travel: I'm just back from a great road trip around Nevada and Arizona -- lots of fun was had, and I even came out $100 up on the blackjack!

In other travels, my mate Eoin recently visited Muff, Co. Donegal, and made sure to get a picture of the event.

Muff is well-reknowned as one of those towns with a silly name; the story goes that they even have a SCUBA diving club, called -- guess what -- "Muff Diving Club". Sadly, the reports are apparently greatly exagerrated. Eoin writes:

I have been hearing the story of the 'muff diving club' for the last 10 years, and now i can categorically state that its an urban legend. No such thing. There was a 'top muff' petrol station though where we picked up a few keyrings. The girl behind the counter was trying to give us all 200 keyrings left in the bag as she was so sick of muppets like us coming in for a laugh.

Finally! My round-the-world journey pics

Pics: After nearly 2 years of peripateticism, I've finally managed to track down my CD-ROMs of scans of a select few of the pictures I took on the round-the-world trip I took back in 2001-2002 (well, it wasn't quite round-the-world, just Down Under and Asia, but who's counting).

Here they are:

And some highlights:

McCarthyite smearing, 21st-century style

Politics: The massive opposition to e-voting without a VVAT by Irish Citizens for Trustworthy Evoting and others, has clearly got Minister Martin Cullen thoroughly needled.

As John Lambe points out here, in the Dail on Wednesday he stated that ICTE are 'not experts in this field', 'have no expertise or international accreditation', and best of all, he has resorted to the 21st-century equivalent of calling ICTE 'reds under the bed' -- they are apparently 'linked to the anti-globalisation movement'. Here's a cut and paste from the online transcripts:

Mr. Bernard Allen, FG: Electronic voting is a good idea but this system has been badly thought through and public confidence has been badly shaken by a Government unwilling to listen to anyone but its own so-called experts. The Government has called the introduction of this system a step forward, a point reiterated by the Minister. I submit that it is a retrograde step based on insufficient knowledge on the use of technology. The Minister has a new toy and thought everyone would like it. They do not. The Irish Computer Society said: 'Any electronic voting system must include a paper-based voter-verified audit trail.' The Minister in his arrogance recently said these people were cranks and Luddites.

Mr. Bernard Durkan, FG: Are they cranks?

Mr. Martin Cullen, FF: They are linked to the anti-globalisation movement. The Deputy should check them out. They are all the same.

Mr. Allen: It is all a--

Mr. Cullen: If Fine Gael bases its policies on such people, it is no wonder it is in decline.

Mr. Durkan: The people concerned are computer experts.

Mr. Allen: We do not know what the Minister's policies are and where he stands on any matter.

Mr. Paul Kehoe, FG: The Minister should know more about policy having been a member of more than one party.

Mr. Allen: Irish technology experts have told the Government its system must include a paper-based voter-verified audit trail.

Mr. Cullen: They are not experts in this field.

Mr. Allen: The Minister has made a serious allegation about genuine people--

Mr. Cullen: They are not accredited to anything. They have no expertise or international accreditation.

(Interruptions).

Mr. Michael Ring, FG: Fianna Fáil are experts on everything. They have filled every tribunal in the country.

Mr. Allen: The Minister has come to this House and--

Acting Chairman (Jerry Cowley, Ind): Deputy Allen should direct his comments through the Chair.

Mr. Allen: The Chair should ask the Minister to cease interrupting.

Mr. Cullen: Such comments are pathetic. It is no wonder Fine Gael is in such a disorderly state.

Mr. Ring: Fianna Fáil are the experts.

Acting Chairman: I remind Members that this is not a Committee Stage debate. We are dealing with Second Stage and I ask Deputies to allow Deputy Allen to continue without interruption, please.

Mr. Allen: The Minister has vilified people who cannot protect themselves.

Mr. Durkan: Outside the House.

Mr. Allen: The Minister should withdraw the allegation against--

Mr. Cullen: I have not vilified them. I said they are not accredited--

Mr. Allen: The Minister said they are linked to the anti-globalisation movement and suggested we should check them out.

Mr. Cullen: Yes, they are.

Acting Chairman: Deputy Allen, please continue.

Mr. Allen: The Minister should withdraw that allegation against people who cannot protect themselves.

Mr. Cullen: I will not.

Acting Chairman: Deputy Allen, please continue.

Mr. Durkan: The Minister has cast aspersions on people outside this House. In accordance with Standing Orders--

Mr. Cullen: I think they are proud of their links.

Mr. Durkan: On a point of order, the making of such an allegation is not in accordance with the Standing Orders of this House. Perhaps the Minister would like to comment.

Acting Chairman: The Chair has ruled on that matter.

Mr. Durkan: With respect, the Chair has no authority to rule on this matter. Standing Orders apply.

Acting Chairman: That Chair has ruled on the matter.

Mr. Durkan: No, I am sorry, I do not agree. On a point of order, the Minister has cast aspersions--

Mr. Cullen: I paid them a compliment.

Mr. Durkan: The Minister has cast aspersions on people outside this House.

Mr. Cullen: They will regard my remarks as a compliment, a badge of honour.

Antarctica

Antarctica: I'm obsessed with the wierd collision of out-of-control bureaucracy, strategic-interests-disguised-as-science, and normal life in a way off-normal place, that is the US Antarctic program. It's fundamentally a microcosm of what future space exploration bases will be like -- lots of high-faluting science talk, quite a bit of 'making sure we have a strategic foothold' reality, and people getting on with life in one of the most amazing places they can.

Via MeFi, Sandwichgirl.com is a great journal site describing her life way down under -- full of great little tidbits like describing Antarctica as 'the island', ie. 'we are all taking bets to see how long it will be before he's kicked off the island'.

It's great, although thoroughly overloaded from all the attention right now.

File alongside Big Dead Place and The Symmes Antarctic Intelligencer -- highlight:

'Once you shelter one magic elf, you gotta shelter 'em all', says NSF Representative Jack Hjorth. 'I've seen it before. Pretty soon all science comes to a standstill and you're runnin' a magic elf halfway house.'

Protesting Against Software Patents

Patents: The FFII are suggesting a 10-day online 'net strike' to protest against the ongoing attempts to legalise software patenting in Europe.

The Commission and the Irish EU Council Presidency are pushing for unlimited patentability of software, heavily lobbied by multinationals and patent lawyers. They are ignoring the democratically voted decision of the European Parliament from 24 September 2003, which has the support of more than 300,000 citizens, 2,000,000 SMEs and dozens of economists and scientists.

As a result, I'm putting up a protest front page on these sites:

If you support the actions of FFII, please join in, or even attend the in-person demonstration in Brussels! We need to make it clear that the small software developers of Europe do not support these undemocratic actions.

And finally, shame on the Irish EU Council presidency for supporting the EPO hook, line and sinker. Thanks, and I know who I'll be voting for in future...

EFF April Fool

Funny: EFFector Vol. 17, No. 11a April 1, 2004. Some pretty funny gems in this one: USPTO to Start Granting Indulgences, Microsoft Wins Patent for Software Industry Monopolization, and SCO to Sue Over Unauthorized Use of Earth's Resources:

Lindon, UT - On the heels of its campaign against users of the Free Software program Linux, the SCO Group today announced that it will begin a new round of lawsuits against users of other free resources, including fire, water, air and land.

'People think they can just use free things without paying for them,' said SCO CEO Daryl McBribe. 'This kind of 'socialism' is anti-American and a violation of the Constitution. It's up to corporations like SCO to crush that kind of idealism.'

Ca Plane Pour Moi, GMail, and XCP

Music: Ever wondered what the lyrics to Plastic Bertrand's classic belgopunk tune really said? (Apart from 'I am the king of the divan', that is.) Wonder no more. (...ok, maybe these are a bit more likely. 'Ey up!', indeed.)

Mail: Google Mail front page. It has MXes -- but they don't answer yet. No SPF record yet, either ;)

Funny: XCP - the XML Control Protocol 'is a drop in replacement for traditional Transmission Control Protocol, or TCP. With the advent of XCP/IP, connection-oriented networking will finally move from the legacy environment of inscrutable bits and bytes to a structured, human-readable world relying upon XML. XCP is the first 4th Generation Protocol, or 4GP. It is designed for a networking environment that is very fast and very reliable - the Internet of today!'