Skip to content

Category: Uncategorized

interesting sysadmin talk next week in Dublin

Networking: Donal Cunningham, president of SAGE-IE, mails to note an interesting talk on in Dublin next week:

The System Administrators' Guild of Ireland and Dublin University Internet Society present...

What : From the ground up; a greenfield deployment in Liberia

Who : Comdt. Kieran Motherway, Corps of Comms. and IS, Defence Forces

Where: Walton Lecture Theatre, Arts Building, TCD

When : Tuesday the 8th of February, 7 p.m.

Why : The Irish Defence Forces deployed to a greenfield site in Liberia in 2004, and had to build Comms/IT infrastructure from the ground up. Comdt. Motherway will talk about the Irish Army's experiences with this deployment, and just how far removed from an air-conditioned, climate-controlled comms room you can get...

Sounds like fun, and I know a few taint.org readers will be interested ;)

Building a Freevo

Freevo: so I'm planning to build myself a PVR, of the home-built, running Linux with mythTV or Freevo, mini-ITX variety.

So far I'm still at the hardware planning stages, but the price looks good -- around $455 (plus shipping) for a working, thoroughly hackable, silent, set-top PVR system.

(Silence is a key aim here -- last thing I want is something noisy taking over the room. But silence typically seems to cost the dollars, once you get into Shuttle gear and the like.)

If anyone wants to follow along, or provide some tips -- I'm going to track progress (very slowly) on this wiki page. Like all wiki pages, it's editable -- although you'll need to create an account to edit pages there (sorry, anti-spam measure).

BTW, lately, there's been a lot of talk about using a Mac mini as a media center. So I took a quick look -- but wow, it's pricey! $499 + $329 for an EyeTV 200 tuner? Dude, that's over 800 dollars, not include shipping or sales tax. Given whatever extras turn out to be appropriate, I wouldn't be surprised if it hits double the mini-ITX's price.

January 24th: a day of partition table misery

Tech: January 24th, besides being the date the first Apple Macintosh went on sale, is supposedly the day of maximal post-xmas misery. Well, it certainly was for me today.

I decided to power on my old desktop to set it up as a back-room fileserver, and twiddled the partition table accordingly to nuke a few unused Windows partitions and maximise usable space.

Somehow or other, some component of my system decided that it would henceforth be non-bootable. It seems some BIOSes don't like partition tables where a high-numbered logical partition have a lower starting sector than a boot logical partition, or something... GRUB just errored out with an obscure 'Error 17', which apparently means that it couldn't find its boot partition any more.

OK, so I needed a boot disk. But I had 1 laptop with a CD/DVD drive but no floppy drive, and a desktop with a floppy drive but no CD drive (due to hardware failure)... and the original linux boot floppy was long gone, seeing as I'd hardly booted this machine in the duration of two house moves. Argh.

A dinky little Cruzer mini 128MB USB flash drive saved the day. (R)ecovery (I)s (P)ossible is a tiny Linux distro that fits into 27MB, well inside the USB drive's limits; it has an exceptionally helpful and detailed README detailing exactly what needs to be done to create a bootable USB flash drive from its ISO image, using just the generic linux toolchain.

Together with fdisk and parted's 'rescue a lost partition' mode, I was able to get the mangled partition table back into shape, mount the boot disk, change the fstab and grub configuration file, and reboot into a working system. phew!

Many thanks to Kent Robotti, who's done a great job with RIP.

On the other hard -- no thanks to whoever came up with the arcane rules behind the IDE partition table... argh.

OpenStreetMap.org

Map: much interesting geowankery going on in London, where they suffer under the same Ordnance Survey monopoly as we do in Ireland.

This message to their mailing list notes a quote from IKONOS of $1,172.50 USD plus shipping for a 1m Color Geo referenced satellite image of central London, covering 67 square kilometers.

Given 'enough processing', data extracted from that map becomes a Derived Work, and have no copyright restrictions. 'Processing' includes 'vector extraction, classification, etc.'

Now, I worked it out -- central Dublin city centre covers about 3km x 4km. At the named rates for London, that works out at an inexpensive $210! Looks like it was imaged in September 2003.

There's something interesting for a local geohacker to add to their list of projects ;)

(There's also some old Landsat-7 data that may be usable.)

‘Spam Kings’ review

Spam: Before xmas, I received a copy of Brian McWilliams' new book, Spam Kings.

It's a great book -- full of behind-the-scenes details on how the spammers operate, how they get away with it on the sending end, how they try to evade filters on the receiving end, and how they're fundamentally running the usual simple scams that have been around since before email spam came into existence. Well worth reading.

In addition, Brian's continuing to write about spam and spammers at the Spam Kings weblog, and will be giving a talk at this year's MIT Spam Conference, tomorrow.

Anyway, pick up a copy if you're interested in the spam problem -- this is one of the best books I've read on the subject, and this kind of information is essential for an understanding of the people we're up against.

Echo chamber goes crazy about ‘nofollow’

Blogs: Just to expand on a linkblog posting I made yesterday, Google's search team have announced support for a new piece of Google functionality; they'll fix their crawlers to ignore links with a rel="nofollow" attribute, for PageRank calculations, the idea being that spammers will stop blog-spamming once they can't get PageRank out of it.

The blog world has been all aflutter:

BurningBird is right, to a degree. In fact, it's been solved before.

Here's a taint.org posting from November 2003 where I point out that by using a trivial Javascript URL one can link to another page without conferring PageRank. The format is:

javascript:document.location=target

The result looks like this, and work in any browser with a basic JS engine, from IE 3.02 and Netscape Navigator 2 onwards. I've been using it for my referrer logs, among other things, for over a year. I wrote a patch that implemented it for external links in the Moin Moin wiki software.

Amazingly, despite my plugging this idea at virtually every opportunity, it seems nobody noticed! At least, nobody among the people who (it would seem) should be looking into comment spam, thinking about how to deal with it, etc.

Disappointing -- the echo chamber keeps talking to itself, once again. Maybe I'll stick with dealing with email spam instead ;)

Ah, whatever. Anyway, this is a nicer fix; relying on JS isn't a good thing. So nice work, Google.

(PS: worth noting that while this is a good plan, comment spam won't be going away any time soon, as Mark Pilgrim noted. Still, here's hoping it'll help in the long term...)

IPC::DirQueue 0.04 released

Perl: at last, a perl-related posting! I've released IPC::DirQueue 0.04; details of what's changed (summary, a couple of bugs fixed) are at that link.

BTW, thanks to Ask and Robert at perl.org, who are providing free SVN repository and list hosting for CPAN modules! And don't overlook the fact that the mailing list/newsgroups each have their own RSS feed, woot!)

Prescient tsunami spam

Spam: I was just looking back through the archives here on taint.org, and noticed this entry from December 2 last year:

A huge 300 ft. high ocean wave is moving towards your continent. Your and many other cities are in a real danger. Approximate wave moving speed is 700 km/h. cmoym eaaa yypbzz

Please read more about this catastrophe here: (link)

We are strongly urging you to evacuate yourself and your family as soon as possible, even though you may live far away from your city. The tsunami will reach the continent in approximately FOUR hours.

It appears that the spam was a phish attack -- the site in question is full of Internet Exploder exploits. It was 'targeted', at least as well as such things ever are, at Australian readers. AUSCERT issued a warning about it at the time.

But how's about that for timing? Spooky! What did those phishers know?

eWeek’s ‘Spammers Upending DNS’ article

Spam: eWeek recently published an article entitled 'Spammers' New Tactic Upends DNS' , which notes that:

One .. technique finding favor with spammers involves sending mass mailings in the middle of the night from a domain that has not yet been registered. After the mailings go out, the spammer registers the domain early the next morning.

By doing this, spammers hope to avoid stiff CAN-SPAM fines through minimal exposure and visibility with a given domain. The ruse, they hope, makes them more difficult to find and prosecute.

The scheme, however, has unintended consequences of its own. During the interval between mailing and registration, the SMTP servers on the recipients' networks attempt Domain Name System look-ups on the nonexistent domain, causing delays and timeouts on the DNS servers and backups in SMTP message queues.

This had me stumped when I read it, since an email from a nonexistent domain is a pretty reliable spamsign (it's used in the NO_DNS_FOR_FROM rule in SpamAssassin, for example, which hits about 2% of spam), has been a rule in the default ruleset for several years, and there's no sign of that behaviour in our spam traps.

After some discussion, Suresh Ramasubramanian came up with this explanation of what's really happening:

Verisign now allows immediate (well, within about 10 minutes) updates of .com/.net zones (also same for .biz) while whois data is still updated once or twice a day. That means if spammer registers (a) new domain he'll be able to use it immediatly (sic) and it'll not yet show up in whois (and so not be immediatly identifiable to spam reporting tools) - and spammers are in fact using this "feature" more and more!

That does sound a much more likely explanation, and matches what's been seen in the traps.

So: WHOIS, not DNS.

IBM Pledges 500 U.S. Patents to Open Source

Patents: wow, this is amazing news! 'IBM today pledged open access to key innovations covered by 500 IBM software patents to individuals and groups working on open source software. IBM believes this is the largest pledge ever of patents of any kind and represents a major shift in the way IBM manages and deploys its intellectual property (IP) portfolio.'

Even better, they are hoping to begin a 'patent commons' for other companies to join, and the OSI definitions of which licenses are judged 'open' apply.

More details:

Of course, it would be better if it were also safe for commercial software development. But this is a valuable bulwark against Microsoft-style patent tactics.

Web-browser style history for the command line

Code: Here's something I came up with recently -- it's actually an evolution of the idea of pushd and popd, as included in BASH. To quote the POD docs:

cdhistory is a perl script used to implement web-browser style "history" for UNIX shells; as you use the cd command to explore the filesystem, your moves are remembered, and you can go "back" through history, and "forward" again, as you like.

Download the perl script here.

Annoying anti-arab Republican talking points, pt. xxviii

Politics: This moronic comic from Pat Oliphant came up in my comics page the other day, and, after a few days of hearing this particular talking point through the usual propaganda channels, I just saw it again. It pissed me off enough that I took a look at the stats.

Naturally, it's bullshit. The top 50 governments pledging tsunami aid, per GDP:

  • Qatar (#2)
  • UAE (#5)
  • Kuwait (#9)
  • Bahrain (#10)
  • Saudi Arabia (#15)

Given that the USA's at #29, and the UK at #22, I think the arab states are coming up with a pretty good result there.

I guess it's hard to look beyond today's talking points when you're still drawing cartoons at the age of 70.

A Firefox Extension plug

Web: Urgh, I still have this damn cold I picked up in Ireland... sniffle cough etc. More vitamin C needed!

Anyway, just a quick plug for a very deserving Firefox extension, one I haven't seen mentioned widely. It's pretty common, when you wish to print out a web page, that you wish you could get rid of the obnoxious extra-wide sidebar tables, gigantic ads, or other extraneous parts of the page. Well, now you can:

Nuke Anything is a Mozilla/Firefox extension which offers two great features in the right-click context menu:

  • Remove this object: this will remove the object you've right-clicked on -- a table TD, paragraphs, images, IFRAMEs, etc.
  • Remove selection: more usefully, this allows you to select exactly what you want to remove with a left-button drag, then right-click to remove it.

It's really useful. I almost never print anything out these days without scrubbing off a few unwanted sidebars ;)

HOWTO: invalidate a patent application with prior art

Patents: here's an interesting technique I heard recently. (credit: I'm not sure who told me about it, but I think it may have come from or via John Levine.)

If you become aware of a patent application (note: not an issued patent!) for which you are aware of possible prior art, you may be able to help invalidate it, or at least ensure any resulting patent is narrow enough to be relatively sane. Here's how.

  • If you have knowledge of techniques that you believe may be prior art, you can send them on to the filers or the patent examiner. At this stage, the onus is on them to prove that the technique is not prior art for the application (once it's granted, the onus would be on you to prove that it is).
  • The filer also must indicate techniques that they are aware of, that may be prior art, during filing; so CC'ing a public forum with a copy of whatever you send to them, may at some point in the future help indicate that they did not do this.

Of course, you have to go find the patent application number, the contact addresses of the filers, and the contact address for the patent examiner to do this ;) But it beats posting a whinge to Slashdot.

An unnamed patent agent comments:

'I believe an examiner is not under obligation to review art sent directly to them, but certainly the applicant and his agents are required to report any art they come across. That means the inventor as well as the law firm representing them.

You should include a cover letter that you saw their application (give details), and that you believe that what you are sending them is prior art, and that now that they have it, they are obligated to report it to the PTO. The same can be done to their counsel.

Probably, anything sent should be sent with some sort of delivery confirmation, and to make sure that the sending of the prior art is of public record, create a Web site where all sent art is listed, along with destination and confirmation information. This would help show inequitable conduct should the patent later be asserted and the art you provided not be shown as of record in the examination.

Mind you - I have not heard of these being done before (bombarding listed inventors and their agents with prior art, forcing them to have to disclose it), but I think it's a great idea. One caution - if you send too much, you over inundate the examiner, and then really good art could get overlooked during examination.

Separately, please keep in mind that the claims in a published application have probably not yet even been seen by the examiner at the PTO. These are the claims that the applicant would love to have the examiner accept, but until prosecution of the application actually commences (and completes), there's no way to know what claims will ultimately result.'

Update: some good additional points:

'The prior art must have been published or been publicly available at least as early as the earliest priority date of the patent. The priority date is either the filing date, or the filing date of a parent application. This information can be found on the cover page of a patent.

A patent's scope is covered by the claims. The claims define what the invention is. All other material in the patent is supporting material, and usually non-binding. In order to be anticipatory (the best kind) prior art for a particular claim, the piece of art must contain or described every element of the claim you are seeking to invalidate. Note that dependent claims add additional elements that the prior art needs to contain if you want to invalidate the dependent claims as well.

Prior art which is not anticipatory may be used in combination with other art or knowledge at the time to show obviousness. This type of art may have some impact during prosecution of a patent, but if a patent has already been issued, obviousness is a real uphill battle to fight in the courts. Few patents have been invalidated because of obviousness in trials.'

Another attorney notes: 'You can actually send it anonymously if you want. Just keep the certified receipt to prove they got it. As long as they know it exists, the onus is on them to disclose it to the PTO.'

'It's best to send them something printed out or on tangible media, along with a brief note explaining what it is and most importantly, when it was first publicly available. Certified means using certified mail or FedEx or something where you have a valid receipt.

As far as (discovering) who the (filer's patent lawyers) are ... it's usually listed on the patent applications. you can search the USPTO website for them.'

And a report that this technique is now in use: 'some patent attorneys are reporting that this approach is a valid one that people have started using.'

Update 2: More assent from another unnamed patent lawyer:

'Anyone who wishes to do so can send a letter to the Patent Office letting them know of any prior art of which they are aware. The Patent Office will then place it in the application file. Anyone who cares about this patent will surely order up a copy of the application file from the Patent Office, and will come into possession of whatever you sent.

Later you can see whatever you sent them. Go to
http://portal.uspto.gov/external/portal/pair and plug in the serial number (for the desired patent). Click on "image file wrapper".'

It's the right thing to do for any patent or patent application.'

Verizon.net blocks the world

Spam: I'm still catching up, but this is just plain hilarious. Pure, solid-gold, insanity. Verizon.net, the ISP branch of the US telco, has decided that the easiest way to fix their spam problems (uh, spam-receiving problems, that is), is now blocking inbound email from non-U.S. IP ranges:

A little birdie with insider knowledge has confirmed that Verizon is blocking all international IP space from RIPE, APNIC, and more, and is only unblocking specific domains, based on their IP address, when complaints are made and escalated.

According to the source 'the security team management thinks this is going to stop their inbound spam problems.'

Well, it may stop their inbound spam problem, but it's also going to stop that pesky 'wanted email making it to their customers' problem.

A quick check from my Ireland-hosted colo box does indeed indicate that this is still the case, and I can't connect to relay.verizon.net (206.46.170.12):

  : jm ftp 1...; telnet 206.46.170.12 25
  Trying 206.46.170.12...
  telnet: Unable to connect to remote host: Connection timed out

Back, in the flurry of a mini-tornado

Meta: Back. Not even 'mini-tornados' at Dublin Airport can keep me away -- although it gave it a damn good try, with a 3 hour delay, a missed connection, and an overnight stay in Chicago. Arggh.

Mail: I generally leave the laptop at home when on vacation, to do some proper winding down. Not sure it was a great idea this time, since I was joe-jobbed by some pretty extensive spam runs recently, resulting in over 30,000 bounces sitting unread in my email when I got back.

Thankfully, Tim Jackson's bogus-virus-warnings.cf SpamAssassin ruleset (with a few updates) got most of them, with only a few hundred getting past. I should really hack on making those more complete, but some of the bounces are really obscure; along the lines of 'Hi from J Random Luser, Esq.! I no longer use this address because it gets too much spam! Please send to this new one instead: jrluser98@example.com!', generally without any obvious identifying headers that indicate it's an autoresponse.

Sigh -- each of those messages is just utterly random, and I can't see much recourse but to come up with some nasty phrase-based content filtering rules, which I was hoping to avoid. But 29,500 hits isn't bad ;)

I'm not sure they'd be suitable yet for use as default SpamAssassin rules, since they now generally just match any kind of bounce message, not specifically joe-job or virus-forgery blowback. But that suits me just fine -- I can live without bounces, as long as I don't have to suffer the bounce blow-back.

Science: Good news from New Scientist -- they're opening up their archives! NS has consistently the best science journalism around, and I've been a subscriber for years. But until recently, they had a lousy approach to their website -- most of the useful stuff, like the archives, were walled-off, subscriber-only features; a classic case of missing the Clue Train. Well, here's an archive search for 'spam' -- pretty impressive, and most of the short articles are available in full, with only the full text for features and opinion pieces requiring a login.

In addition, they've added a massive batch of RSS feeds. Sadly, no full article text excerpts, however. But still -- getting the clue, eventually -- this way they may actually get links on the web, in place of the mangled and chinese-whispered versions of their articles republished in the UK newspapers...

Ireland: Due to monopolistic pricing of Irish GIS data, consumer GPS maps of Ireland's road system are appalling, and this page collects a few great demos -- for example, MS Autoroute quintuples the distance from Galway to Roundstone! That's a major tourist route, BTW. I knew it was bad, but not that bad...

Anyway, I'm still waaay behind, but slowly catching up.

Xmas hols

Meta: I'm back in Dublin for a couple of weeks over xmas, so I won't be updating this weblog very much. See you in January!

BTW I flew back via Chicago, which is obviously the stopover of choice to Dublin from Silicon Valley -- surrounded by 1 iBook per every 8 passengers. ;)

PS: looks like they forgot Poland!

An Open Letter to Sound System Developers

Linux: after about 3 months of tweaking and twisting, performed by someone who's been using UNIX for over a decade, I've finally got sound working the way I want it on my Linux desktop. In other words, I can hear sounds made by Flash applets, and I don't have to shut down the best music player on the platform every time another app wants to make a sound.

This is pretty clearly absurd.

So here's my open letter to the developers of the various systems (GStreamer, aRts, ALSA, EsounD polypaudio, et al):

  • Please DO do some testing with crappy sound hardware. I don't care if your sound system works great with a SoundBlasterLive 2006 with the kryptonite connectors, I have a laptop, for god's sake. That means software mixing is essential, because cheapo hardware doesn't do hardware mixing.
  • As an extension: please DO include software mixing by default. ALSA's pretty good in general, but having to hack out 55 lines of hand-tweaked config file before software mixing works, is insane. (Especially when the Wiki documenting that is full of notes that some of the magic numbers may not work on your hardware.)
  • Please DO use existing APIs if possible. That means esd. I'm looking at you, aRts. At least the latest sound project, polypaudio, looks like it's getting this right.
  • I DON'T care about network transparency, realtime response, or having a wah-wah pedal effect built into my sound server. That's just silly. Use a modular architecture to allow that in future, but concentrate on getting the basic stuff working first!
  • Please DON'T hardcode output device or output 'sink' names into the source. Looking at the kgst component of KDE here.

Meh.

Anyway, here's the scoop on what I had to do to get software mixing working in both GNOME, KDE, and Firefox, on my Thinkpad T40 running Debian unstable. Once I figured out the magic incantations, it now seems to be working without stutters or hangs.

Sometime in the next few months, of course, I plan to upgrade to Ubuntu Linux, and all bets will once again be off ;)

BSA’s Spam Statistics

Spam: The Business Software Alliance, a UK anti-piracy body representing many of the major software vendors, recently issued a spam-related press release which got a lot of attention in the UK press (they have great press contacts!).

To quote John Graham-Cumming's newsletter on the subject:

1 in 5 British Consumers Buy Software from Spam: that's according to a survey by the Business Software Alliance. I find that a pretty surprisingly high number and considering it comes from an advocacy group that tries to get people to buy legitimate copies of software I expect it's not totally accurate. The one thing I find really surprising from the survey are these two statistics: 23% of spam is read by the person receiving it and 22% of people have bought software. Apparently, 11% of people surveyed like the idea of buying through spam because the software is cheaper.

It's still an interesting figure, but the BSA has come up with some pretty suspect statistics in the past, so pinch of salt applies. As jgc points out, the BSA have a vested interest in making the problem sound worse than it may be in reality.

Still, the survey PDF can be read here, and is worth a look.

EU Software Patent tricks — very fishy antics

Patents: This is really absurd -- according to this ZDNet UK article, it now looks like the EU Council is considering railroading the EU software patent directive through, by hiding it as an 'A-item' in a Fisheries Council Meeting the week before xmas:

Laura Creighton, the vice-president of the Foundation for a Free Information Infrastructure (FFII), is concerned that the EU Council could be contemplating passing the directive without discussion in an unrelated meeting.

'Before today it was possible for generous people to look charitably at this text (the proposed patent directive) as an example of a tragic mistake, not malice,' said Creighton in a statement on the FFII Web site. 'But not with this last-minute manoeuvring.'

'Only the most committed opponent to the democratic process would believe that the proper response to the widespread consensus that there is something profoundly wrong with the Council's text is to race it through with an A-item approval the week before Christmas in a Fisheries Council Meeting. The bad smell coming from Brussels has nothing to do with the fish.'

Reportedly, A-items are dealt with by asking the assembled councillors if they have any objections to any of the outstanding items. They're not listed in detail at the meeting, so this way the directive can be passed in what is effectively a submarine (boom boom!) manner.

Related: Alan Cox has not been invited to the UK Patents office's public meeting on software patents tomorrow.

In a Talkback to ZDNet UK's earlier story highlighting the issue, Cox wrote: 'I too was mysteriously overlooked despite having written to my MP and received an answer.' .... Cox, who has previously been invited to speak on software patents at the EU, said the Patent Office apparently fears 'every word I have to say about their plans'. He went on to add: 'Unfortunately with all the underhand game playing both in the EU council of ministers and in UK government and patent circles it isn't the slightest surprise.'

Also related: Jason Schultz (EFF) on the Commerce One web-services patent auction last week:

Here, the patents at issue were less valuable to companies that actually produce Web services products than they were to firms that produce nothing but lawsuits and licensing threats. In other words, patents like these have become worth more as weapons than as protections for companies competing in the marketplace.

Many have compared these new patent licensing firms to terrorists, and in some ways, the analogy is apt. When the Soviet Union collapsed, one of the biggest worries was that rogue military personnel might sell off one or more of the USSR's nuclear missiles to a terrorist group. Securing those weapons became a top priority. The reason was fear -- fear that the terrorists, who had little to nothing at stake in terms of world peace and national stability, would use the missiles to extort or manipulate the world political climate. Unlike the United States or China, which could be retaliated against and which had a stake in stability, terrorists were essentially immune from attack, and thrived on instability.

With the patents of bankrupt dot-coms, the dynamics are similar. Rogue licensing firms buy up these patents and then threaten legitimate innovators and producers. They have no products on which a countersuit can be based and no interest in stable marketplaces, competition or consumer benefit. Their only interest is in the bottom line.

While profit itself is often a worthy objective, it is not always synonymous with innovation. Every dollar a tech company pays to patent lawyers or licensing firms is one less dollar available for R&D or new hires. Thus, many companies that offer new products end up paying a 'tax' on innovation instead of receiving a reward. When this happens, it's a signal that the patent system is broken. Forcing companies to pay lawyers instead of creating jobs and new products is the wrong direction for our economy to be headed and not the result our patent system should be promoting.

playing around with Google Suggest

Web: Google Suggest, a drop-down list of suggestions -- with hitrates! The one letter hits are interesting, too.

"spam" hitrates, the top 3 (aside from "spam" itself):

  • "spam filter": 6,400,000 results
  • "spamcop": 1,570,000
  • "spamassassin": 1,350,000

in the top 3. getting there!

unfortunately, you have to get as far as "justin ma" before my name shows up, so not doing too great in that competition. ;)

too busy worrying about patents to care about copyrights

Patents: oh, this is painfully ironic.

patents4innovation.org is a PR site set up by EICTA, a consortium of several pro-software-patent multinational companies, to put some PR money into lobbying for the legalisation of swpats in the EU. I've mentioned it before in the context of another boo-boo. Well, here's the next one.

According to FFII, they recently took a Creative-Commons-licensed article from another website, and:

  • republished it without the required attribution to the author
  • translated it, creating a 'derived work', against the terms of the license
  • and then failed to notify readers of the licensing terms, as required

In other words, they managed to infringe the terms of its copyright-based licensing in multiple clauses.

No wonder they claim that patents are required to protect people's inventions. It seems they just don't understand how copyright-based licensing works ;)

(The article's been taken down from the p4i site, but not before the boo-boo was spotted by an eagle-eyed FFII'er.)

Interesting/bizarre recent spam

Spam: some good crazy spam recently -- firstly, some Seventh Day Adventist lunacy:

THE PAPACY IS THE ANTICHRIST THAT IS TRYING TO CHANGE THE LAW OF GOD. DANIEL 7:25

THIS IS THE LAST WARNING.
THE LAW OF GOD IS ETERNAL BECAUSE GOD IS ETERNAL 14:12. MT. 5:17 SATURDAY SEVENTH DAY IS THE TRUE LORD'S DAY. EXO. 20.8-11 SUNDAY IS A FALSE PAGAN DAY. IT IS NOT IN THE BIBLE. IT WAS USED TO WORSHIP SATAN

It runs on in that vein for quite a while. Interestingly, most of the text from there on in is 'gappy' -- in other words, the spammer has inserted spaces between each character of a word -- even inside link addresses. As a result, they no longer work. oops!

And a new one to me -- natural-disaster spam (via Mark Pilkington):

THIS IS AN OFFICIAL WARNING!
fngva uvtt chloez

A huge 300 ft. high ocean wave is moving towards your continent. Your and many other cities are in a real danger.
Approximate wave moving speed is 700 km/h.
cmoym eaaa yypbzz

Please read more about this catastrophe here: (link)

We are strongly urging you to evacuate yourself and your family as soon as possible,
even though you may live far away from your city. The tsunami will reach the continent in approximately FOUR hours.

venbz nwvw exepmi
YOU HAVE BEEN WARNED!

I've removed the link, btw -- the site it links to contains a bunch of nasty malware-installing IE-bug exploits. In case you were wondering: you can tell it's genuine because it says IT'S AN OFFICIAL WARNING at the top.

(ObSpamComment: note -- this here's a good example of why spam is unsolicited bulk email, not unsolicited commercial email; neither are selling anything. one's religious craziness, the other one's trying to r00t your machine.)

Moving House

Life: I'm moving house -- I've just filled about 20 boxes, now to get moving them! Sadly, there's no wifi in range of my new house, so the upshot is I may be offline for a few days. Boo.

Fun Times Ahead with Nathan Myhrvold

Patents: Newsweek: Factory of the Future?:

The dino's ferociously bared teeth hint at elements of Intellectual Ventures' bold business plan. Myhrvold and his partner, former Microsoft chief software architect Edward Jung, have created the quintessential company for the 21st century. It doesn't actually make anything ... Only patent attorneys populate the quiet hallways. ...

Sources familiar with Myhrvold's strategy say that he has raised $350 million from some of the largest companies in high tech: Microsoft, Intel, Sony, Nokia and Apple. Google and eBay also recently invested. With this large bankroll, the company is out buying existing patents in droves. (Myhrvold won't comment on these activities, but sources say he has already purchased about 1,000 patents.) The strategy is to set up a sort of patent marketplace. Patent owners get money upfront for the dusty ideas sitting on their shelves, the investors get the rights to use the ideas without being sued and Myhrvold gets to rent those same ideas to other companies that need them to continue creating products. ...

"We're concerned that these giant pools of patent rights are going to prevent entrepreneurs from entering markets, as opposed to being used to promote innovation," says one worried Silicon Valley venture capitalist.

Now that's scary...

JFK Reloaded

Games: OK, JFK Reloaded is very, very wierd.

Read the insanely detailed FAQ and boggle at the author's obsessive research and fetishistic recreation of the events at Dealey Plaza, November 22nd 1963.

Quite worrying, to be honest!

EFF’s clueless spam filtering white paper

Spam: The EFF are a great organisation -- damn, I even helped set up an organisation based on its goals in Ireland, back in the day! But this white paper is shockingly clueless.

(Note: this posting has been updated. Original left intact, but there's an update below worth noting.)

For example:

Spam Assassin, a popular program that does ad hoc pattern matching, assigns 'points' to various features of an email to determine whether it is spam. ... One of the major problems with this system is that messages from certain countries -- like China, for example -- can be blocked purely on the basis of where they come from and what language they're in. The implications for free speech here are very troubling indeed: ... thus anti-spam technology unintentionally works as a political censorship mechanism.

SpamAssassin does not give points for country of origin, or language the message arrives in, unless the user explicitly either (a) adds rules from an external source, or (b) modifies the 'ok_languages' setting in their configuration, from the default, to specify that they do not want to receive messages in particular languages. No country- or language-blocking happens by default. This is by design.

It's a shame that the authors felt the need to outright fabricate a danger, here.

The white paper features more broad generalisations about 'spam filters', mostly using unsubstantiated friend-of-a-friend stories, without detailed data. And I do know that there have been cases of MoveOn.org, at least, being a source of UBE, in the past -- so it's not valid to claim that this is all a 'free speech' issue; political UBE is still spam.

They need to realise there's a lot of very smart, very reasonable anti-spammers out there, and most of us agree with the rest of their goals, except for their spam position. This is hurting them.

Still, it appears they're finally getting a clue about requiring subscription requests be confirmed using closed-loop opt-in, so that's good. More political newsletters, and political campaigns, need to get this clue -- just because it's political speech does not mean it's not spam. (I have several thousand political spams in my spam folder -- most from that German anti-immigration virus from earlier this year.)

Note that Rod is unsure if they're practicing what they preach...

Update: Annalee Newitz has been in touch, and pointed out that the white paper in fact says 'mails ... can be blocked', rather than 'are blocked' based on country of origin. In other words, it's purely a matter of this being possible, rather than the default, and that administrators apply these customisations.

In addition, she notes that the conclusions recommend that ISPs and administrators of spam blocking systems allow end users to control their own filtering settings, saying 'If a user wants to block all mail from China, great. If a sysadmin does it for a bunch of users without permission, then that is a problem in our opinion.'

So I agree with that. Misdirected outrage hereby turned off ;)

(Mind you, I still think they need to work more with the reasonable anti-spammers... and fix that unconfirmed sign-up that Rod mentioned, if it's really still unconfirmed!)

ApacheCon, and cranes falling into the sea

Trips: So I'm just back from ApacheCon 2004, which took place in the lovely Alexis Park building site ;)

Good fun was had -- very interesting to meet all the faces behind the names from various mailing lists and blogs, and get the inside track on how the ASF really works... there's quite a lot you don't get to understand from the outside, or even from being a committer. So, a useful trip.

Most of the talks were, naturally, very web-oriented -- we'll have to see what we can do about that, next time around! One useful tidbit: I didn't realise, but found out at the conference, that the ASF ConCom are very generous with paying speakers' expenses. So maybe next time I'll join the speaker line-up, too.

A major goal, one we achieved, was an impromptu SpamAssassin developer summit, 5 days sitting down together hammering on bugs and plans, with 4 of the main developers present (myself, Daniel, Theo and Michael). Pretty much achieved, although there were some thorny bugs to deal with... one interesting factor is that we may now be moving towards emulating the Apache httpd's preforking model to deal with a memory/performance issue we're seeing in 3.0.x.

Finally -- this sequence of photos has been cropping up all over the internets. When I saw it, I immediately thought it looked a lot like Ireland -- and Roundstone, Co. Galway, in particular. Sure enough, it appears it is! I guess the Connemara landscape of Roundstone's bay is pretty memorable, after all...

patents4innovation.org

Patents: Patents4Innovation is a new site set up by several European multinational companies to lobby for the legalisation of software patenting in Europe.

Their FAQ is good for a laugh, including an answer that basically says that the Patent Office needs to do a better job, and another which states that opponents of software patents have been unable to demonstrate any 'convincing' evidence of swpats causing economic harm (despite the FTC and PriceWaterhouseCoopers reports, which are pretty high-profile organisations).

But the best quote -- in fact, the only quote -- appears on their Testimonials and Quotes page. Here is in full:

"... successful and profitable high-tech enterprises seem to be characterised by holding patents and copyrights. These instruments might reduce the vulnerability of firms with respect to competition and equip them with a more favourable market position."

Source: the "Observatory of European SMEs 2002 / No 6: High-tech SMEs in Europe", commissioned by the European Commission

(My emphasis.) Of course, there's nothing worse than having to compete on a level playing field. ;)

(Thanks to Christian Beauprez for spotting that!)

New Scientist’s psychic website

Web: The lovely C sent me a link of note -- it's the eglu, 'the world's most stylish and innovative chicken house and is the perfect way to keep chickens as pets'. (She has a thing about keeping chickens.)

So I was all set to link to that on NoMoreSocks.newscientist.com, New Scientist's nifty new xmas-pressies site; but -- get this: it will not load in Firefox 1.0PR, 1.0, or Konqueror at all -- in fact, using telnet, the site doesn't actually respond to requests on port 80 from my linux desktop.

The only browser it seems to work with is MS Internet Explorer in VMWare, presumably using MSIE's psychic powers to contact it without going through TCP/IP.

Mysteriously, it can be lynxed from my server in Ireland, but similarly doesn't work for C's Firefox installation on her desktop. How wierd!

One Really Stupid Feature

Usability: So, I've just found out about a useless feature of my microwave oven the hard way. The microwave's manual notes:

Demo Mode

The Demo Mode is ideal for learning how to use the microwave oven. When set, functions can be entered without actually turning on the magnetron. The microwave oven light will come on, the fan will run and, if on, the turntable will rotate.

To Turn On/Off: The microwave oven and Timer must be off. Touch and hold TIMER SET.OFF for 5 seconds until 2 tones sound and 'd' appears on the display. Repeat to turn off and remove 'd' from the display.

Great! How useful! What a friendly UI! A tiny 'd' that indicates that, although your microwave looks like it's cooking away, it's actually doing nothing.

If you ever want to prank a friend with a Whirlpool oven, I'd recommend this one ;)

Global Guerrillas

Politics: Global Guerrillas: 'Networked organizations, infrastructure disruption, and the emerging marketplace of violence. An open notebook on the epochal war of the 21st Century.'

This stuff is scary -- and I think it's right; the author thinks Bin Laden sees the US as an over-stretched 1980's Russia, vulnerable to economic collapse; his last communique talks about what he calls the 'bleed-until-bankruptcy plan'.

Bruce Sterling says 'You ever read my 80s novel ISLANDS IN THE NET? The one with a sinister, imaginary book of military strategy in it called THE LAWRENCE DOCTRINE AND POSTINDUSTRIAL INSURGENCY? Well, this blog feels a lot like I imagined that book would have felt. Kinda floods the mind with creepily subversive 21st century insight.'

Definitely worth reading.

‘Stubberfield’ falls victim to first felony anti-spam conviction

Spam: 2 found guilty in first felony spam conviction: 'LEESBURG, Va. - A brother and sister who sent unsolicited junk e-mail to millions of America Online customers were convicted Wednesday in the nation's first felony prosecution of distributors of spam.'

Jeremy D. Jaynes, 30, (aka. Gaven Stubberfield) and Jessica DeGroot, 28, convicted to nine years in prison and a $7,500 fine respectively.

Nine years -- wow, that's a serious conviction for spamming... Virginia clearly takes this very seriously, as the home of AOL. Let's see if this causes any of the remaining spammers to think twice.

Four More Wars

Politics: Disaster. I can't believe it.

NoSoftwarePatents.com, and Michel Rocard appointed to draw up EP swpat response

Patents: I haven't been blogging much, as I've got the damn flu. I thought I was clear after a minor bout, then it came back around again for another run... urgh. Now I'm all hot, confused, achy, and (due to it affecting my ears and therefore sense of balance) clumsy. Damn you, influenza virus! :(

But there's some good news -- NoSoftwarePatents.com has opened, with a nice, clean, clear website full of excellent content. Not only does it cover the usual list of basics, with good examples like FFII's web-shop example, it goes into more detail about parallels; here's a great one:

Patents on software are just as wrong as expanding the patent system to literature. With patents on story elements, no movie could be published without having to firstly check whether there is any general idea in the storyline that someone patented during the last 20 years. Here's an example: At first sight, Dirty Dancing and Titanic are two very distinct movies. However, if there were patents on story elements, then the makers of Dirty Dancing could have sued the studio of Titanic. Both movies have a scene in which a poor boy takes a rich girl from a party of her social peers to a dancing party of his group, and she enjoys it. Dirty Dancing came out only nine years before Titanic, so any patent would still have been in force. No one knows whether James Cameron had that Dirty Dancing scene in mind as he wrote the Titanic script. Maybe Cameron never saw Dirty Dancing but the patent (if it existed) could be used against him anyway.

That's exactly the right parallel to make! Software patents cover the tens to hundreds of little algorithms you need to put together to make a software product, and comparing an algorithm in a software product to a scene concept in a movie, illustrates that nicely.

Each page has a little quote from heavy-hitters like Bill Gates, Oracle, Deutsche Bank, PriceWaterhouseCoopers, etc., coming out against swpats. There's also a section dealing with the disinformation that the other side has been putting out.

Next time you need to send a URL over to educate someone about this issue, this is the one to reach for.

Some news on the EU Software Patents Directive -- IDA eGovernment News reports that 'the formal approval of the draft Directive on the patentability of computer-implemented inventions by the EU Competitiveness Council has been postponed due to translation delays', and notes that 'on 6/10/2004, the Parliament's Internal Market Committee selected former French Prime Minister Michel Rocard to draw up its response.' It goes on:

Unsurprisingly, the appointment of Mr Rocard -- an outspoken adversary of software patents -- was welcomed by opponents of the proposed directive and criticised by its supporters. That Michel Rocard is taking over the dossier reflects the fact that the wider economic, infrastructural and social implications for Europe are now seen more clearly. Also, in the Council a learning process has begun, and it will be supported by the Parliament's move, Mr Holger Blasum of the Foundation for a Free Information Infrastructure told journalists. On the oher hand, Francisco Mingorance of the Business Software Alliance pointed out that Mr Rocard hasn't shown any sympathy to the directive in the past. We can only hope he will be sufficiently open to the view of persons and groups which have a different opinion, he said.

Well, that sounds like good news to me ;)

Why implanted ID chips are bad for privacy

Security: The RFID vendors are clearly on a roll, with all manner of uses being proposed. The most recent story is that VeriChip plans to implant them subdermally in hospital patients.

The company line is that it's privacy-safe, since it doesn't expose health records per se -- just the patient's ID number. However, that's missing the point, in my opinion.

RFID chips will broadcast their ID whenever they are within range of a compatible scanner, and the range (in this case) is several feet -- although the story notes that their readers used to track farmed salmon work from 10-12 feet, and the Schmoo Group guys I met last month had no doubts that a high-powered directional antenna like their wi-fi sniper rifle could extend that. There's no encryption, or handshaking, in these chips, it sounds like.

There's no mention if the chip is removed after you leave hospital; some comments about the idea behind this is that it may help if you're involved in an accident, and want your info available to healthcare users, in which case you'd have the chip implanted and broadcasting at other times, in other places, as well.

So, if you've got one of these implanted, it'll broadcast a unique code to readers in range at all times. If an attacker can scan while you're nearby, and picks up that code, they know that it's you, and you only. They only have to match that ID code to a visual identification once, and henceforth you can be tracked by that ID code.

There's a possibility that they'll fix this, by upping the CPU power and incorporating some decent public-key encryption -- but then you need a PKI big enough to track every implanted citizen in the entire country, and the costs will go up and up. I'd find that doubtful. (Mind you, they seem to assume that having a centralized secure database of medical records is a fait accompli in most of the articles anyway, so...)

Cambodians Eager to Dine on Rats (fwd)

Funny: AFP: Cambodians Eager to Dine on Rats:

'At first I just cooked them for my family to eat, but guests who tried them said they were tasty, so I started selling a few fried rats to the villagers,' he said. Business boomed so he devoted his menu to them.

' We only eat the small rats -- we dare not eat the big ones because they have too much hair.'

Big in Laos, too -- although I don't think I've heard of sit-down restaurants selling them. When I was travelling in Laos, one of the first tips I heard from other travellers was, 'if you see something that looks like a fried rat -- it is'. urrgh.

(BTW, there's actually good reasons not to eat rat-meat; wild rats and mice are truly filthy animals, vectors of all sorts of nasty diseases.)

Selves and Others now publishing RSS feeds

News: Selves and Others is a site that cropped up a couple of months ago, tracking the output of many of the left's strongest voices, for example:

Well, one feature they were missing was RSS feeds, allowing users to track new articles by a specific author as they're published. They've just added it; the good old orange XML button now appears on each author's page. Excellent!

Playing US games on a European PlayStation 2

Games: when I moved from Ireland to the US, I brought along my PS2; I hadn't had it that long, and I wasn't going to leave it behind (despite many offers to give it a good home ;).

Of course, Sony include plenty of trade-restrictive features in the PS2; European games won't play on a US PS2, and vice versa. So until now, I've been playing the few games I brought along from Europe, with the help of a YPbPr VGA converter, allowing the PS2 to display on a VGA monitor, and a transformer to transform 110V US current to 220V.

But that was before the superb japanese craziness of Katamari Damacy came along, and with GTA: San Andreas due out next month, something had to be done.

So -- after a little shopping, I found the solution -- rather than get into serious stuff like soldering, I got this -- the Slide Card. It's a 1.5-inch long piece of plastic, with a carefully placed notch. It requires one piece of PS2 modification -- you first of all have to remove the front of the CD panel. This just requires popping out one screw and a couple of clips, painless. You can then leave it off -- it's purely cosmetic -- or stick it back on if you really want to, at a future date.

Then, when you want to play an import game, the protocol goes like this:

  • put in the Slide Card boot DVD, power on the PS2
  • wait for the Swap Magic splash screen
  • insert the little plastic Slide Card, and slowly drag it left-to-right until it hits a piece of plastic internally
  • use it to pull out the CD tray, place the import DVD into the tray, and push it back in
  • use the Slide Card again, grabbing a little internal peg part with the notch in the card, and dragging the card right-to-left to load the CD into place.
  • hit 'X' on the PS2 controller, and the game boots!

So, this is a nifty solution; it basically works around the disc-replacement logic in the PS2, without any soldering or hackery required. And I've successfully used it after a night at the bar on several occasions, so that's the true test of how twiddly it is ;)

Unfortunately, by now I've probably spent nearly as much on hardware to play US PS2 games with a European PS2, as I would have if I'd just bought a US PS2. But hey...

Indymedia cross-border takedown reaches Slashdot

Web: The slashdot story. The comments contain a massive amount of noise, but there are some highlights...

Some details of the backend; it appears Indymedia need more mirrors, and the imc-tech list and #tech channel are the best contact locations to get in touch. The comment also notes that the Mir CMS used by most IMCs generates static HTML -- which is a good thing! I hereby withdraw my kvetching about server-side dynamic scripting in that case ;)

The techie who 'had the contract with Rackspace' comments, and provides a link to his weblog, which contains copies of the trouble tickets.

He also notes that the possible illegal posting was a newswire submission -- therefore not 'published' per se, just uploaded in the same way an unmoderated-up slashdot comment is.

And finally -- he notes that the EFF are offering to represent himself and Indymedia pro bono. Yay EFF!

The Electronic Frontier Foundation (EFF) is currently assisting Indymedia investigate possible responses to the seizure of its information. More than 20 Indymedia-related websites, along with Indymedia's online radio, were hosted on the servers, which were dedicated machines provided by Rackspace.

'This seizure has grave implications for free speech and privacy. The Constitution does not permit the government unilaterally to cut off the speech of an independent media outlet, especially without providing a reason or even allowing Indymedia the information necessary to contest the seizure,' said EFF Staff Attorney Kurt Opsahl.

This is great news. Top-secret takedowns are not a good thing, especially when they span three national borders...

More on the Indymedia shutdown

Law: t r u t h o u t quotes this press release from Rackspace:

In the present matter regarding Indymedia, Rackspace Managed Hosting, a U.S. based company with offices in London, is acting in compliance with a court order pursuant to a Mutual Legal Assistance Treaty (MLAT), which establishes procedures for countries to assist each other in investigations such as international terrorism, kidnapping and money laundering. Rackspace responded to a Commissioner's subpoena, duly issued under Title 28, United States Code, Section 1782 in an investigation that did not arise in the United States. Rackspace is acting as a good corporate citizen and is cooperating with international law enforcement authorities. The court prohibits Rackspace from commenting further on this matter.

(my emphasis.) I wonder which of those 3 Indymedia is supposed to have been infringing? It's pretty clear how Rackspace feel about this situation, I think.

It seems MLATs have been used before to shut down Indymedia sites in the US; this cryptome mirror of Montreal IMC pages documents one such case. Here's a summary from a quoted email there:

Heres a quite interesting story on the power of mlats and what we will have to look forward to with the COE treaty :

A cop car was broken into in Quebec and a security doc relating to measures for the Free Trade Area of the Americas summit protests was stolen and posted in the net in Seattle. At the behest of the RCMP, a magistrate judge issued an order to grab the records from a Seattle web site called the 'independent media center' using the US/CAN mlat. They were then visited by the FBI/Secret Service. They then had a gag order on this for several days before it was released today.

Great precedent. I wonder if when my car gets broken into again, I can use the cybercrime treaty to find my stereo again...

And snippets from the IMC press release of the time:

On the evening of Saturday, April 21, a day which saw tens of thousands demonstrate against the FTAA in the streets of Quebec City, the Independent Media Center in Seattle was served with a sealed court order by two FBI agents and an agent of the US Secret Service. The terms of the sealed order prevented IMC volunteers from publicizing its contents; volunteers immediately began discussions with legal counsel to amend the order. This morning, April 27, Magistrate Judge Monica Benton issued an amended order, freeing us to discuss the situation without the threat of being held in contempt.

The original order, also issued by Judge Benton, directed the IMC to supply the FBI with 'all user connection logs' for April 20 and 21st from a web server occupying an IP address which the Secret Service believed belonged to the IMC. The order stated that this was part of an 'ongoing criminal investigation' into acts that could constitute violations of Canadian law, specifically theft and mischief. IMC legal counsel David Sobel, of the Electronic Privacy Information Center, comments: 'As the U.S. Supreme Court has recognized, the First Amendment protects the right to communicate anonymously with the press and for political purposes. An order compelling the disclosure of information identifying an indiscriminately large number of users of a website devoted to political discourse raises very serious constitutional issues. To provide the same protection to the press and anonymous sources in the Internet world as with more traditional media, the Government must be severely limited in its ability to demand their Internet identity--their 'Internet Protocol addresses.' A federal statute already requires that such efforts against the press be approved by the Attorney General, and only where essential and after alternatives have been exhausted. There is no suggestion that these standards were met here.

The sealed court order also directed the IMC not to disclose 'the existence of this Application or Order, or the existence of this investigation, unless or until ordered by this court.' Such a prior restraint on a media organization goes to the heart of the First Amendment. Ironically, the Seattle Post-Intelligencer learned about the existence of the order from 'federal sources,' suggesting that the purpose of the gag order was simply to allow the government to spin the issue its way.

The order did not specify what acts were being investigated, and the Secret Service agent acknowledged that the IMC itself was not suspected of criminal activity. No violation of US law was alleged.

Of course, cryptome is still chugging away as it always has been; simple HTML and no server-side dynamic scripting, means easy offshore mirroring ;)

How to turn a stale project site into a useful Wiki

Web: Almost every project and organisation has, at some stage, bemoaned having stale data on their website, and wished there was a better way to keep it up to date; or wished their FAQ was more complete; or wished they had the time to HTML-ize all their know-how and get it up there.

Well, here's what we did in SpamAssassin to deal with this problem. (Seeing as I've talked about this three times in the past month, I'll write it up here so I can just point at the URL next time!)

First off, we experimented with having the site checked into CVS, FAQ-o-matic, and the Python FAQ software (which was pretty good). All were OK, but very specific in format, using the traditional question-answer FAQ layout -- that's good for FAQs, but not so good for a lot of other stuff -- and keeping it updated was still limited to a small group, therefore the info got stale again.

So we moved to a Wiki. Here's my tips for Wiki-izing your website so that the end results are better than what went in.

Use good wiki software: unusable software will be a pain to use, and the info will still go stale. We used Moin Moin - http://moin.sourceforge.net/ - partly because I like Python (it's nearly perl! ;), it can produce RSS, and it was pretty easy to install.

Don't worry: people won't vandalise it (much). It turns out that vandalism and people throwing up crappy info isn't a serious problem at all. You should increase the barrier, in the following ways:

Require user accounts: set the security policy so that a user account must be set up before editing is possible. This means you won't get wiki-spammed, and also has the side effect of imposing a pretty big barrier to casual vandals.

Send changes to a list: set all changes to be mailed to a mailing list as diffs. This is the most important tip. If you already have a mailing list with the knowledgeable part of the community on it, use that list -- because they're the ones who'll be able to recognise if erroneous info is put up, and will be annoyed about this enough to bother fixing it. There's a bonus side-effect of this; even if some people didn't like the wiki to start with, they'll eventually be needled into using it by wanting to fix stuff they perceive as wrong. And then they get sucked in ;)

Use diff for the mailed changes: Moin by default will only send out change messages saying 'something changed on this page!'. That's not good enough, unfortunately -- you want to mail out what the new text looks like, and highlight exactly where the change happened. Moin can do this nicely, with this patch, which adds a mail_commits_address, where all diffs on every page are sent, using the normal diff mechanism.

Ensure the wiki software can revert quickly: If someone does make a bad change, Moin supports one-click reversion of the page to what it was beforehand. That's great for dealing with spam, or clueless vandalism.

Keep one or two static pages: If you're worried about some script kiddie thinking that defacing a wiki makes them look cool, then keep one or two of the primary user-facing pages as static data. For example, take a look at the link-bar at the top of http://spamassassin.apache.org/ ; five of the ten links are to static pages, the other five are now wiki-ized. In particular, our front page and our downloads page are both static, but our docs are predominantly Wiki'd.

Publicize Mozex: most techie groups will have techie users, and we hate using browser text-boxes to edit text. Mozex -- http://mozex.mozdev.org/ -- saves the day here -- it's a godsend.

Shepherd new changes: in the early stages, you want one or two people who tidy up changes from Wiki newbies, as they go in. They need to keep it looking pretty, and perform Refactoring of stuff that could be laid out better or should become multiple pages. Eventually, others will get the hang of that (and do a much better job than you do ;).

That's the lot. Most of these are to, essentially, migrate aspects of your already-existing and already-working community into this new outlet. In our experience, it's worked really well -- our Wiki is now the most reliable source of info about SpamAssassin, and is extensive and up-to-date.

Indymedia server drives seized

Politics: Indymedia's hard drives in Rackspace UK seized by FBI order, seemingly as a 'courtesy' to Swiss police. There's several morals to be learned:

  • Rackspace UK are happy to roll over for the US feds;
  • it appears the action was taken using powers granted under the USA-Patriot Act;
  • hosting in Europe is not safe from bad US laws.

However, the UK site is back on the air, and reportedly they're recovering nicely; 'All this goes to prove that Indymedia is decentralised enough (but not perfectly) to survive an attack and that as a cooperative international network, we rock!'

more on H5N1 Bird Flu

Health: A few hours after ( ;) I link-blogged this New Scientist article about a case of the H5N1 avian flu transmitting itself between humans, Boing Boing put up this entry titled 'Bird Flu risk extremely low', which concludes that the risk is effectively not worth worrying about.

It's fundamentally wrong, and is well worth pointing out as a result. As Quinn at ambiguous.org says, it's not the danger now that's important here -- it's the potential.

I read New Scientist religiously, so I've been following it, and this search on H5N1 gives the perfect illustration of why this is well worth worrying about:

(Now, while it's worth worrying about, it's not us end-users who should be doing the worrying. It's the politicians who need to ensure CDC and the WHO are funded well, the terrible state of vaccine development and production be sorted out, the lack of outbreak monitoring infrastructure be addressed, and research into these strains is funded and given a priority, in case things do go all pear-shaped influenza-wise.)

New fronts for patenting

Patents: Sun files for patents on per-employee software pricing plans (/.). 'Method for licensing software to an entity, including determining a per-employee cost for the software, determining a number of employees of the entity, and determining a total licensing cost using the number of employees and the per-employee cost, wherein the total licensing cost comprises a software license for all employees of the entity and all customers of the entity.'

But, in my opinion, here's the good news -- this is a patent on a license agreement. In other words, this is a new front for patents -- the field of law.

Once the lawyers start running into situations where trivial concepts in their license agreements are patented, you can be sure the situation will start to turn around. ;)

Firefox 1.0PR’s software installation UI

Security: Given the current prevalence of phishing attacks and spyware infestations, designing a good user interface that protects naive users against malware is now more urgent than ever.

Firefox is, of course, widely touted as more secure than MSIE. This is by and large true, due partly to MS' emphasis in their UIs on one-step 'easy' installation and confirmation-dialog reduction (in my opinion) -- but also due to the fact that spyware companies don't yet see Firefox as a target to the same extent.

This changed recently -- spyware 'toolbars' started to appear for Firefox as well. It was quite a surprise to see a dialog pop up when accessing an otherwise normal-looking (though advertising-heavy) page, using my Linux desktop, prompting me to install some 'toolbar' .xpi file!

Firefox 1.0PR now includes code to deal with this. Here's how it works.

If a site I'm viewing attempts to install an XPI file, I get this prompt:

Note that it's NOT a dialog. This is pretty handy, because it means that I won't get annoying dialogs all the time if I do accidentally go to a unscrupulous site; it just appears like the part of the page. In the clueless user case, they may not even notice that they've been protected, which reduces the risk that they'll install the extension anyway.

(However, I would have extended it by using an icon or look-and-feel that indicated that this was a 'trustworthy' part of the UI, rather than possibly part of the page.)

If I hit the 'Edit Options...' button, I get this:

A simple-enough dialog containing the list of sites permitted to install extensions. update.mozilla.org is in there by default, and I've added texturizer.net so I can install from their more extensive list of older extensions. The address of the current site has been dropped in automatically.

To permit the site, I have to hit 'Allow', then 'OK'. So I do that, and hit the 'install' link on the webpage again:

And there's the Software Installation dialog. Note the red Unsigned warning, the proportion of text that is a warning about installing bad stuff (fully half!), and -- this is interesting -- a greyed-out 'Install' button.

The button is on a timer -- it becomes clickable after 2 seconds. This, presumably, is to ensure that people read the dialog! Reportedly, users no longer read dialogs, instead hitting OK on every dialog that appears. In my opinion, this is arguably due to 'the boy who cried wolf' syndrome: by default, MSIE and older Mozilla versions will ask all sorts of stupid questions about 'are you sure you want to send stuff on the intarweb?' whenever you use Google. If anything is guaranteed to induce dialog fatigue, it's that feature.

(Update: actually, that's not the reason. Reportedly, it's a workaround for a couple of social-engineering attacks, whereby an attacker could persuade the user to type a word ending in 'Y', and time the dialog to appear just before 'Y' is typed -- causing the keyboard shortcut for 'Yes' to take effect; or persuade the user to double-click in the right spot, and similarly time the dialog to appear in the right place, in time for the second click. Still, I maintain the measure is useful to deal with the 'dialog fatigue' issue too. ;) Thanks to Smyler and Rod for pointing this out.)

I would have gone further:

  • the 'a software install was blocked' page element should have an indication that it's 'trustworthy content'
  • both dialogs should default to 'Cancel', to avoid users deliberately pressing 'OK'
  • I would possibly require a 'yes, I read this' tickbox to be ticked before the software is installed.

Interesting though. This is the way internet-facing UIs are going to have to develop, in my opinion.

Slides from Toorcon 2004

Spam: my slides from the presentation I gave at Toorcon 2004, 'Spam Forensics: Reverse-Engineering Spammer Tactics', are now up. Hope they prove enlightening ;)

Back from Toorcon

Travel: Toorcon was great fun! Lots of interesting conversations.

Unfortunately they had a cruddy internet connection, so I'm majorly backlogged, and can't write about any of it just yet ;)

Bush at the UN

Politics: So I was listening to that this morning. Did I hear correctly? Did Bush really say that one of the good side-effects of Iraq's invasion, was that there were now hopefully less attacks inside other countries? sure looks like it:

'Coalition forces now serving in Iraq are confronting the terrorists and foreign fighters so peaceful nations around the world will never have to face them within our own borders.'

I'm sure the Iraqi civilians will love that. 'Hey guys, sorry about all the missing limbs, but you're doing a really good job of being flypaper so we don't get hurt. Cheers! Have a 15% corporate tax rate!'

ToorCon

Conferences: Hey -- I'm talking at ToorCon 2004 down in San Diego this weekend! Come along and check it out, if you can.

I'd better hurry up and file my presentation slides pronto ;) The topic is:

Spam Forensics: Reverse-Engineering Spammer Tactics

In this talk, I'll discuss how the SpamAssassin project has identified reliable signatures indicating that a message is spam, by reverse-engineering spammer tactics from the spam mails themselves. I'll also discuss several specific features that we have identified, how we found them, and why the spammers add them.

Sitescooper is WorldChanging!

Green: Wow -- UC Berkeley's Lab Notes newsletter this month includes an article noting the benefits to the environment of reading your news on a PDA instead of getting a delivered newspaper. Check this out:

In a new study, UC Berkeley researchers report that receiving your news wirelessly on a PDA instead of delivered to your door requires up to 140 times less carbon dioxide, several orders of magnitude less greenhouse gases, and the consumption of 26 to 67 times less water.

...

To tease out the truth, Horvath and graduate student Michael Toffel dissected nearly all of the environmentally-relevant processes involved in both wireless news delivery and teleconferencing. In the case of newspapers, the researchers focused on the environmental effects of reading the New York Times in Berkeley, California, from the manufacture of newsprint and ink to the delivery from a nearby printing press to disposal of the newspaper. This data was then compared to such factors as the energy used to manufacture a PDA, including its microprocessor and battery, and the electricity required by wireless and Internet service providers to deliver news content to the device.

Sitescooper is therefore a WorldChanging tool!

The EU software patents battle returns

Patents: Now that the summer break is over, software patents are back on the EU's agenda. The FFII (via EDRI-gram) reports

On 24 September 2004, the European Council will probably meet to rubber-stamp the 'political agreement' achieved on 18 May 2004 on the highly controversial software patents directive (2002/0047 COM-COD).

According to the FFII the text was designed to mislead ministers about its real effects. 'It consists of many sentences of the form or 'software is ... unpatentable, unless ... [condition, which, upon closer scrutiny, turns to be always true]'.' And, states FFII, 'It can be said with certainty that only a minority of governments really agrees with what was negotiated, but several governments were misrepresented by their negotiators, who broke intra-ministerial agreements or even violated instructions from their superiors.'

More info:

Lara Doody Smith

Life: Luke writes:

Lean and I were joined by Lara at 6.10pm on Saturday 28th September. Lara is a little (8lb 7oz, so not _that_ little) girl. And she is gorgeous. Of course.

Congrats! I'll be dropping in on the three of them next week, looking forward to it...

Planetary Backgrounds now using Coral

Web: My Nearly-Live Planetary Desktop Backgrounds site is now using NYU's Coral Content Distribution Network instead of FreeCache.org. (FreeCache wasn't caching the files, because they were too small. drat.)

Coral is a 'decentralized, self-organizing, peer-to-peer web-content distribution network', using a distributed sloppy hash table and peer-to-peer DNS redirection infrastructure.

At least, apparently. ;) I haven't read the papers yet, but what I do know is that so far, it seems to be working perfectly -- each file is requested exactly once by the CDN servers:

  193.10.133.129 - - [31/Aug/2004:16:50:31 +0100] "GET
  /xplanet/tmp/200408311455.399750/day_clouds_800x600.png
  HTTP/1.1" 200 706936 "-" "CoralWebPrx/0.1 (See
  http://www.scs.cs.nyu.edu/coral/)"

and never requested again. That's a big saving... nifty!

Linux and small hardware vendors

Linux: Everyone who's used a non-MS system will have learned -- typically the hard way -- that not all hardware is equal. Not just in terms of specs, flexibility and power, but also in terms of whether or not it can be used at all.

Most hardware vendors consider their specification and interface documentation to be their crown jewels; giving access to these without a signed NDA is impossible. On the other hand, for free software developers, signing an NDA makes life quite difficult -- it can be done, but nobody else can help you maintain it further without signing an NDA, the resulting code may 'disclose' too much of the 'IP', and so on. In a lot of cases, the vendor isn't interested in giving access to the specs, even with an NDA -- it's their IP and why isn't the customer just using Windows?

The end result: lots of hardware with crappy support on non-MS operating systems.

Things aren't as bad as they used to be, though -- since nowadays the high-end hardware is more likely to support standards, and Linux is a top choice on embedded hardware (set-top boxes for example), so it has a much higher profile. But cheap, end-user oriented PCs still wind up with components from vendors who couldn't be bothered with non-Windows customers, and that can mean using a hacked-up, reverse-engineered driver and hoping it works. (That's not to denigrate reverse-engineered drivers. some of them work great. But fundamentally, the vendors are making a mistake here.)

So it's pretty impressive to see that LaCie are now sponsoring development of k3b, the CD/DVD burning application for KDE!

Good timing too, I was about to buy a DVD burner ;)

kgst output to ALSA/Artsd/ESD instead of just OSS

Linux: Here's a patch that adds support for ALSA/Artsd/ESD output from kgst, the KDE gstreamer middleware used by JuK.

Background: JuK is a great music player app for KDE. However, it hogs
the sound device while running, which means that nothing else gets access to play sound until the app is shut down. This is suboptimal.

The reason it does this is because it plays sound via this chain of components: juk -> kgst -> gstreamer -> sink. Unfortunately, the
kgst component doesn't allow control over what output sink to use, instead hard-coding the string 'osssink' -- the OSS drivers, for traditional Linux /dev/dsp sound. My laptop doesn't support mixing in the sound hardware, which means I need to use a software mixer. 'osssink' doesn't support software mixing, instead giving the caller exclusive access to the sound card, and other apps will just have to wait for it to finish.

(As to why JuK doesn't just play mp3s by running 'mpg321 name-of-file.mp3', and let us specify the '-o' switch to use, I wish I knew. (ObOldbieGripe: component-based architectures are full of this kind of needless over-complexity ;)

Anyway, the patch in the bug above lets the user provide an environment variable for a string for kgst to use instead of 'osssink'.

Life Hacks: the magic of flat files

Tech: This is the second entry talking about 'Life Hacks'. Possibly the best tip I came away from the talk with, is this one:

All geeks have a todo.txt file. They use texteditors (Word, BBEdit, Emacs, Notepad) not Outlook or whathaveyou.

What we keep in our todo is the stuff we want to forget. Geeks say they remember details well, but they forget their spouses' birthdays and the dry-cleaning. Because it's not interesting.

It's the 10-second rule: if you can't file something in 10 seconds, you won't do it. Todo.txt involves cut-and-paste, the simplest interface we can imagine.

It's also the simplest way to find intercomation. EMACS, Moz and Panther have incremental search: when you type a "t" it goes to the first mention of "t", add "to" and you jump to the first instance of "to", etc.

Power-users don't trust complicated apps. Every time power-geeks has had a crash, s/he moves away from it. You can't trust software unless you've written it -- and then you're just more forgiving. Text files are portable (except for CRLF issues) between mac and win and *nix. Geeks will try the Brain, etc, but they want to stay in text.

I was already doing this, having learned the latter lesson ;), but I was making one mistake -- I was trying to keep the TODO.txt file small by clearing out old stuff, done stuff, and cut-and-paste snippets of command lines, and by moving things into files in 'storage' directories.

That doesn't work. You think you'll be able to grep for it later, but you'll have forgotten what to grep for. You'll even have forgotten what storage directory you used. The solution is to keep it all in one big file, and use i-search. That really does work.

In fairness, I actually have two files of this type. One is the "real" TODO.txt. But the other is a GPG-encrypted file containing usernames, URLs, passwords, nameservers, VPN settings, etc. I have a feeling this is another common Life Hack idiom, too...

Another great tip in the same vein, from JWZ -- make an /etc/LOG:

Every machine I admin has a file called /etc/LOG where I keep a script of every system-level change I make (installing software, etc.) I rsync these LOG files around (keeping redundant copies of all of them in several places) so that if/when I need to re-build a server from scratch, it's just a matter of following the script.

This has been working out great (when I remember to do it. Discipline! ;)

Life Hacks: getting back to the command-line

Tech: So Danny O'Brien's 'Life Hacks' talk is one of the most worthwhile reflections on productivity (and productivity technology) I've heard. (Cory Doctorow's transcript from NotCon 2004, video from ETCon.)

There's a couple of things I wanted to write about it, so I'll do them in separate blog entries.

(First off, I'd love to see Ward Cunningham's 'cluster files by time' hack, it sounds very useful. But that's not what I wanted to write about ;)

People don't extract stuff from big complex apps using OLE and so on; it's brittle, and undocumented. Instead they write little command-line scriptlets. Sometimes they do little bits of 'open this URL in a new window' OLE-type stuff to use in a pipeline, but that's about it. And fundamentally, they pipe.

This ties into the post that reminded me to write about it -- Diego Doval's atomflow, which is essentially a small set of command-line apps for Atom storage. Diego notes:

Now, here's what's interesting. I have of course been using pipes for years. And yet the power and simplicity of this approach had simply not occurred to me at all. I have been so focused on end-user products for so long that my thoughts naturally move to complex uber-systems that do everything in an integrated way. But that is overkill in this case.

Exactly! He's not the only one to get that recently -- MS and Google are two very high-profile organisations that have picked up the insight; it's the Egypt way.

There's fundamentally a breakage point where shrink-wrapped GUI apps cannot do everything you want done, and you have to start developing code yourself -- and the best APIs for that, after 30 years, has been the command-line and pipe metaphor.

(Also, complex uber-apps are what people think is needed -- however, that's just a UI scheme that's prevailing at the moment. Bear in mind that anyone using the web today uses a command line every day. A command line will not necessarily confuse users.)

Tying back into the Life Hacks stuff -- one thing that hasn't yet been done properly as a command-line-and-pipe tool, though, is web-scraping. Right now, if you scrape, you've got to do either (a) lots of munging in a single big fat script of your own devising, if you're lucky using something like WWW::Mechanize (which is excellent!); (b) use a scraping app like sitescooper; or (c) get hacky with a shell script that runs wget and greps bits of output out in a really brittle way.

I've been considering a 'next-generation sitescooper' a little bit occasionally over the past year, and I think the best way to do it is to split its functionality up into individual scripts/perl modules:

  • one to download files, maintaining a cache, taking likely freshness into account, and dealing with crappy HTTP/HTTPS wierdness like cookies, logins and redirects;
  • one to diff HTML;
  • one to lobotomise (ie. simplify) HTML;
  • one to scrape out the 'good bits' using sitescooper-style regions

Tie those into HTML Tidy and XMLStarlet, and you have an excellent command-line scraping framework.

Still haven't got any time to do all that though. :(

Image Watermarking With ‘pamcomp’

Web: My Dad runs a couple of websites -- his architectural photography business, and Andalucia Photo Gallery, a side project selling some lovely photos from the Andalusia region of Spain.

Needless to say, as the family geek, guess who coded all that up? Using WebMake, naturally ;) This was the main reason I wrote the 'thumbnail_tag' plugin.

You'll note, however, that the image to right is watermarked, quite small, and encoded with a low quality setting. It turned out after a couple of years of operation, that the images were being downloaded and used in print all over the place -- from both sites!

It seems photo piracy is rampant. Even with terms of use clearly linked on the sites, it's still commonplace for print publications to swipe the images -- and not just the little guys, either -- some big commercial names have apparently used the images without asking (or paying licensing fees).

The Andalucia gallery site was a favourite; being a good hit for 'travel photos spain' meant lots of images being used for holiday pages in magazines, newspapers, and so on.

Needless to say, digital watermarking software doesn't work -- it's trivial to load an image into Photoshop, resize or crop, and resave, apparently. Even if PS did respect the watermarks, netpbm doesn't, and a watermarked image isn't identifiable as such once it appears in print anyway! So we went for the blunt-tool approach, adding visible watermarks to the images.

It's pretty easy -- pamcomp allows you to overlay one image on top of another, using a third as an 'alpha mask' to control transparency. The results are pretty nice and not too intrusive.

It's a shame it has to be done, though... :(

MS Patents sudo(8)

Patents: The varchars.com scraped RSS feeds now include new patent grants and applications by certain companies! Interesting, although given that most developers are advised not to look, not advisable ;)

However, I glanced at the MS one -- and immediately spotted this gem: US Patent 6,775,781, filed by Microsoft, is a patent on the concept of 'a process configured to run under an administrative privilege level' which, based on authorization information 'in a data store', may perform actions at administrative privilege on behalf of a 'user process'.

This, and the patent claims, perfectly describe the operation of sudo, fundamentally as it's operated since running on a 4.1BSD VAX-11/750 in 1980.

20 years head start on a patent application -- surely that must qualify as prior art ;)

RFID Security

Security: It looks like the security people are starting to take a look at RFID, and it's not pretty.

I link-blogged this the other day -- RFDump is a tool to display and modify data in RFID tags -- including deployed ones, at least in some cases. (Think rewriting the price tags in a shop, scrambling the tracking numbers on a warehouse full of goods, or corrupting frequent-shopper data on a card.)

It looks like this was also discussed at USENIX Security '04 in an RSA presentation (those notes are swarming with typos, but the content's there ;)

That talk has some interesting stuff -- 'blocker' tags which spoof readers with gibberish data, or crash the collision-detection network protocol; while that's being discussed as a security tool here, if the protocol is that hackable, and the hardware is available, I could see that having additional interesting effects in a supermarket. Of course, range is an issue -- but that hasn't stopped Bluetooth hacking, wardriving, etc.

If you ask me, it looks an awful lot like RFID is chock-full of security holes, and the features that make it so attractive (low power use, low cost, tiny size) will be the very features that militate against adding security. We could be in for interesting times here...

A ‘Boulder Pledge scoreboard’ website

Spam: Ask Slashdot: How Powerful is the Turn-Off Power of Spam? The question is, 'How often do you make the decision to NOT buy something form a company because you know they engage in spamming activities?'

This is an old idea -- it goes back to a December 1996 column by Roger Ebert, of all people, who proposes the following pledge that all internet users should take:

Under no circumstances will I ever purchase anything offered to me as the result of an unsolicited e-mail message. Nor will I forward chain letters, petitions, mass mailings, or virus warnings to large numbers of others. This is my contribution to the survival of the online community.

8 years later, it's more important than ever.

However, it's complicated by one additional factor -- not everyone knows which products and companies use spam to advertise. For example, did you know that Kraft routinely advertise their Gevalia coffee through spam?

My suggestion -- a daring individual (that rules me out ;) should set up a website where samples of major-product-advertising spam are collected from (trusted) reporters. A quick scoreboard based on how many reports a particular company accumulates, and we have a Boulder Pledge reputation service.

Some simple rules should be applied:

  • Messages arriving at never-used spamtrap addresses, or scraped addresses from USENET or the web, especially if the message hits multiple of those addresses (indicating a high volume), is the basis for a listing;
  • Failure to respect opt-outs, of course, would be a biggie;
  • Using a known spamhaus, or sending via open proxies in Shandong, would be a massive thumbs-down;
  • Failure to clean up it's act after being made aware of the problem, oh dear.

It'd be essential to take an extremely careful approach to this; any hint of personal axe-grinding, and the site would be useless, written off as just the work of 'another anti-spam kook'.

Essentially, this'd be a Fortune-500-oriented version of spamvertized.org.

Reportedly, many of the large companies using spam to advertise are fully aware at a management level that they are responsible for spamming. (That line about open proxies in Shandong is no joke -- at least one Fortune 500 company has hired a spamhaus that does this.)

Doubtless, some spamvertisers may be victim to an overzealous but clueless marketing department, on the other hand -- but either way, a public 'name and shame' forum gives a great impetus for them to avoid this problem, at least once they've been bitten the first time.

In some cases, it's dodgy 'affiliates' that use spam to advertise their products -- but a company that operates affiliates really should post a policy that says that affiliates found to be spamming will be terminated and have their commissions forfeited; reportedly, that has been found in other programs to quickly cut off the problem.

Spamusement rocks!

Spam: oh man, Spamusement started off well, and has just been getting better and better; * HEATH WARNING * had me laughing out loud, and the idea of linking the entries since August 8 as a series is genius.

Announcing IPC::DirQueue

Perl: So, I wrote a new CPAN module recently -- IPC::DirQueue. It implements a nifty design pattern for slightly larger systems, ones where multiple processes, possibly on multiple machines, must collaborate to deal with incoming task submissions. To quote the POD:

This module implements a FIFO queueing infrastructure, using a directory as the communications and storage media. No daemon process is required to manage the queue; all communication takes place via the filesystem.

A common UNIX system design pattern is to use a tool like lpr as a task queueing system; for example, this article describes the use of lpr as an MP3 jukebox.

However, lpr isn't as efficient as it could be. When used in this way, you have to restart each task processor for every new task. If you have a lot of startup overhead, this can be very inefficient. With IPC::DirQueue, a processing server can run persistently and cache data needed across multiple tasks efficiently; it will not be restarted unless you restart it.

Multiple enqueueing and dequeueing processes on multiple hosts (NFS-safe locking is used) can run simultaneously, and safely, on the same queue.

Since multiple dequeuers can run simultaneously, this provides a good way to process a variable level of incoming tasks using a pre-defined number of worker processes.

If you need more CPU power working on a queue, you can simply start another dequeuer to help out. If you need less, kill off a few dequeuers.

If you need to take down the server to perform some maintainance or upgrades, just kill the dequeuer processes, perform the work, and start up new ones. Since there's no 'socket' or similar point of failure aside from the directory itself, the queue will just quietly fill with waiting jobs until the new dequeuer is ready.

Arbitrary 'name = value' metadata pairs can be transferred alongside data files. In fact, in some cases, you may find it easier to send unused and empty data files, and just use the 'metadata' fields to transfer the details of what will be worked on.

Sound interesting? Here's the tarball.

CEAS Roundup

Spam: So, CEAS was great fun, and very educational:

  • Got to meet up with various antispammers, including Daniel and Theo from the SpamAssassin dev team, Jeff Chan from SURBL, Dan Kohn from Habeas, Catherine Hampton from The SpamBouncer, Miles Libbey, John Levine, Neil Schwartzman -- lots of good chats.
  • MS really know how to feed a conference! I hear rumours there was an extra-special tinned-meat-product-based dish at the banquet...
  • But their firewalling tendencies put a serious damper on keeping in touch with the outside world, at least until we set up an SSH tunnel on port 443 ;)
  • During a lull, Dan Kohn fired off a hands-up census -- a good 75% of the attendees (roughly) admitted to using SpamAssassin!

My highlight papers:

  • IBM's Chung-Kwei pattern-discovery system -- the one which Mark dug up. Very interesting stuff; it turns out that bioinformatics is full of large corpora of data (genomes) which you then need to find patterns in. Funnily enough, so is SpamAssassin: s/genomes/spam/, s/patterns/regular expressions/. The more advanced pattern-discovery algorithms even allow complex patterns to contain alternative blocks, 'don't-cares' and similar regular-expression-like features.

    The really good bit of Chung-Kwei is the Teiresias algorithm (more pages, online demo). Of course, being IBM research, it's probably patented to the hilt, and may be tricky to license; but it's certainly pointed us in a whole new interesting direction -- anyone know any bioinformaticians?

    IBM is really gearing up on anti-spam research. 4 of the 6 papers listed on their website were presented this year, at CEAS.

  • Another good paper was On Attacking Statistical Spam Filters, by Gregory L. Wittel and S. Felix Wu, which (similarly to Henry Stern's submission, which I helped a little with) dealt with an attack on Bayesian filters.

    This is interesting stuff; we're pretty sure it's not as serious as it could possibly be, in SpamAssassin's implementation, but it's still a serious attack.

  • The Impact of Feature Selection on Signature-Driven Spam Detection was an interesting paper on AOL's new signature schemes. (The conference was sponsored by Cloudmark, BTW, but those guys were nowhere to be seen -- in which case they missed this presentation ;)
  • Reputation Network Analysis for Email Filtering was interesting, in that it mirrors to a degree the thinking behind web-o-trust.org, but in my opinion suffered due to a lack of thought about avoiding spoofing (by including IP address information in the FOAF file, it could do this now). However, once SPF becomes pervasive, this could be combined with that to generate personalised webs of trust usable for email whitelisting.
  • Resisting SPAM Delivery by TCP Damping was very nifty; plug a classifier into your MTA, and thereby detect connections from spam relays. Once you've found them, you then throttle down their connection as they attempt to deliver spam. Some other TCP-level tricks can do nifty stuff like massively increasing the bandwidth consumption of the spamming machines. Very very nice!

I took copious notes on the SpamAssassin wiki, if anyone's curious.

Patents in an open source world

Patents: Newsforge: Patents in an open source world, by Lawrence Rosen (founding partner of Rosenlaw and Einschlag).

Interesting article, but I'm not sure summary point number 2 ('continue to document our own "prior art" to prevent others from patenting things they weren't the first to invent') really helps, when the patent examiners clearly haven't performed the simplest Google check. I've found obvious prior art in 30 seconds, by plugging 3 words from patent claims into Google in the past (and yes, I have a reasonable idea how to read patent claims by now).

Point number 3 is interesting, since it contradicts most other advice I've read regarding patent searches: 'Conduct a reasonably diligent search for patents we might infringe. At least search the portfolios of our major competitors. (This, by the way, is also a great way to make sure we're aware of important technology advances by our competitors.) Maintain a commercially reasonable balance between doing nothing about patents and being obsessed with reviewing every one of them.'

However, this comment really is interesting and raises something major that I'd never heard of before -- users of proprietary software can also face a significant risk from the patent threat. In particular, according to the linked comment, Microsoft licensed some patented technology from a company called Timeline Inc., but the license was not sublicenseable -- in other words, it did not grant their customers the rights to fully use the technology! (in fairness to MS, this was established later in court.) Result: href="http://trends.newsforge.com/comments.pl?sid=39443&cid=96153">MS SQL server OEMs and ISVs are now being sued.

Post-Apocalyptic Fiction

Reading: Both jim winstead and Nelson Minar have praised Earth Abides , a 1949 post-apocalyptic novel where 'all but a handful of people die from a mystery disease', and the ensuing narrative 'follows one man's attempt to rebuild something like a society.' It seems a tip from original happy mutant Mark Frauenfelder was the pointer for both of 'em.

I'm a huge fan of the genre; I think it's something about our age group, growing up in the shadow of Reagan's 'Evil Empire' speeches, Threads and (much less terrifying) The Day After.

Given that, it looks like Earth Abides goes straight into the wishlist. However, I should make another couple of reading tips while I'm at it, in the same genre:

First off, Jack London's short story
The Scarlet Plague (1912) is a clear antecedent to Earth Abides. In this story, too, a plague hits the planet and wipes out most of civilization; an old man talks to children who've known nothing but the post-apocalypse period. It's pretty short and well worth a read.

But my main recommendation is Kim Stanley Robinson's The Wild Shore (1984), first book of his Three Calfornias trilogy, and his debut novel.

It takes place in 2047, 60 years after a massive nuclear attack on the US, by Russian infiltrators (pretty dated, eh ;). The narrator is a teenager in a primitive agrarian community on the coast of southern Orange County. His group are farmers, living far away from the previously built-up areas; the people who live amongst those ruins are shunned, and the different tribes meet only occasionally to trade. Disposable butane lighters are a treasured commodity.

He gradually discovers that the US was once a superpower, and that they are now being kept in a virtually stone-age state by outside powers. The interesting factor here is that most sci-fi authors, at this point, would embark on a jingoistic, militaristic armed struggle; it initially seems that's what's happening, but Robinson takes a very interesting tack, in his own style, and this really makes the book something special.

(I won't go too far into it, but if you really want to know and don't mind spoilers, this site thoroughly spills the beans.)

Counterfeit Cops

IP: A funny IPR-enforcement-related story from New Scientist (sorry, subscriber-only link):

Just before delegates (to the 28 May 'Global Congress on Combating Counterfeiting') left Brussels to ponder their future anti-counterfeiting measures, a salutary tale started doing the rounds.

The WCO (World Customs Organization) produces a CD database of the codes needed to identify goods by type so that local customs authorities can collect the appropriate duties. The discs sell for EUR 1000 apiece, but WCO investigators have found that staff at some border posts, which are supposedly the front line in counterfeit detection, are not using the official CDs. Instead - you've guessed it - they are buying cut-price pirated copies, complete with crudely photocopied, plainly fake covers and sleeve notes.

Physician, heal thyself!

Kentucky sez ‘Opt-Out Still Doesn’t Work’

Spam: Some fantastic data in this paper from the Kentucky Long-Term Policy Research Center.

It's a brief 2-pager detailing the effectiveness of the CAN-SPAM Act in reducing the spam load, using a set of test addresses. The methodology is pretty good.

One point in particular is very important: 'opting out' from spam Just Does Not Work. This graph tells the whole story:

After opting out from spams received, the amount of spam received at those 'opted out' test addresses actually rose. (This even after CAN-SPAM made such activity explicitly illegal.)

Some other data:

  • obfuscating addresses on web pages is still working; 7.7 times the spam is received if you don't bother doing so.
  • e-mail harvesting also continues after CAN-SPAM made it illegal.

If anyone needed proof, this shows that spammers are quite happy to break the law; strong enforcement 'teeth' are needed for any anti-spam legislation. (UK, take note: the thoroughly useless system whereby spam complaints must be submitted on paper isn't going to help!)

The Technical Details document also notes something interesting: one test address was set up to test 'opting out' of legitimate mass mail from some (unnamed) big websites, and continued to receive ads 'sometimes months after opting-out'. For shame!

(thx to John Levine for forwarding the links.)

Spam: Michael Radwin on open HTTP redirectors, and in particular noting that Yahoo! have (finally) closed their main one down. One down, several hundred to go ;)

Good history of the exploitation techniques that spammers have been using, too.

BEST SONG EVER — identified!

Funny: Some of the taint.org readership (that's you, Nishad) may be familiar with BEST SONG EVER.mp3 -- it's an insane, 10-minute workout: one guy ranting at a high pitch in some east-asian language at an incredible speed over some cheesy Casio, hardly taking a breath, punctuated by bizarre 7-Zark-7-style ribbits and squawks. By the end of it, he's nearly hoarse. It is incredibly bizarre. Turkopop has nothing on this.

Well, it's origin has been discovered -- he's called E Pak Sa, and the style is called 'Pansori'. His version is a modern take on this ancient traditional style -- 'While singing, he would imitate the sound of all of the instruments used in the prelude and interlude, and even the sound of the whistle used to gather the tourists.' From there, he grew in popularity, especially in Japan:

'Sell-out concerts, myriad television appearances, riots at in-stores, and Japanese teens speaking Korean are all products of E Pak Sa's impact in Japan. E had infiltrated the popular culture of Japan and paved the way for other Korean artist to do the same.'

And guess what -- his Encyclopedia of Pon-Chak album can be listened to online! The YMCA cover -- track 2 -- is strongly recommended.

Ross Anderson not quite so cool anymore

Security: Ross Anderson, crypto and security guru extraordinaire, moonlights as -- wait for it -- a street bagpipe player:

I play the pipes (the Great Highland Bagpipe and the Scottish smallpipes). I played competitively as a teenager, and thereafter paid my way through university by working as a street musician in Germany, France, the Netherlands and Denmark.

NOOOOOO! ANYTHING BUT THE BAGPIPES!!!

Only joking. But yes, he really does play the bagpipes. And that submission to the EU's consultation on the management of copyright and related rights is worth a read, to get an idea of how the new increased enforcement of music copyright has had chilling effects on the viability of the UK's folk music scene. (found via Karl-Friedrich Lenz.)

Kiera Knightley

Funny: Kiera Knightley's photoshop boobjob has been all over the place recently -- it's a pretty extensive reworking. But then, that's standard practice nowadays...

However, best comment goes to stephendann:

In photo 2, she has the quad damage. The skin colour darkens, the chest expands, the stomach contracts and the character skin is obviously altered so the rest of the players know she's supercharged. In POTC:King Arthur, it's a more subtle damage modified than (the) UT2K4 glowing purple bow.

LOL!

Great SSH tip, and how to fix a KDE glitch

Unix: via Ted Leung, Adam Rosi-Kessel's Linux Tips page has some very useful tips, and this one's great -- to avoid
getting SSH connection resets, add the following to your .ssh/config:

    serveraliveinterval 300
    serveralivecountmax 10 

This will insure (jm: sic) that ssh will occasional send an ACK type request every 300 seconds so that the connection doesn't die.

As a similar tip that took a while to track down -- KDE users who've upgraded between KDE releases, will probably by now have seen lots of messages like this:

  nameofapp (KIconLoader): WARNING: Icon directory /usr/share/icons/hicolor/
  group 48x48/stock/text not valid.

It took a bit of googling about to find the cure:

  • run in a shell (I cannot find this on any menu): kdebugdialog --fullmode
  • select: debug area: 264 kdecore (KIconLoader)
  • Change the Warning Output to 'None'
  • select: OK

DVD pirate’s pitch ends in arrest

Funny: BBC: DVD pirate's pitch ends in arrest:

A man has been arrested after trying to sell counterfeit DVDs - at a Trading Standards Office.

The man had apparently missed the sign on the office in Beehive Lane, Chelmsford, Essex, and asked if anyone would like to buy pirated films. Staff said they were very interested indeed in what he had to sell, but when he realised where he was he ran off, leaving his wares and £210 in cash.

Police later arrested the man in a supermarket in Chelmsford.

Hacking Netflix

Movies: Hacking Netflix, via torrez.

Jason Kottke points out a great quote on a Friendster cross-site scripting attack -- this great quote: 'We have a policy that we are not being hacked.'

He also speculates that Google used the GMail invite-network data for whitelisting -- but whitelisting based on email address alone is trivially exploitable, so I'd doubt it.

I'm just back from a trip over to Cape Cod to meet family (halfway between here and Ireland, y'see ;) -- lots and lots of luvverly lobster and sundry shellfish -- and after a 6 day trip, had 5000 spams and a couple of thousand nonspam mails to deal with. Thankfully SpamAssassin dealt with the spams (only about 5 false negatives, no false positives I could spot) -- but I'm going to have to do something about that volume of mail. drowning in the stuff. argh.

Microsoft 0wnz ‘http’

Web: Back in 2002, it occurred to someone to check the Google search results for 'http', to figure out what the most popular sites were.

Looks like it's changed -- here's the top five results from a Google search for 'http' now:

  • 1: Microsoft
  • 2: AltaVista (!!)
  • 3: Yahoo!
  • 4: My Excite
  • 5: Google

My guess: older links are getting good PageRank, using whatever new tweaked algorithm they're using. But AltaVista beating Google? ;)

RTE’s Bush Interview

TV: RTE's 'Prime Time' secured a fantastic interview with GWB, with Carole Coleman asking a few very pointed questions. Watch it with RealPlayer, or listen to the audio in MP3 (2.7Mb).

There's a pretty accurate transcript here:

Let me finish! How many times do I have to tell you how to do your job? See, I gotta insult France at least once. Then I gotta claim 'merica to be the most generous nation in the whole wide world, even though it's not true. And listen, let me mention that democracy in Pakistan, too. And guess what? I'm the first president to ever call for a Palestinian state and I'm damn proud of it - just look at the size of my smirk now. Listen, as long as I keep repeating myself and mouthing empty platitudes, you won't have a chance to call me on any of the bullshit coming out of my mouth.

OK, the official one is here.

It appears that the White House just dropped the ball on this one; reportedly, they had her list of questions three days in advance, but given that they suggested that she 'ask him a question on the outfit that Taoiseach Bertie Ahern wore to the G8 summit' (!!!), they weren't paying attention, and expected some kind of giggling moronic schoolgirl, or something.

Hilariously, the White House has since complained to RTE, the Irish Embassy, the Irish Government, and the reporter herself. Probably God, too. I doubt Prime Time will ever get a White House interview again, but given what they clearly expect from the poodles in the White House press corps, that's hardly much of a loss.

(I'd love to see what'd happen if he had to deal with Paxman ;)

Also, went to see Fahrenheit 9/11. Fantastic movie, and best of all, incredibly well-attended.

My favourite moment: the reminder of just how easily the US news media sold itself out during the war. Seeing Katie Couric blurting 'Navy Seals rock!!' like some kind of starstruck 5-year-old with an Action Man toy, was a classic. It's good to see that this will be immortalized in celluloid, as it was truly shocking at the time. (Not much has changed; Judith Miller is still writing for the NYT.)

Samuel L. Jackson’s ‘Irish’ comment

Ireland: Here's a hot UL that's floating around the irish web right now --

In a British program about Samuel L Jackson and Colin Farrell's lastest movie SWAT presented by British presenter, Kate Thornton, the following exchange occured:
  • Thornton: What was it like working with Colin (Farrell), cos he
    • is just so hot in the U.K. right now?
  • Jackson: He's pretty hot in the U.S. too.
  • Thornton: Yeah, but he is one of our own.
  • Jackson: Isn't he from Ireland?
  • Thornton: Yeah, but we can claim him cos Ireland is beside us.
  • Jackson: You see that's your problem right there. You British keep claiming people that don't belong to you. We had that problem here in America too, it was called slavery.

... yeah, right. ;)

(Update: Actually, believe it or not, that's more or less how it really went. Here's the transcript.)

Some commentary at
TheReggaeBoyz.com (quote: 'I NEARLY DEAD TO RASS!!!!') and Kuro5hin.

It looks like the TV programme does exist; no scripts online, unfortunately, so we'll never figure out if this one really happened, I think.

IMO, it's made up for sure. That last line is just a little too harsh for a primetime schmooze-a-gram, at the very least. Plus, it's the kind of thing only an Irishman would give a shit about -- the perpetual adoption of Irish celebs and worthies by the UK media is a continual source of irritation for the Irish -- as Dervala puts it:

'No, Oscar Wilde was ours. You put him in jail, though. And Shaw was ours. And Yeats. And Johnny Rotten.'

Announcing a new script

Web: Minor software announcement -- after some time using HTMLThumbnail, album, and even WebMake to build photo galleries, I finally got peeved enough, and gave in to the temptation of 'not invented here'. ;)

Presenting Uffizi, a CSS- and template-driven, themable perl script to generate photo galleries. Quoting the POD:

  • it's very self-contained, apart from dependencies on Image::Size and the ImageMagick convert command
  • fast, efficient incremental rebuilding
  • generates full CSS-styled, templated and valid HTML
  • every part of the generated HTML can be modified through the templates
  • generates reasonably-sized images as well as thumbnails, with a link to the full-sized image
  • secure -- all pages are static HTML, so your webserver won't get r00ted through a silly photo album script

I am, of course, using it on my own photo pages, and I'm very happy with it; it's been a while since I had to hack it. (I need to get it to thumbnail MPEGs as well, but apart from that it's teh nifty IMO.)

SpamAssassin now an Apache TLP!

Spam: SpamAssassin is now officially an Apache top-level project! InternetNews.com coverage:

The Apache Software Foundation is taking the spam fight to a new level -- literally -- with the promotion of its Spam Assassin project to top-level status.

Hooray ;)

The ‘humans are 99.84% accurate’ figure

Spam: 'The spam-classifying accuracy of a human being is 99.84%'. This statement has passed into SlashDot lore as the gospel truth, so time for some debunking.

First off, that's not what Bill Yerazunis said in the CRM-114 Sparse Binary Polynomial Hashing and the CRM114 Discriminator paper. Here's the real quote:

the human author's measured accuracy as an antispam filter is only 99.84% on the first pass

Here's a copy of the original mail:

I manually classified the same set of 1900 messages twice, and found three errors in my own classifications, hence I have a 99.84% success rate.

(my emphasis). In other words, the author sat down and ran through 1900 messages manually, then ran through them again, and checked to see how many messages in the first batch disagreed with the second.

Let's consider an alternative situation, where a user is presented with one message, and asked to take their time, give it a full examination and some thought, and then classify the message. I would consider that more likely to be classified correctly, since fatigue will not be an issue (after 1900 messages, I'm pretty tired of eyeballing), and neither will time pressure (taking 20 seconds on each of 1900 mails would require 10.5 hours, and would be excruciatingly boring to boot).

In addition, the study wasn't clear on exactly how much information from each mail was presented. Too little (just the subject line) or too much (every header and raw HTML), and a human will be more likely to make mistakes than if the mail is rendered fully, and the extraneous header info hidden. In my experience, I've never hand-classified 1900 messages purely through either method, because it's just too tiring, and I know I'll make quite a few mistakes. The UI for this work is important.

And finally, the figure is derived from a study with one user performing a task once. There's no way you could use that figure in a serious setting -- it's not valid statistical science. Here's Henry's comment:

Yerazunis' study of "human classification performance" is fundamentally flawed. He did a "user study" where he sat down and re-classified a few thousand of his personal e-mails and wrote down how many mistakes he made. He repeats this experiment once and calls his results "conclusive." There are several reasons why this is not a sound methodology:
  • a) He has only one test subject (himself). You cannot infer much about the population from a sample size of 1.
  • b) He has already seen the messages before. We have very good associative memory. You will also notice that he makes fewer mistakes on the second run which indicates that a human's classification accuracy (on the same messages) increases with experience. For this very reason, it is of the utmost importance to test classification performance on unseen data. After all, the problem tends towards "duplicate detection" when you've seen the data before hand.
  • c) He evaluates his own performance. When someone's own ego is on the line, you would expect that it would be very difficult to remain objective.

So, to correct the statement:

'The spam-classifying accuracy of this one guy, when classifying nearly two thousand mails by hand, was 99.84%, once.'

Cormack and Lynam’s study on supervised spam detection

Spam: or, 'SlashDot spam drama'. So, a few days ago, I forwarded a link to a paper I'd been sent -- it's a great paper, and I'm not just saying that because SpamAssassin did well -- it really tests some of the popular open-source spam filters comprehensively, and correctly. (The authors have 24 years of information retrieval research between them.)

The results have been pretty incendiary. ;) Here's a timeline with links, in case you were wondering where we are right now:

A UNIX shell tip

UNIX: I've just made the first change to my core bash configuration in years, to add -b to the set command-line. It triggered some thinking about when the last one was.

It turns out, that apart from writing scripts and aliases frequently, I haven't changed my commandline UI in any respect, since about 2 years ago. By contrast, I've been hacking about with GUI settings continually, new desktop backgrounds, themes, colours, etc. Odd!

Anyway, here's the tip -- it's very handy, I find.

I changed to using a 2-line prompt, with the first line containing the time and the full working directory, in a 'magic' cut-and-pasteable format:

        : exit=0 Thu Jun 24 17:55:29 PDT 2004; cd /home/jm/DL
        : jm 1203...; 

Note that the prompt starts with ":", which means that bash/sh will ignore the line until it hits ";". The end result is that the entire line evaluates to "cd /home/jm/DL" when pasted. Hey presto, cd'ing several terminals to the same dir just involves triple-clicking in one, and middle-button-pasting into the others. nifty! Similarly, the second line has a little bit of prompt, but that snippet will be ignored when cut and pasted.

Having the exit status of the last command (bash var: $?) is useful too. The code:

  do_prompt () {
    echo ": exit=$? `date`; cd $PWD"
  }
  PROMPT_COMMAND='do_prompt $?'   # executed before every prompt
  do_prompt 0                     # set up first prompt
  PS1=": `whoami` \!"
  PS2="... >>; "            # continuation prompt
  PS1="$PS1...; "

The Web-App generation

Software: Mark Twomey, in response to all the Win32 API stuff recently:

We now have a generation of computer users ... who have never received or sent email from a so called 'rich client', never had to send a postal order off to order something from some distant vendor, and are not amazed by something like a search engine. ....

Those ('rich client') people remind me of minicomputer users who crapped on the 'crummy little operating systems' used on 'crummy little desktop computers.'

He's right, you know -- for de yoot, Windows is generally just a way to access Hotmail.

Ahmed Chalabi and Iran’s encryption

Security: some crypto drama.

Ahmad Chalabi apparently told the Iranian government that the NSA had broken their secret code, according to 'US intelligence officials': NYTimes: Chalabi Reportedly Told Iran That U.S. Had Code. This story is still running -- Bruce Schneier has just posted his expert opinion, as has Ross Anderson. As I noted on Eric Rescorla's weblog, here's my (non-expert) theory ;)

It's known that the Iranians used Crypto AG equipment up until about 1992, and it's been widely reported that Crypto AG's systems were backdoored by the NSA and traffic routinely decrypted. (also, Baltimore Sun story, 1995)

Reportedly, the Anglo-Irish discussions of the 1985 were a rather one-sided affair, because the Irish government used Crypto AG machines to communicate between their Embassy in London and Dublin, and intercepts of their reports were fed back to the UK government.

In addition, according to this article (backup), the NSA also provided Iraq with intercepts of Iranian secret traffic, while Iraq was a US ally -- which could explain why Chalabi would have known about it.

It also speculates as to how it was done:

'Knowledgeable sources indicate that the Crypto AG enciphering process, developed in cooperation with the NSA and the German company Siemans, involved secretly embedding the decryption key in the cipher text. Those who knew where to look could monitor the encrypted communication, then extract the decryption key that was also part of the transmission, and recover the plain text message. Decryption of a message by a knowledgeable third party was not any more difficult than it was for the intended receiver. (More than one method was used. Sometimes the algorithm was simply deficient, with built-in exploitable weaknesses.)'

So my opinion is that Chalabi's claim was very old news from the 80's and early 90's -- which pretty much fits in with the rest of his tip-offs to everyone else ;)

“Vice-President Hunter Thompson”

Politics: Kerry in Colorado:

"Just to put your minds all at ease, I have four words for you that I know will relieve you greatly," Kerry told the fund-raiser. "How does this sound? Vice President Hunter Thompson."

Travel: Great posting on culture shock and 'going native' at Yankee Fog.

Hacks: Dan Kaminsky's LayerOne presentation hits Slashdot. Definitely one of the highlights of that conference.

Spam: confession for two: a spammer spills it all. Interesting -- especially since the spammer winds up earning less than he would have working for Starbucks.

It's also worth noting this posting from Gary Smith on the sa-users list, in which Gary filled out a spam form with some not-entirely-valid info -- with hilarious results!

So I did talk to some of these lenders. Apparently they buy leads from www.lendergateway.com . One guy that I talked to was irritated because it costs him $100 per lead they sell him and it's supposed to only be sold to him. He apologized quite a bit and was nice enough to give me the information on who sold him the names. The number he game me goes to voicemail which I'm going to try later. A couple other people told me what I can do with myself and one lady kept saying that she couldn't give me information on who provided her with my information.

The stupid thing is each time I talk to them I tell them I'm on a cell and that I need their name and number and I'll call them right back. They give it to me... So when they hang up I start calling again and again. I've been irritating the hell out of them...

Anyways, that's the fun storing of what happens when these forms are filled out.

$100 per spurious 'lead' would make a serious dent, if enough spurious leads showed up... ;)