Skip to content

Justin's Linklog Posts

Interview with two phish-scene infiltrators

Published January 28, 2008

/. posted a link to this interview with Nitesh Dhanjani and Billy Rios, two guys who have infiltrated the "phishing underground".

It’s a good article — lots of detail on the current toolset of a typical phisher, and some details on the community itself:

I had always thought that most phishers were clever hackers evading authorities using the latest evasion techniques and tools. The reality of the matter is most of the phishers we tracked were sloppy and unsophisticated. The tools they used were rarely created by the phisher deploying the actual scam, and for the most part it seemed the phisher merely downloaded kits and tools from some place and reused over and over and over again. It also seemed that many phishers don’t even really understand how the phishing kits they’ve deployed work! We also came across many phishing kits and tools that had simple backdoors written into the source code (essentially, phishers phishing phishers). These backdoors are easily spotted by anyone who has even a basic idea of how the source code flow worked, yet was undetected by many phishers. Maybe a few phishers out there are skilled, but the majority are clueless.

Here’s something I’ve noted about spammers, too — there’s no honour among thieves:

The number of backdoors we saw was staggering. The servers serving the phishing sites had backdoors, the code used in the phishing kits had backdoors, the tools used by phishers had backdoors. Phishers aren’t afraid to steal from regulars people and they are also not afraid to steal from other phishers. Some of the backdoors were meant to keep control over a compromised server, while other simply stole information that had been stolen by other phishers! We came across several forums where phishers, scammers, and carders basically identified other phishers, scammers, and carders that had scammed them. These shady characters may work with each other but they sure don’t trust each other, that’s for sure.

And this is a very important point about blacklists:

Phishers are likely to abuse the blacklists published for [anti-phishing] plugins for their own benefit. The blacklists are a list of known phishing sites that the plugins consume in order to identify what websites are fraudulent. These blacklists therefore contain IP addresses and host names of servers hosting phishing sites. Since phishing sites are commonly installed on servers that have been compromised, and phishers don’t bother to patch systems they have installed their kits on, this list translates to a ‘list of easily compromisable hosts’ for other phishers.

On the latter point, this is one of the key benefits of DNS blocklists, compared to the downloaded, text-based style that Google initially used for its anti-phishing toolbar. To query a DNSBL, you need to know the address you’re looking for first of all; but with a text file, you can read the lists in their entirety, without knowing the address in advance. (Google is now apparently tending to use the enchash format, which fixes this.)

And a final word:

For the next few years, we are going to continue to apply band-aids around the problem of data leakage, and continue to play whack-a-mole with the phishers without solving the actual problem at hand. In order to make any significant progress, we must come up with a brand new system that does away with depending on static identifiers. We will know weâ??ve accomplished this when we will be able to publish our credit reports publicly without fearing for our identities.

(I’d place more importance on the liability of the financial institutions, myself — I think they get away with placing too much blame on the victims of fraud and identity theft.)

Good interview — worth reading.

Insane Dell.ie markup

Published January 21, 2008

A good deal came up on a mailing list I’m on: SAMSUNG 245BW Black High Glossy 24" 5ms DVI Widescreen LCD Monitor for $459.99, or $409.99 after rebate, via Newegg.

A follow-up from a German poster: he’d just picked up a Dell 2407WFP-HC ‘for the low, low price of 659 EUR’.

We marvelled at the price difference — then I looked up Dell.ie forcomparison. I thought 659 EUR was bad, but Dell.ie is asking for 1,117.74 Euros inc VAT for the same product — insane!!

What possible excuse could there be for that? EUR 458.74 worth of shipping maybe? Do they encase it in platinum? That’s nearly three times the price of the Newegg monitor.

Update: Duh. I’m an idiot. That’s a 2707WFP, not a 2407WFP; it’s 3" bigger and quite a bit fancier. It appears Dell.ie is no longer selling the 2407WFP.

Bad law in North Dakota

Published January 17, 2008

This is very bad news for North Dakota-based anti-spammers — a guy called David Ritz is being sued there by alleged porn spammer Jerry Reynolds, for performing DNS lookups, a DNS zone transfer and a Whois lookup. It appears the judge has found Ritz guilty.

This is astonishingly bad lawmaking by the judge. These are entirely innocuous tools, part of every network administrator’s toolkit for debugging and examining internet traffic legitimately. There’s nothing remotely criminal or malicious in their use, and the judge has allowed himself to be misled.

North Dakota Judge Gets it Wrong:

‘Ritz’s behavior in conducting a zone transfer was unauthorized within the meaning of the North Dakota Computer Crime Law. A zone transfer is simply asking a DNS server for all the particular public info it provides about a given domain. This is a common task performed by system administrators for many purposes. The judge is saying that DNS zone transfers are now illegal in North Dakota.’

More details from Ed Falk

David’s legal defense fund

My Commodore 64 demos

Published January 10, 2008

I recently came across my record at the Commodore Scene Database, and was happy to find that someone had found and uploaded two demos I had written, back in my days as a member of the C=64 demo scene between 1988 and 1990:

(I was a member of the groups ‘Excess’ and ‘Thundertronix’ / ‘TNT’, going by the handle of ‘Mantis’.)

With the help of CBA, I was overjoyed to track down another long-lost demo, my crowning achievement on the platform:

If you’re curious, feel free to go read those wiki pages or download the .d64’s — they run fine in VICE, the Commodore emulator (amazingly). If you’ve only got time to check one, check Rhaphanadosis; it’s much better than the others.

I’m very impressed with VICE. As far as I can tell, it’s perfectly bug-for-bug compatible with the real hardware, playing all of the demos perfectly (apart from a little additional speed due to differing hardware performance). If you haven’t already got VICE set up, bear in mind that after installing it, you’ll need a copy of the C=64’s ROM images; here’s a local set.

Also, the Commodore Scene Database is pretty awesome — it’s a full-scale IMDB-style setup, tracking the history of the Commodore demo scene in massive detail. Nice work guys!

The demos were written 100% in 6502/6510 assembly. I developed them using an Action Replay cartridge’s built-in monitor; it had an assembler, but one which didn’t support symbolic addressing. In other words, every piece of assembly used hand-computed branch offsets, and every variable and subroutine was tracked — on paper — by memory location, rather than using symbolic labels. If you want to know what the monitor was like, the VICE built-in monitor is almost identical!

I wrote these when I was 16; part 4 of Rhaphandosis notes the date as being 20 May 1989.

It’s interesting reading the scrollers, and doing web and CSDB searches in follow-up to see what happened next — one of the other Excess members, Raistlin is now Robert Troughton, a successful game developer in the UK with several major titles under his belt.

A Google search for Thundertronix finds a copy of "sex’n’crime" zine, issue 17, July 1990, which notes:

one of the new groups formed in 1990 (jm: slightly off, I think) is THUNDERTRONIX, better known as TNT. they are based in ireland and are doing very well for themselves. they have, in my mind, one of the best coders in the uk, namely MANTIS. he is currently coding a game with many new routines, etc… hopefully he should get some demos out soon!

woo! Er, unfortunately that game never went anywhere. ah well. ;)

BTW, it’s funny reading my scrollers in those demos. At the time, I was convinced that the c=64 was a dead platform — yet here we are in 2008, and there’s still a thriving demo scene on the Commodore. Incredible!

Vincent Browne on RTE’s coke habit

Published January 4, 2008

Before Christmas, it seemed you could hardly read a newspaper, listen to the radio or watch TV in Ireland without being bombarded with stories about how the country was awash in cocaine.

It’s an attractive story, tying in nicely with the death of lingerie model Katy French, hand-wringing over Ireland’s recent ‘celtic tiger’ wealth, a supposed loss of our traditions, etc. etc. RTE, our national broadcaster, made a tabloid series called ‘High Society’, which cashed in on the issue in a particularly crass way — crappy "reconstructions" of actors chopping lines with voiceovers, dodgy-looking men handing over money to ominous music, that kind of thing.

Well, just before Christmas, <a href="http://www.ireland.com/newspaper/opinion/2007/1219/1197996964671.html"> Vincent Browne wrote a fantastic op-ed in the Irish Times regarding this. I have to quote this particularly perceptive passage:

Cocaine abuse is a social problem, but the thrust of much of RTE’s coverage of the phenomenon is to suggest that it is a widespread, pervasive problem. There are no recent statistics available on the prevalence of cocaine consumption in Ireland – the last survey was done four years ago. The National Advisory Committee on Drugs (NACD) will be publishing a prevalence report next month and we will know then the size of the phenomenon.

But we have some indicators about the scale of cocaine use. The European drug agency EMCDDA estimates that 3 per cent of all adults in Europe aged between 15 and 64 have used cocaine at least once in their lives.

A third of these took cocaine during the previous year and half of these took cocaine during the previous month. This means that about 0.5 per cent of the adult population took cocaine over the previous month. And the data suggests that, for at least two-thirds of those who have ever taken cocaine, the drug is not a problem for them.

In the US the statistics are higher. Almost 15 per cent of the population aged between 12 and 64 have taken cocaine in their lives and 2.5 per cent took cocaine over the previous year. Again, this is suggestive that cocaine use for most people is not a problem, otherwise the number of people who took cocaine during the previous year as a proportion of the number of people who ever took cocaine would be far higher.

The figures for Ireland are likely to be that about 4 per cent of the adult population have taken cocaine in their lifetime, with about 1 per cent having taken cocaine in the previous year and 0.5 per cent having taken cocaine in the previous month.

It would be better if people did not take cocaine, but the prevalent contention that the consumption of cocaine at all is necessarily harmful and addictive is obviously false.

It would also be better if people did not drink here, for the problems related to the consumption of alcohol are far, far greater than in the case of cocaine.

Instead of presenting a balanced picture of the cocaine phenomenon, RTE has greatly exaggerated the issue, in a way more typically associated with tabloid journalism.

Well said!

Spambots stealing GMail and Hotmail passwords?

Published December 21, 2007

I just received this mail from a friend:

Dear friend

Welcome to stwoxy.com ! We are one of the largest electronic distributors and wholesalers in Beijing China. We offer qualified digital products: Motorcycles?TVs, Notebooks, phones. PSP, projectors, GPS, DVD, DV, DC, MP3/4 and so on, which are of world famous brands, such as Sony, IBM, PHILIPS, NOKIA, DELL and so on. All our items are brand new from the manufactures and they come with 1-3 years’ after service. These days we are expanding our overseas market, and every item is sold in extremely low price. Such chances should never be missed, ladies and gentlemen, do come to stwoxy.com! you will surely have a big surprise! We are looking forward to hearing from you!

It was sent from a HTTP connection into GMail, and was delivered from there using valid DKIM, Domain Keys and SPF signatures. In addition, it was sent to all the addresses in his address book. In other words, this was no run-of-the-mill impersonation spam — for this one, the spammer obtained my friend’s username and password somehow, logged into GMail, scraped the address book, and then sent spam via GMail that way.

My friend says he didn’t access GMail using a desktop mail client, but did have his Google password saved in his web browser (a pretty typical configuration). My theory is that some virus/malware has infected his desktop machine, captured the saved-passwords file from the web browser configuration, and used that to log into GMail. Alternatively, it could also be a guessable username and password which was picked up via dictionary attack, I guess…

This is the first case I’ve heard of where spammers are actively stealing user account authentication tokens, in order to take over the accounts for spamming. (We’d long predicted it, of course, since it’s a natural response to "pay for mail" schemes… but since there’s no widely-used pay-for-mail system available yet, it’s premature!)

It seems this is not just a GMail thing, btw. Here’s a report of the same thing happening to some French guy via HotMail last month (or in english). I don’t speak Dutch, but this forum post looks like it might be the same situation.

If you’re curious, here’s a copy of the spam, delivered to a Yahoo! group; it appears these spammers aren’t too sophisticated in terms of the text they’re sending, since they haven’t morphed that text, HTML, or even the domain in the link yet. It’s just the malware that’s sophisticated, at this stage.

GNOME, Google and the UNIX user interface

Published December 20, 2007

Recently, after a flurry of annoying user interface issues, I’ve switched my RSS reader from Liferea to Google Reader. Interestingly, it turns out that Google Reader actually fits better with the traditional UNIX user interface concept, I’ve found.

What triggered this was an upgrade from Liferea 1.0.x to 1.4.4 as part of Ubuntu Gutsy; this brought with it a lot of changed behaviours, such as ‘drag-and-drop of feed URL to HTML view no longer subscribes’, and one crucial UI issue, ‘"Skim through articles" only works with ctrl+space’.

I’ve been a long-time UNIX user, dating back to the days where curses-based interfaces were the norm. As such, I tend to drive commonly-used applications using keyboard commands where possible. (This isn’t a purely UNIX thing; Windows has the phenomenon of the keyboard-wielding "power user", too.)

Liferea was attractive, since it offered the ability to skim through articles quickly by just pressing the "Space" key; simply press space to page down, or to skip to the next unread article if at the end of the current one. Unfortunately, Liferea 1.4.x breaks this, and it wasn’t going to be fixed, since apparently a GNOME app shouldn’t behave this way:

GTK explicitely does implement as a key binding for several of it’s widgets. Rebinding means to break the default behaviour for such widgets (tree views, buttons, input fields). [….] Liferea as a web-browsing application should behave like any other web browser and like every other GNOME/GTK application as much as possible.

Now, I don’t know if it’s GNOME’s fault, or what, but for a UNIX desktop app to break with UNIX UI conventions, that’s a bad move in my opinion. I gave it a bit of argument in the bug tracker, but eventually gave up as I clearly wasn’t getting anywhere. :(

Instead, based on recommendation from friends, I gave Google Reader a try, and quickly figured out its extensive collection of keyboard shortcuts. Now, I’m skimming through my feeds in even less time than it took with Liferea, simply by hitting "ga" to go to my "all unread items" list, then "j", "j", "j" to skip through the postings one by one. Sweet!

It’s interesting to note that other Google web apps use the same concepts; Gmail also has a hefty set, and can be driven using them in a manner very reminiscent of the classic UNIX mailreader, Mutt. So, despite being designed with end-users in mind by extremely clever professional user experience designers, these apps still find space for power-user keyboard operation. Take note, GNOME.

Anyway, I’m not too bothered. Google Reader brings other benefits, such as fixing this bug: ‘please add ability to go to previous entry in Unread feed’, avoiding ‘constant memory leak requires daily restarts’, and, of course, the utility of being able to track the same set of feeds and keep track of which items I’ve read in two places (work and home).

If only it was open source ;)

Planet Antispam update

Published December 7, 2007

A brief update on Planet Antispam…

I’ve just added MailChannels’ Anti-Spam Blog. Now — in the interests of disclosure — I’m a member of MailChannels’ Technical Advisory Board. However, that didn’t affect this — their blog has had consistently good, interesting posts dealing with anti-spam-related topics, and without too much plugging of their own products. ;)

Also added recently:

If you know of any other good email anti-spam-related blogs, drop a line in the comments here. (Note that I’m trying to keep it email-related, however, so we’re not covering web-spam.)

Spammers “giving up” according to Google

Published November 30, 2007

According to this Wired story, Google reckons spammers are giving up on spam:

a remarkable trend is underfoot, according to Brad Taylor, a staff software engineer at Google: The number of spam attempts — that is, the number of junk messages sent out by spammers — is flat, and may even be declining for the first time in years.

Actually, this is a wilful misunderstanding of what the Googler in question really said, which was that ‘attempts to spam Gmail users have been leveling off over the last year and more recently, even declining slightly’. In other words, they didn’t make an observation about the state of the spam problem on an internet-wide basis — just about the "local" situation as it pertains to Gmail. Bad reporting there, Wired.

But, in passing…

David Berlind at ZDNet recently blogged a rather grumpy response to InfoWorld coverage of CEAS 2007. He raised a very important point:

If I could say something to the author of that story, it would be that so long as any anti-spam solution is not deployed universally throughout the Internet’s e-mail system (in other words, so long as some anti-spam tech is not a standard), that anti-spam solution actually makes the spam problem worse. You read that right. Worse. Proprietary anti-spam solutions make the global spam problem worse. They are digging us deeper into the hole that the Internet is already in because everyone who makes those solutions is under the false belief that "s/he who is finally successful at filtering out all spam while allowing the legitimate mail in wins."

Google’s blog post is a case in point: ‘we’re keeping more spam out of your inbox than ever before, so more and more, you can use Gmail for things you enjoy without even realizing that the spam filter is there most of the time.’

That’s great — but it doesn’t help anyone except Gmail. It’s a myopic view of the spam problem, and David’s point stands.

(I disagree with his later conclusion that the only way forward is for Google, MS, AOL and Yahoo! to get together and ‘commit to jointly supporting the same technical solutions’ — when the usual BigCos get together, they tend to focus on their own priorities. Take what happened back in 2005 with nofollow for blog-spam — while it helped the search giants with their own overriding priority, which was to tweak their algorithms to filter out the spam on the search results page, it did nothing to slow the spam flood itself, which has continued unabated.)

We need more open-source, and open-data, anti-spam work.

Informed

Published November 30, 2007

This should be in the running for "least informative dialog ever".

(The information in question was that Firefox had been upgraded by the Ubuntu Gutsy Update Manager app, if you’re curious…)

Working around O2 Ireland

Published November 28, 2007

I’m pretty conservative with my mobile phones — until recently, my mobiles were all cheap, low-end, super-lightweight Nokias with long battery life and low "worry factor" (ie. not a big deal if they were lost or stolen). Very sensible.

I’ve finally started catching up with the gadgetorati, though — my current phone is now a Sony Ericsson K550i, which is still small and light, but has nice features like a 2 megapixel camera, a decent amount of onboard flash space, and a good implementation of Java, hence support for GMail and Google Maps. (Thanks to Joe for the recommendation!)

The only downside is that it came from my operator, O2 Ireland, with some broken configuration settings. (This shouldn’t be surprising, of course — I don’t think I’ve ever heard of a phone arriving with working data connectivity, from any operator, anywhere in the world.)

Anyway, here’s what I’ve done so far to fix it. Hopefully this might be helpful for random google searchers.

1. "Failed to resolve hostname" when publishing photos:

Generally, when I’d try to publish a photo using its Blogger support, I’d get a "failed to resolve hostname" error message. Investigating further, I found that the "O2 WAP" service used a proxy server — turning that off fixed the problem nicely. Nice reliable proxy you’ve got there, O2 ;)

Here’s how to do that. Open the menu, then select Settings -> Connectivity -> Internet settings -> Internet Profiles. Select O2 WAP and hit More -> Settings. Select Use proxy and change it to No, then hit Save. Problem solved.

2. Cannot send email from the device:

O2’s default mail server has a tendency to refuse to accept outbound mail from the phone. Switching to GMail for outbound SMTP works fine. Notice a trend here?

Open the menu, Messaging -> Email -> Settings -> New account. Set the Account name to "gmail". Scroll down to Email address, set it to "yourname@gmail.com". Connection type is "POP3", Username and Password are whatever your GMail account uses. Outgoing server is "smtp.gmail.com". Enter Advanced settings, and set Encryption to "TLS/SSL". Set Outgoing port to "25". Press the back button, then select the "gmail" account’s tickbox to make it active, before pressing back again to exit the configuration screen.

3. The "side" buttons go online:

By default, if you hit the "globe" button or the "open window" button on the side of the phone, to the left and right of the main joystick, it’s set to open various URLs at www.o2.ie. These buttons are prime UI real estate, and easily accidentally hit; I don’t want to go online (and possibly incur a charge) if they’re pressed.

Easily fixed. Open the menu, then select Settings -> Connectivity -> Internet settings -> Internet Profiles. Select O2 WAP and hit More -> Advanced, then Change homepage and enter "file:///" under Address and hit Save. It’ll now issue an ugly warning if you press those buttons, but at least it won’t go online. (It’d be nice to get a nicer fix for this.)

I’m sure there’s plenty more; if you’ve got this phone and have any tips to share, feel free to drop a comment below.

In particular, I’d love to know how to further "de-O2ify" the UI; the top 3 buttons on the menu screen are taken up with worthless operator spam ("O2 Music Store", "O2 Menu" and "Entertainment", all of which go to various URLs at www.o2.ie), while the useful Applications and Alarm screens, which I use all the time, are hidden in a submenu. ugh.

Investing in real estate

Published November 21, 2007

Screen real estate, that is — 3600×1050 pixels of it:

(That’s a Samsung SyncMaster 225bw226bw connected to a Thinkpad T61p running Ubuntu Gutsy, if you’re curious.)

‘Dead spammer’ story: yep, spam

Published November 15, 2007

Remember the ‘Russian ‘make penis fast’ spammer murdered’ fake blog posting I wrote about last month? I was right — the site has now become a spammer link farm.

There’s now a new category in the right-hand sidebar of the fake blog post. See if you can spot the odd one out:

  • Programming
  • Personal
  • Web 2.0
  • Python
  • Penis exercises
  • Uncategorized

Sure enough, "Penis exercises" is the only valid outlink from the page (all the others lead to the ‘sorry, closed due to too much traffic’ page). It leads to a page discussing the usual ‘make penis fast’ topics, with a batch more links to more pages along the same lines. If you follow the links a little, the whole thing appears to be hawking some device called "Size Genetics". Totally spammy.

New job!

Published November 13, 2007

So, as I’ve hinted previously, I’ve left Vast to work full-time at a new gig: PutPlace.

I’ll be working on more EC2/S3/SQS-related large-scale cluster stuff, and on their open-source plans… looking forward to that. They’re a great team — lots of familiar faces from the Iona days — and it finally gets me out of telecommuting from home, back into an office again after 5 years ;)

Joe has <a href="http://putplace.wordpress.com/2007/11/13/justin-mason-joins-putplace/"> put up a nice blog post welcoming me. Cheers Joe!

Now to get to grips with Python. (I still love Perl though. ;)

Fedex Ireland and unfair duty charges

Published November 12, 2007

I’ve been on vacation for a week, introducing Bea to the many joys of the bogs of Connemara. I think she liked it.

While I was away, I appeared in Ireland’s newspaper of record, the Irish Times, specifically in Conor Pope’s ‘Pricewatch’ consumer-affairs column, under the byline "Shopped to the taxman". Here’s a cut-and-paste of some relevant snippets:

Justin Mason [hey, that’s me] contacted Pricewatch after being hit with just such a charge. In August, he and his wife, who were expecting a baby, received a package from friends in the US [thanks Nishad and Janet!] containing amongst other things, some hats, socks and a little hoodie for their baby.

"It was shipped via FedEx, got here in good time and was very cute," he says. The couple were delighted, until a couple of weeks later, when they received an invoice from FedEx looking for EUR 34.47, made up of EUR 2.49 duty, EUR 19.88 VAT and EUR 10 in "administration fees", plus an additional EUR 2.10 VAT on the "administration fee".

"This strikes me as pretty unfair, maybe there’s duty payable, but I’ve never had to pay VAT on a gift I’ve received before? On top of that, being charged one-third of the price as an administrative fee? Ouch!"

The couple disputed the fee and were told if they didn’t pay, the invoice would be sent to a debt collection agency and non-payment would affect their credit rating. A couple of weeks later, another gift arrived from the US, followed by another invoice looking for EUR 7.84 in duty, plus the EUR 10 administration fee and EUR 2.10 VAT on that fee. Mason disputed the charge and was eventually told it would be waived as it had a value of less than $50 (EUR 34.70) and was clearly labelled as a gift. There is tax relief called Small Parcel Standard Relief on goods purchased from outside the EU, which is EUR 22 for bought goods and EUR 45 for gifts, so the tax should never have been applied by FedEx.

We contacted FedEx and UPS, highlighting our readers’ concerns. A spokesman for FedEx said the administration charge has always been in place in Ireland and was applied "to ensure customers receive their packages quickly".

He said that if it did not pay the VAT and duty, "packages would not be cleared through customs until the customer has paid them, thus adding severe delays to the delivery process".

So, to be honest, I’m not impressed at all with Fedex’ response here. I was hoping they’d be more helpful, especially once it hit the most significant consumer-affairs column in the country — but not at all :(

To recap — since Conor didn’t mention it — here are my problems with the charges:

  • the packages were both genuine, unsolicited, gifts. Surely having to pay duty on a gift is not applicable; it certainly makes receiving a gift a particularly unpleasant experience!

  • the first package contained baby clothes, which are VAT-free in Irish tax law anyway.

  • we cannot seem to get contact details for someone at Customs and Excise to talk to about this, and Fedex have failed to get back to us since then.

Not sure what the next step is…

There’s also a little follow-on discussion at Conor’s blog.

Update: good news. A couple of days ago, a letter arrived from Fedex UK, containing 2 credit notes; both invoices had been reduced to EUR 0.00, citing "incorrect application of duty" for one, and "customer satisfaction policy" for the other. Hooray!

Surprise smash hit in the Irish Blogs Top 100

Published November 2, 2007

Damien posted an interesting suggestion for the Irish Blogs Top 100 the other day — during discussion of which, it emerged that there were a few overlooked Irish blogs which hadn’t yet shown up on the planet.journals.ie Irish blogs aggregator, and therefore were not appearing in the Top 100. These were:

Anyway, they’re in now. When I first spun up the script and checked the results, though I was a bit shocked and had to do a bit of a double-take — at number 1, far beyond Damien’s number 2, was InPhotos.org, with a Technorati Rank of 1 and 102,857 inbound links from 88,772 blogs, compared to Damien’s Rank of 7946 with 1,606 links from 519 blogs.

Insane! I guess being in the default WordPress install makes a bit of difference there ;)

Interestingly, InPhotos.org, with a Technorati Authority of 88,434, is far beyond the most popular blog listed on the Technorati Popular Blogs page. It seems that page is a hand-tweaked set of blogs, and not just a "Technorati global Top 100", then, despite what one might naively assume…

PS: Damien’s original suggestion, btw, was to measure blog popularity using Google Reader and Feedburner’s audience stats. However, I can’t do that without a public API I’m allowed to scrape. Does anyone know of one?

Also worth noting that I recently added del.icio.us bookmarks as a metric of popularity, to go with the Technorati stuff. It’s interesting to see how those rankings differ — bloggers and bookmarkers don’t always agree, with bookmarkers preferring MP3s, Second Life, and politics I reckon.

the Ron Paul spam scandal

Published November 1, 2007

A US presidential candidate called Ron Paul has been advertised in spam. There’s currently a massive shitstorm raging about the true source of the spam — it was delivered via an infected consumer broadband machine, so the source is of course untraceable from the email alone.

Of course, being spam, I received a copy ;) Here’s a spample, if you’re curious.

The unusual "Content-Type" header format (matching the STOX_REPLY_TYPE SpamAssassin rule) has been seen in a lot of pump-and-dump stock spam recently. (It’s also shown up in Storm output, but this isn’t from Storm.) It’s been around for at least 6 months, so it’s probably a built-in behaviour of a downloaded spamware app, rather than a frequently-updated web-hosted spamware site.

My guess — I’d say the spam was sent using the same spamware application that one of the larger, recent pump-and-dump spammers has been using — so a reasonably sophisticated app, and not just an ancient copy of DarkMailer or whatever.

It’ll be interesting to see how this pans out…

Changes to the Irish learner driver system

Published October 26, 2007

The Irish Road Safety Authority have just revised Irish law as it relates to ‘learner drivers’, the 15% of drivers who haven’t yet passed a driving test. (This includes me — my US driving license doesn’t allow me to drive a manual-transmission car in Ireland, so I’m still a learner over here!)

They helpfully released the details as a rather broad PDF entitled ‘Road Safety Strategy 2007-2012‘, which covers the changes along with other plans and statistics; and a more focused document, ‘Learner Permit and Changes to the Driver Licensing System‘, dealing with just the learner-permit system.

Unfortunately, the latter was released as an MS Word document. Given the problems this raises — lack of searchability, integration with the web, etc. — I thought it’d be helpful for searchers if I put up the text in full here, so here it is.

Introduction of Learner Permit and Changes to the Driver Licensing System – Changes to the Driver Licensing System announced on 25 October 2007

In this document you will find information about changes to the driver licensing regime. These changes affect learner drivers and recognise the fact that learner drivers are a vulnerable group of road users. The changes also serve to emphasise the importance of the learning phase for drivers, one element of this is the replacement of provisional licences with learner permits. The changes also highlight the important role played by the driver who accompanies a learner driver.

Over time the intention is to expand the range of conditions applying to a learner permit and to develop a graduated licensing system where there will be a number of different restrictions/conditions applying at different stages. These restrictions will apply while driving with a learner permit and in the initial years of driving with a full driving licence.

Specific details about each of the current changes together with questions and answers on the impact of each change are set out below.

Provisional licences are being replaced by learner permits to emphasise the fact that the holder is a probationary driver and is learning to drive. Existing provisional licences will continue in force until their expiry date. On renewal the person will be issued with a learner permit.

Q: When will learner permits start to issue?

A: Learner permits will issue as and from 30 October 2007.

Q: Does the learner permit system apply to all driving licence categories?

A: Yes, the learner permit system will apply to all licence categories.

Q: Is there any change to the period of validity or the fee for a learner permit compared to that for a provisional licence?

A: No, the duration and fee remain the same as applied to provisional licences.

Q: Are there any changes to apply under the learner permit system?

A: A number of changes detailed below are being introduced for drivers with a learner permit. These are also being applied to drivers with a current provisional licence.

The holder of category B (Car) learner permit (provisional licence) must be accompanied by and under the supervision of a qualified person at all times. This change removes an exemption that, up to now, allowed a person on a second provisional licence to drive unaccompanied. To drive unaccompanied will be a penal offence and the person will be subject to prosecution.

Q: When does this new rule come into effect?

A: This is coming into effect as and from 30 October 2007.

Q: I am currently on a second (provisional licence) learner permit for driving a car, and was not required to be accompanied heretofore with this (provisional licence) learner permit. Must I now be accompanied?

A: Yes, you must be accompanied at all times when driving with a (provisional licence) learner permit for a car.

Q: I have passed the driving test in a vehicle with an automatic transmission and now hold a (provisional licence) learner permit for driving a car with a manual transmission, can I drive this car unaccompanied.

A: No, you must be accompanied by a qualified person until such time as you pass the driving test for a manual transmission car.

Q: In respect of which licence categories is a person who holds a (provisional licence) learner permit required to be accompanied by a qualified person?

A: Drivers with a (provisional licence) learner permit for vehicles of category B, C1, C, D1, D, EB, EC1, EC, ED1 or ED, (Cars, Trucks, Buses and Articulated Vehicles) must be accompanied by and under the supervision of a qualified person.

An accompanying qualified person must hold a full driving licence for the vehicle category for at least two years. It will be a penal offence for the driver not to be accompanied by a qualified person so licenced to drive.

Q. When is this change coming into effect?

A. This change will apply as and from 30 October 2007.

Q: If I am a learner driver driving a car and the accompanying person has held a driving licence for two years in respect of a motorcycle, or a tractor/work vehicle, can this person act as an accompanying qualified person?

A: No, the accompanying qualified person must hold a driving licence for two years for the category of vehicle you are driving.

Q: If a person has passed a driving test to drive the vehicle category, can this person act as an accompanying qualified person?

A: No.

Q: If a person has held a full driving licence for an automatic vehicle for two years, may this person act as the accompanying person?

A: Yes, but only if the learner driver is driving an automatic transmission vehicle in the same category. If s/he is driving a manual transmission vehicle, the accompanying qualified person has to hold a full driving licence for at least two years for a manual transmission vehicle.

Q: If I have a learner permit (provisional licence) in category C1 (small truck) can I be accompanied by a person who holds a full driving licence for category B for two years and for category C1 for one year?

A: No, the accompanying qualified person must hold a full driving licence for two years in respect of the vehicle category which you wish to drive, in this case category C1.

Q: If the accompanying driver has heId his / her driving licence since six years ago but has been disqualified for 2 of the last 3 years, may he /she act as an accompanying driver?

A: No, the accompanying qualified person, at the time you are driving, must hold a full driving licence for two years in respect of the vehicle category which you wish to drive. He/she must not have been disqualified for any period of the previous two years.

The carrying of a passenger by a motorcyclist with a (provisional licence) learner permit is a penal offence.

Q. When is this change coming into effect?

A. This change will apply as and from 30 October 2007.

Q: Can I carry a passenger on any motorcycle category for which I hold a learner permit (provisional licence) ?

A: No, you must have a full driving licence for the motorcycle in order to be able to carry a passenger.

Q: Can I carry a passenger on a category A motorcycle for which I hold a learner permit/ provisional licence if I have a full driving licence for category A1?

A: No.

Q: If I pass the motorcycle driving test, can I carry a passenger?

A: No, you must first exchange your certificate of competency (driving test pass certificate) for a full driving licence to be able to carry a passenger.

It is a penal offence for a holder of a category W (Tractor/Works vehicle) learner permit (provisional licence) to carry a passenger unless the vehicle is constructed or adapted to carry a passenger and the passenger is a qualified person, ie. a person who holds a full driving licence for the vehicle category for at least two years.

Q. When is this change coming into effect?

A. This change will apply as and from 30 October 2007.

Q: When can I carry a passenger?

A: When the passenger holds a driving licence for the vehicle category for at least two years, and where the vehicle is constructed or adapted to carry a passenger.

Q: Can I carry a passenger who is a qualified person if there is no passenger seat?

A: No, the vehicle must be constructed/ adapted for the carriage of a passenger.

It is a penal offence for the holder of a learner permit (provisional licence) in respect of any licence category to carry in the vehicle any passenger for reward.

Q. When is this change coming into effect?

A. This change will apply as and from 30 October 2007.

Q: Can I carry a passenger for reward in the course of my employment?

A: No, you may not do so while driving under a learner permit (provisional licence).

Q: If I have a category D1 learner permit (provisional licence) to drive a minibus, can I carry a passenger for reward?

A: No, you may not do so while driving under a learner permit (provisional licence).

It is a penal offence for the holder of a learner permit (provisional licence) for vehicles of category B, C1, C, D1, D, EB, EC1, EC, ED1 or ED, to drive such a vehicle unless there are displayed on the vehicle rectangular plates or signs bearing the letter ‘L’ not less than 15 centimetres high in red on a white ground, in clearly visible vertical positions to the front and rear of the vehicle.

Q. When is this change coming into effect?

A. This change will apply as and from 30 October 2007.

Q: If I have a category B full driving licence and a learner permit for category C (truck) or category D1 (minibus) must I display L plates?

A: Yes, you must display L plates on the truck or minibus if driving on a learner permit.

It will be a penal offence for the holder of a learner permit (provisional licence) for vehicles of category B, C1, C, D1 or D, to drive such a vehicle while the vehicle is drawing a trailer.

Q: If I have a category B driving licence and a learner permit for category C1 (small truck) can I draw a trailer?

A: No, you may not drive a truck while drawing a trailer if you hold a learner permit (provisional licence) for a truck. You must have the trailer entitlement for the category on the learner permit (provisional licence) in order to draw a trailer.

Learner Motorcyclist to display ‘L’ plates on a high visibility tabard.

Q: From what date will motorcyclists have to display L plates on a high visibility tabard?

A: It takes effect as and from 1 December 2007.

Q: Which learner motorcyclists are required to display L plates on a high visibility tabard?

A: All persons with a learner permit (provisional licence) for category A, A1, or M, must when driving such a vehicle display a yellow fluorescent tabard bearing the letter ‘L’ not less than 15 centimetres high in red on a white ground, in clearly visible vertical positions worn over the chest clothing. The ‘L’ plates are to be to the front and rear of the person’s torso. It will be a penal offence not to so display L plates.

A person who is a first time holder of a learner permit (provisional licence) cannot take a driving test for a six month period after the commencement date of the permit (provisional licence).

Q. When is this change coming into effect?

A. This change will apply to driving test applicants with an appointment date for a test on or after 1 December 2007 and who hold a learner permit (provisional licence) for less than six months. At this point driving tests are scheduled up to this date and the change will not affect existing appointment holders.

Q: Does the change apply to all licence categories?

A: Yes, It applies to all licence categories.

Q: Why is the six month limitation being applied?

A: The purpose of the provisional licence/learner permit is to allow a learner driver to gain experience of driving. Research shows that the longer a learner is supervised while driving, the less likely s/he is to be involved in an accident. For this reason the six months limitation is being applied.

Q: I hold a first learner permit (provisional licence ) for less than six months. I have an appointment already arranged for a driving test. Can I take the test?

A: Yes, the change is being introduced with effect from 1 December 2007 and should not affect existing appointments for driving tests.

Upcoming Mike Culver talk about AWS

Published October 25, 2007

Mike Culver, Amazon’s "Web Services Evangelist", will be in Dublin next week to evangelize about the goodness that is Amazon S3, EC2, SQS and so on. It seems he’ll be talking at the following locations:

  • in the Auditorium of the Digital Exchange, Crane Street, Dublin 8 on Tuesday October 30th, 3-5pm; here’s a flyer the Amazonites have been passing around. (upcoming.org page)

  • according to Damien, later that evening, he’s in the Westin Hotel on Westmoreland St., D2, starting at 7pm; note, it seems you need to book places at this, see Damien’s post.

  • and again at the Irish Linux User’s Group on Thursday November 1st at 19:30 in the Irish Computer Society in Dublin (map).

I guess these are all going to be same talk, bar the Q&A ;)

There was some kind of an ICTE get-together mooted for Friday 2nd.

Also, the ILUG annual general meeting is scheduled on the following Saturday, 3rd November, also at the ICS. Gareth Eason notes ‘we’re hoping to start at 3pm sharp, with talks from Dave Wilson (HEAnet), Frank Duignan, John Looney (Google), and others, followed by a relaxing wind-down in the Schoolhouse pub later on.’ (upcoming.org page)

Hopefully I’ll get to at least one of the AWS talks (probably the Digital Exchange one) and the ILUG AGM… busy week!

BBC’s iPlayer — what a mess

Published October 24, 2007

I haven’t paid a whole lot of attention to the BBC’s "iPlayer" project, since, as a non-UK resident, I’m not allowed to use it anyway. But this interview at Groklaw with Mark Taylor, President of the UK Open Source Consortium, was really quite eye-opening. Here’s some choice snippets.

On the management team’s Microsoft links:

The iPlayer is not what it claimed to be, it is built top-to-bottom on a Microsoft-only stack. The BBC management team who are responsible for the iPlayer are a checklist of senior employees from Microsoft who were involved with Windows Media. A gentleman called Erik Huggers who’s responsible for the iPlayer project in the BBC, his immediately previous job was director at Microsoft for Europe, Middle East & Africa responsible for Windows Media. He presided over the division of Windows Media when it was the subject of the European Commission’s antitrust case. He was the senior director responsible. He’s now shown up responsible for the iPlayer project.

On their attempts to bullshit the BBC Trust on the cross-platform issue:

In the consultations that the BBC Trust made, there were 10,000 responses from the public. And the overwhelming majority of them, over 80% — which is an unheard-of figure in these kind of things — said, we don’t like the platform. We don’t like it being single-platform. So it’s a big issue. And the BBC Trust said to us, "Why the vehemence? Why have people reacted this way?" And I explained the ‘Auntie’ analogy. It’s people don’t expect that from the BBC. It’s got this huge history of integrity, doing the right thing, standing up to bullies. (laughter) They’ve done this for a very long time. And people find that it’s surprising. And they said, "Yeah, but," you know, the BBC guys said, "Well, trust us. This is going to be cross-platform." And we said, "Well, how? It’s completely single-platform." They say that, but we haven’t been able to find anyone who’s been able to explain how they’re going to achieve that at the moment, even though they’re entirely locked into one single platform.

(aside: MS did this at one point with Internet Explorer — remember, there was some mystery team in Germany that supposedly had IE ported to Solaris, hence it therefore qualified as ‘cross-platform’.)

On the architecture of the product:

Q: it’s a Verisign Kontiki architecture, it’s peer-to-peer, and in fact one of the more worrying aspects is that you have no control over your node. It loads at boot time under Windows, the BBC can use as much of your bandwidth as they please (laughter), in fact I think OFCOM … made some kind of estimate as to how many hundreds of millions of pounds that would cost everyone […]. There is a hidden directory called "My Deliveries" which pre-caches large preview files, it phones home to the Microsoft DRM servers of course, it logs all the iPlayer activity and errors with identifiers in an unencrypted file. Now, does this assessment agree with what you’ve looked at?

Mark Taylor: Yes.

Q: What are the privacy implications for an implementation like this?

Mark Taylor: Well, just briefly going back to the assessment thing, yes it does log precisely RSS and stuff like that and more importantly, anyone technically informed who’s had a look at it — even more importantly, the user’s assessment as well and — frankly horrified if you go and spend some time in the BBC iPlayer forums, it’s eye-opening to see the sheer horror of the users, some of them technically not — you know, relatively early-stage users — but when it gets explained to them by some of the longer-using users of it, it’s concentrated misery. (laughter)

[…]

it’s a remarkable thing with them as well, there’s a lot of pain going on in the user forums, and some of the main technical support questions in there are "how do I remove Kontiki from my computer?" See, it’s not just while iPlayer is running that Kontiki is going, it’s booted up. When the machine boots up, it runs in the background, and it’s eating people’s bandwidth all the time. (laughter) In the UK we still have massive amounts of people who’ve got bandwidth capping from their ISPs and we’ve got poor users on the online forums saying, "Well, my internet connection has just finished, my ISP tells me I’ve used up all of my bandwidth."

Q: It uses up their quota, but they can’t throttle it, they can’t reduce it —

Mark Taylor: No, they can’t throttle it. […] It’s malware as well as spyware.

And to top this off, there’s a (frankly insane) budget of UKP 130,000,000 to build this — that’s $266,000,000 — for something that could be built better by just hiring the guys behind UKNova and simply negotiating with the rights-holders directly.

Holy crap. Talk about a technical disaster masquerading as a solution to a business problem…

Plug: Decorama stickers

Published October 17, 2007

Plug plug! We picked up some really cute stencils for the nursery a few months back, but took our time putting them up — we were a bit daunted by the instructions — and only got around to putting them up last week. (We needn’t have worried — it was really easy.)

They’re Decorama vinyl stickers from Bored Inc.. I can’t recommend them enough — their art is fantastic, the quality’s great, and Bored Inc. were really friendly and helpful about the whole transaction.

If you’re looking to do something similar, I’d definitely recommend their stuff.

‘Blended threat’ = Storm

Published October 17, 2007

[Commtouch have apparently released an ‘Email Threats Trend Report’ for the third quarter of 2007], which contains this factoid:

Blended threat messages — or spam messages with links to malicious URLs — accounted for up to 8% of all global email traffic during the peaks of various attacks during the quarter […]

Spam with malware hyperlinks inside: One technique which reached a new high during the quarter was innocent-appearing spam messages that contained hyperlinks to malware-sites. This type of spam utilizes vast zombie botnets to launch ‘drive-by downloads’ and evade detection by most anti-virus engines. Several blended spam attacks of this type focused on leisure-time activities, such as sports and video games. Messages invited consumers to download "fun" software such as NFL game-tracking and video games from what appeared to be legitimate websites. Instead, consumers voluntarily downloaded malware onto their computers.

Those short messages that invited downloads of NFL game-tracking software ("Get Your Free NFL Game Tracker", "Football Fan Essentials", "Are you ready for football season?" etc.), and video games ("Wow, free games!", "New game software, with over 1000 games—FREE", "Holy cow, 1000 free games online" etc.), is all output from the Storm worm — I wouldn’t call it a new kind of "blended threat" per se. I’m surprised that Commtouch didn’t name it; maybe they don’t realise it’s Storm?

I’d say it’s output is higher than 8% of my incoming spam, although it has reduced its spam output quite a bit recently.

‘Dead spammer’ story a hoax

Published October 11, 2007

Update: yep, it’s spam.

Earlier today, Digg and Reddit featured this story:

Alexey Tolstokozhev (btw, in Russian his name means ‘Thick Skin’), a Russian spammer, found murdered in his luxury house near Moscow. He has been shot several times with one bullet stuck in his head. According to authorities, this last head shot is a clear mark of russian hit men (known as "killers" in Russia).

Since then, it’s received plenty of attention — I even posted it to my link blog myself. Unfortunately, I’m now certain it’s a fake. (Igor at the McAfee AVERT blog concurs.)

Here are my reasons:

  • There are still no corroborating stories in the press, several hours later;

  • ‘Alexey Tolstokozhev’ doesn’t appear in ROKSO, or even Google;

  • The entire site claims to have been shut down due to load, all except for that one page — there isn’t a single link that can be reached that works;

  • Indeed, Google has no other pages indexed on that site, which is pretty odd for a weblog;

  • And most fishy of all, the domain was registered yesterday, using a privacy-protection service, on Estdomains (which has a poor reputation).

All very fishy. My guess is that in a week’s time, that page will be a linkfarm, picking up all that Google juice for free. In other words, loonov.com is a spam site…

Update: Greetings, Slashdot comment readers! Hopefully that uncritical article (which was posted after this one) will be fixed to note the hoax soon…

Other voices have since added their agreement — Alex Eckelberry at Sunbelt software added his a few minutes after I posted this, and the Register wrote an article this morning about it.

(BTW, just to save some face — I’d like to note that I smelled a rat at the time I posted it initially, qualifying the link with a sceptical ‘hmm’. I’m not that gullible ;)

Update 2: the /. story was fixed by Zonk: ‘Good story. Unfortunately, probably a fake.’

Scary Storm figure

Published October 11, 2007

This study of the Storm worm (via) contains this rather terrifying factoid:

Figure 12 illustrates a time-volume graph of TCP packets, SMTP packets, spam messages, and smtp servers. Our analysis of this graph reveals the following findings. First, we find that except for the first 5 minutes almost all the TCP communication is dominated by spam. Second, we measured that hosts generate on average of 100 successful spam messages per five minutes, which translates to 1200 spam messages per hour or 28,800 messages per day. If we mutiply this by the estimated size for the Storm network (which we suspect varies between 1 million and 5 million, we derive that the total number of spam messages that could be generated by Storm is somewhere between 28 billion and 140 billon per day.

While such numbers might be mind-boggling they are inline with observed spam volumes in the Internet, e.g., overall volume of spam messages in the Internet per day in 2006 was estimated to be around 140 billion [2]; Spamhaus claims to have been blocking over 50 billion spam messages per day in October 2006 [10], and AOL was blocking 1.5 billion spam messages per day in its network in June 2006 [5]. These numbers suggest that Storm could be responsible for anywhere between 17% and 50% of all spam that is generated on the Internet.

28 to 140 billion messages per day. That is a lot of spam.

Minor nitpick with the paper — it notes that

Storm retrieves emails found in [certain] files and gathers information about possible hosts, users, and mailing lists that are referenced in these files. In particular, it looks for strings like “yahoo.com”, “gmail.com”, “rating@”, “f-secur”, “news”, “update”, “anyone@”, “bugs@”, “contract@”, “feste”, “gold-certs@”, “help@”, “info@”, “nobody@”, “noone@”, “kasp”, “admin”, “icrosoft”, “support”, “ntivi”, “unix”, “bsd”, “linux”, “listserv”, “certific”, “sopho”, “@foo”, “@iana”, “free-av”, “@messagelab”, “winzip”, “google”, “winrar”, “samples” , “abuse”, “panda”, “cafee”, “spam”, “pgp”, “@avp.” , “noreply” , “local”, “root@”, and “postmaster@”.

I would postulate that those strings are a stoplist — that in fact the worm avoids sending spam to addresses containing those strings. The presence of "abuse" and "postmaster" in particular would suggest that.

Long-lived spam via Yahoo! search

Published October 3, 2007

Back in May, I noticed some spam in my Moin Moin wiki, and fixed it.

As this Yahoo! Site Explorer view of taint.org demonstrates, Yahoo!’s search is still showing these results, partly; despite the spam content being long deleted (example ), they still show the spam title and URL, despite the fact that the title and text no longer contains those spam keywords.

Annoyingly, I’m still seeing referrer clickthroughs from search.yahoo.com to these deleted pages from lusers looking for porn, as a result. Come on Yahoo!, fix your search to notice the title change at least, so people don’t think the pages still contain porn!

Eircom WEP key-generation algorithm reversed

Published October 1, 2007

Over the weekend, this really hit the Irish blogosphere — several Irish guys have apparently figured out the algorithm used by Eircom to generate WEP keys.

I blogged that page in the link-blog this morning, but it’s worth writing about a little more. WEP is apparently easy to crack nowadays, so in a way all those wifi users were insecure anyway — but this is interesting as a case study of how not to write a key generator:

  • Compiled code != secret: the first mistake Eircom made was to generate the WEP key entirely from a little "secret" text, some "secret" shuffles, and the serial number of the hardware. There should always be some randomness in there. Compiled code running on a user’s desktop, is not secret.

  • Don’t share secrets: Secondly, it’s a good demo of why you don’t generate two separate key values from the same source data. In this case, both the WEP key and the SSID are generated from the Netopia router’s serial number — and sufficient bits are accidentally exposed in the SSID to enable computation of the WEP key. (This is kind of moot in many cases, since the serial number is also exposed in the MAC address, in even more detail.)

As far as I can tell — although it’s not quite clear who did what — that guy Kevin Devine did a pretty great job of reversing this code. Nice one.

I’m impressed that there’s now an app which detects the static tables (S-boxes, constants etc.) used in crypto algorithms — that idea seems very clever in retrospect, hadn’t occurred to me.

Here’s a boards.ie thread where this exploit was discussed; there are plenty more details there, if you’re curious. It seems this has been quietly floating around back-channels since the start of September.

(By the way, am I missing something, or did Eircom ship unstripped binaries for the key generator library? I could swear that when I looked at the Boards thread earlier today, there was a cut-and-paste from IDA Pro listing a function prototype. Oh dear; if so, add that to the ‘case study’ list above. ;)

It seems Eircom are now recommending all customers switch to WPA — good luck with that, since it’ll break all those Nintendo DSes. That won’t be popular!

Update: the original page seems to be down, but here’s the source for the command-line decoder: dessid.c. See also EirWep.

Oh noes!

Published September 24, 2007

dsc05400
Originally uploaded by jmason

Sorry to readers of Planet Antispam — it had stopped updating for a week, after the server move. I’d forgotten to restart the cron job… now fixed.

Taint.org Has Moved

Published September 21, 2007

I’m moving pretty much all my home sites and infrastructure from the venerable "dogma.boxhost.net" to a new host, "soman.fdntech.com". This weblog has just made the jump. Please leave a comment if you notice anything awry.

There may be a few rough edges, since I upgraded to WordPress 2.2.2 in the process; for example, my sooper-s3kr1t "what is my name" anti-spam protocol was set to not require a preview of all posted comments, or the correct answer — in just over an hour I received 25 spam comments… so it’s good to know it’s working ;)

Dublin-area Intro To Open Streetmap

Published August 31, 2007

A last-minute notice — the Irish Linux Users’ Group are organising an introduction to Open Streetmap tomorrow:

Open Streetmap : An Intro

The ILUG committee is organising an introduction to the Open Streetmap project on Saturday, 1st September, 2007 in Dublin.

This will include info on how to use your GPS and upload your data to the project, to contribute to a free and open map of the world.

The Hamlet Pub, Balbriggan (N 53.61396 W 6.20608 degrees)

Sat, 1st Sep 2007 2pm ~ 5pm

If you have a GPS and a laptop, please feel free to bring them. Wireless internet is available in the venue.

To register interest, please e-mail chairman-at-linux.ie

Not Cosmo

Published August 30, 2007

So, we were all set to name our new arrival Cosmo, assuming it was a boy. We were certain it was going to be a boy. Guess what? It wasn’t… so now we have to narrow down the girl-name shortlist in a hurry!

Isn’t she lovely? Lots more photees at Flickr.

Anyway, I may be hard to get hold of for a while… this lady will be keeping me busy I think ;)

Update: Looks like the name is Beatrice Lily Mason, although there’s still a fair bit of indecision, unfortunately ;)

Update 2: Beatrice Lily Gray Mason. Final answer!

Stupid Unicode Tricks

Published August 27, 2007

Cool Unicode trick, via Mantari — cut and paste this character into a Unicode-aware application (like this post’s comment box!), then type something and see what happens:

‫‬‭‮‪‫‬‭‮҉

My Nokia 770

Published August 16, 2007

A couple of weeks back, there was quite a bit of buzz in the Irish blogosphere and elsewhere about the Nokia 770; prices for new N770s had dropped from $290ish to a very reasonable $140 / EUR130-ish price-point. I, along with a good few others, bought one.

I bought mine through Expansys, with a free 1GB RS-MMC memory card. They’ve sold out and no longer have any N770s listed; however, Buy.com still seem to have them in stock, so if you’re interested, you can probably still pick one up. (It seems Nokia is trying to sell off their remaining N770 stock, cheap, with plans to drop support for the software platform. I’m fine with this, but it may put other buyers off.)

I’ve now been using it for a while, and am still happy. ;) Here are my recommended top apps:

Slimserver. Originally designed to operate as the backend software for the Squeezebox thin-client MP3 player, this has a fantastic UI built for the N770, and its MP3 stream output works perfectly on the tablet.

This is by far the neatest way to get at a 6000-song music library without a laptop; there was some talk in the GNOME community of making a decent DAAP client, but so far there’s no working results there that I could find. :(

maemo-mapper. This is a fantastic mapping app for the tablet; it presents map tiles downloaded from OpenStreetMap or Google Maps in an N770-optimized format, with the usual nice draggable UI. Bonus: it’ll work offline, so you can follow a route while online, then take the tablet along to help navigate.

Tip: once you start maemo-mapper, click the "Download…" button in the "Repository Manager" and it’ll download details for the 5 most useful map repositories, including Google and Virtual Earth.

FBReader. A very nice document reader; much nicer than trying to read long HTML pages in the builtin web browser, especially since it allows you to turn the device on its side.

In general, the Opera Mini browser works fine; be sure to enable Javascript and set up a swap file on the RS-MMC card first. It does all the basic HTML and rudimentary AJAX; Google Calendar is a no-go, but GMail and even Google Maps works adequately, modulo minor bugs. Plain Old HTML sites like Wikipedia, IMDB and so on all work great.

As long as you’re realistic about the platform, it won’t disappoint — video requires custom transcoding, for example, and proprietary apps like Flash and RealPlayer lag behind their desktop equivalents, but as far as I can tell that’s the case for every embedded platform. (Since I spent a couple of years developing such a platform, I’m quite comfortable with this.)

A really really nifty thing about the N770 is that it’s now entirely hackable — within 30 minutes of powering on, I was able to get a terminal window open with a root prompt, and was adding ext3 partitions to the RS-MMC card. Apps are installed using "apt-get". The terminal even has word-completion system optimized for the UNIX command-line – nice ;)

This SomethingAwful thread contains plenty more good tips. I’m happy I bought it — so many of these gadgets can wind up as an overpriced door-stop, but this is easily worth what I paid for it.

Update: this thread at InternetTabletTalk seems pretty chock-full of good advice, too.

Test my auto-generated ruleset

Published August 14, 2007

(I posted this to the SA users and dev lists, too.)

I’ve been working on a new way to auto-generate body rules recently (see previous posts). The results are checked into SVN trunk daily in the "rulesrc/sandbox/jm/20_sought.cf" file.

We haven’t had much time to figure out how to produce auto-generated 3.2.x rule updates for our entire ruleset at updates.SpamAssassin.org, so instead of dealing with that, I’ve taken a shortcut around it ;) I’m now making just the "20_sought.cf" ruleset available as a standalone, unofficial sa-update ruleset at sought.rules.yerp.org.

Before using it, you’ll need the GPG key:

  wget http://yerp.org/rules/GPG.KEY
  sudo sa-update --import GPG.KEY

then use this to update:

  sudo sa-update \
        --gpgkey 6C6191E3 --channel sought.rules.yerp.org \
        [...other channels...] \
        --channel updates.spamassassin.org

(similar to how you’d use Daryl’s sa-update version of the SARE rulesets.)

Feel free to run sa-update as frequently as you like.

Please consider it alpha; I may take it down in a few months depending on how it goes, or if we can get it working as part of the core updates. In the meantime though, I’m curious to hear how you get on with it. (In particular, copies of false positives would be very welcome.)

Update: it’s been very successful, so I’d now consider it in production.

The Prime Time Group pump-and-dump

Published August 14, 2007

Spamnation.info links to an interesting article by Computerworld’s Gregg Keizer about the massive PRTH.PK spam run.

As usual, there are no shortage of suckers:

The spam blast did drive up Prime Time’s share price from Monday’s low of around 7 cents to Wednesday’s high of 11 cents, a 57% jump. Thursday morning, however, the bottom dropped out, and the stock fell to under 7 cents. Trading volumes peaked Wednesday as well, at around 1.7 million shares, substantially higher than any day in the month prior. "You can actually see the wave of activity in the stock and compare it with the volume of spam that we trapped," said [Sophos analyst Ron] O’Brien.

But here’s an interesting new tactic by the good guys:

Last Wednesday afternoon, Prime Time announced that it was ordering a Non Objecting Beneficial Owners (NOBO) list to get a clearer picture of who owned its shares. "The NOBO list will be used to determine the naked short positions in Prime Time Group Inc.," the company said in a statement. "The finding will then be reported to the [National Association of Securities Dealers] to take action against the violators of the naked short regulations."

"Naked short" is a investment term that refers to selling short, essentially a bet that the price will drop, but with a twist: "naked" means that the investor sells short without first making sure he can borrow the shares from another investor holding a "long" position on the stock.

I hope this works; it’d be great to see the profit mechanism behind pump-and-dump spam killed off.

Spamnation notes:

Incidentally, the greeting card spam that built the botnet used to promote PRTH.PK and CYTV.OB also continues. It has iterated through another couple of generations: the current incarnation tells recipients to collect their custom Musical ecard or custom Movie-quality ecard or other variants on that theme. We’ve seen about 150 of these in the past three days, suggesting that the unknown senders are probably well on their way to building up another botnet for their next stock spam run.

Spreading trojans via greeting-card spam is a trademark of the gigantic Storm botnet, AFAIK: SecureWorks info, MessageLabs info, spam levels causing DDoS for Canadian networks, DDoS threat for EDU sector.

The Haughey 419 returns

Published August 7, 2007

A few months back, Blogorrah noted an amazing 419 scam, claiming to be a missive from ex-Taoiseach of Ireland Charlie Haughey‘s wife, Maureen. It’s really quite appropriate Charlie becoming the subject of a scam himself, given what he did to this country. But anyway… over the weekend, a new variant on the theme emerged:

From Mrs Maureen Haughey, ROI

My Dear Friend,

I am Maureen Haughey, widow of former Taoiseach of the Republic of Ireland, Charles J. Haughey and daughter of former Taoiseach of the Republic of Ireland and heir to de Valera, Sean F. Lemass.The Press has written a lot about unresolved mysteries and corruption surrounding CharlesÂ’s dealings, but I tell you something,my Charlie was a good man. He was human and he did whatever he did.

People marvel why I stuck with Charlie and didn’t speak during the mess that came with the exposure of his affairs with Terry Keane (I just hate to think of her). I had to stand by him through the tribunal times…. it was to do with what I’m doing now. No one knew the details of all Charlie’s financial dealings but me. I remain the only one who knows all who got loans from Charlie and didn’t come back to pay when he was disgraced. I am the only one who knows about these monies and the other Ansbacher accounts.

I write to you, an old weary woman, sick and almost tired of living. My end is near but I will not depart until my final mission is accomplished and I also write this with an unshaken belief in the power of aspirations and dreams of a human being. The Irish government thinks it can shave and reduce me to a poor widow but I have the winning ace. A few years ago, when we werenÂ’t sure if my Charlie would be convicted, he kept some money in trust for me in a Security and Finance company. He did not open the account in our names so it will not be traced to us to enable the past remain the past. The name on the account is Cedric de Vregille. I never thought Charlie would leave me so soon and it never occurred to me to ask if this name were fictitious or not or a name of any of his friends. I have tried to find this man but to no avail. The amount he deposited in this name is 30,000,000 (Thirty Million Euros).

I want an honest person to come forward and lay claims to this amount, moreover to use the funds as instructed by me. I have all the documents needed, I just need a face for the name. I have mapped out 30% of the funds for you, as you will help us (you and I) execute this job.

As soon as I receive your acceptance for this work I shall give you necessary details of my solicitor who will facilitate the release of the funds in your name. Please reply me via my personal email: maureen_haughey67@yahoo.co.uk

For my security and the sake of letting sleeping dogs lie, I strongly advice that you keep our dealings confidential. You can read more about my charlie from:

http://www.ireland.com/focus/haughey/ITstories/story11.htm

http://www.teachersparadise.com/ency/en/wikipedia/c/ch/charles_haughey.html

http://www.everything2.com/index.pl?node_id=548983&lastnode_id=0

Thank You.

Message sent using UebiMiau 2.7.2

It was sent via a webmail system at nildram.co.uk, from a proxy in Australia.

The writing is amazingly ornate — ‘I write to you, an old weary woman, sick and almost tired of living’, ‘the Irish government thinks it can shave and reduce me to a poor widow but I have the winning ace’, etc. Very odd stuff. Also, it looks spell-checked. And, once again, poor old cyclist Cedric de Vregille gets dragged into it, too! I wonder what he did to deserve that ;)

If you fancy scambaiting, ‘maureenhaughey67@yahoo.co.uk’ is the one to go for. These guys seem to be having a good go of it — ‘The thought of the Irish government trying to shave an old woman has shocked and appauled me, so I will assist in anyway possible.’_ ha!

Rule Discovery Progress Update

Published August 4, 2007

Back in March, I wrote a post about a new rule discovery algorithm I’d come up with, based on the BLAST bioinformatics algorithm. I’m still hacking on that; it’s gradually meandering towards production status, as time permits, so here’s an update on that progress.

There have been various tweaks to improve memory efficiency; I won’t go into those here, since they’re all in SVN history anyway. But the results are that the algorithm can now extract rules from 3500 spam and 50000 ham messages without consuming more than 36 MB of RAM, or hitting disk. It can also now generate a SpamAssassin rules file directly, and apply a basic set of QA parameters (required hit rate, required length of pattern, etc.).

On top of this, I’ve come up with a workflow to automatically generate a usable batch of rules, on a daily basis, from a spam and ham corpus. This works as follows:

  • Take a sample of the past 4 days traffic from our spamtrap network. Today this was about 3000 messages.

  • add the hand-vetted spam from my own accounts over the same period (this helps reduce bias, since spamtraps tend to collect a certain type of spam), about 3400 messages.

  • discard spams that scored over 10 points (to concentrate on the stuff we’re missing).

  • Pass the remaining 3517 spams, and text strings from over 50000 nonspam messages, into the "seek-phrases-in-log" script, specifying a minimum pattern length of 30 characters, and a minimum hitrate of 1% (in today’s corpus, a rule would have to hit at least 34 messages to qualify).

  • That script gronks for a couple of minutes, then produces an output rules file, in this case containing 28 rules, for human vetting. (Since I’ve started this workflow, I’ve only had to remove a couple of rules at this step, and not for false positives; instead, they were leaking spamtrap addresses.)

  • Once I’ve vetted it, I check it into rulesrc/sandbox/jm/20_sought.cf for testing by the SpamAssassin rule QA system.

The QA results for the ruleset from yesterday (Aug 3) can be seen here, and give a pretty good idea of how these rules have been performing over the past week or two; out of the nearly 70000 messages hit by the rules, only 2 ham mails are hit — 0.0009%.

In fact, I measured the ruleset’s overall performance in the logs provided by the 4 mass-check contributors who provided up-to-date data in yesterday’s nightly mass-check; bb-jm, jm, daf, dos, and theo (all SpamAssassin committers):

Contributor Hits Spams Percent
bb-jm 4249 24996 17.00%
jm 3450 14994 23.00%
daf 1236 35563 3.48%
dos 32867 100223 32.79%
theo 28077 382562 7.34%

(bb-jm and jm are both me; they scan different subsets of my mail.)

The "Percent" column measures the percentage of their spam collection that is hit by at least one of these rules; it works out to an average of 16.72% across all contributors. This is underestimating the true hitrate on "fresh" spam, too, since the mass-check corpora also include some really old spam collections (daf’s collection, for example, looks like it hasn’t been updated since the start of July).

Even better, a look at the score-map for these rules shows that they are, indeed, hitting the low-scoring spam that other rules don’t hit.

That’s pretty good going for an entirely-automated ruleset!

The next step is to come up with scores, and publish these for end-user use. I haven’t figured out how this’ll work yet; possibly we could even put them into the default "sa-update" channel, although the automated nature of these rules may mean this isn’t a goer.

If you’re interested, the hits-over-time graph for one of the rules (body JM_SEEK_ICZPZW / Home Networking For Dummies 3rd Edition \$10 /) can be viewed here.

Host monitoring with Jaiku

Published July 24, 2007

A few weeks back, we were having trouble with dogma, our shared server where taint.org is hosted, which would occasionally be unavailable for unknown reasons. We needed to monitor its availability so that it could be fixed when it crashed again, and we’d be able to investigate quickly. Since it was happening mostly out of working hours, SMS notification was essential.

Normally, that kind of monitoring is pretty basic stuff, and there’s plenty of services out there, from Host-Tracker.com to the more complex self-hosted apps like monit and Nagios which can do that. But looking around, I found that none of them offered SMS notification for free, and since this was our personal-use server, I wasn’t willing to sign up for a $10-per-month paid account to support it, or buy any hardware to act as a private SMS gateway.

Instead, I thought of Jaiku — the Finnish company which offers a microblogging/presence platform similar to Twitter. Jaiku had a couple of cool features:

  • SMS notifications
  • it’s possible to broadcast messages to a "channel", which others could subscribe to, IRC-style
  • it has an open API

This would allow me to notify any interested party of dogma’s downtime, allowing subscribers to subscribe and unsubscribe using whatever notification systems Jaiku support.

With a little perl and LWP, I rigged up a quick monitoring script to check http://taint.org/ via HTTP, and report if it was unavailable over the course of 5 retries in 50 seconds. If it was broken, the script sends a JSON-formatted POST request to Jaiku’s "presence.send" method, informing the target channel of the issue. (Perl source here.)

You can see the ‘#dogmastatus’ channel here — as you can see, we fixed the problem with dogma just over 2 weeks ago ;)

It’s worth noting that I had to set up an additional user, "downtimebot", on Jaiku to send the messages — otherwise I’d never see them on my configured mobile phone! Jaiku uses the optimisation that, if I sent the message, there’s no need to cc me with a copy of what I just sent; logical enough.

Anyway, if you’re interested in dogma’s availability (there might be one or two taint.org readers who are), feel free to add yourself to the #dogmastatus channel and receive any updates.

Update: Fergal noted that it’s pretty simple to use Cape Clear’s assembly framework to perform a HTTP ping test with output to Jabber/XMPP. nifty!