High Scalability - How Twitter Stores 250 Million Tweets a Day Using MySQL : MySQL as a storage backend -- basically an InnoDB store
mysql twitter scalability gizzard innodb performance database
2011
Using a Feistel Network for full-cycle permutation : nice algorithm. requires that the permuted set's size be a power of 2 however - although for smaller sets you can just skip to the next output value, since they're not going to repeat
algorithm - Generating shuffled range using a PRNG rather than shuffling - Stack Overflow : some reasonably good answers on using an LFSR or LCG to generate a full-cycle permutation with no repeats
French President's Residence 'Busted' For BitTorrent Piracy | TorrentFreak : 'According to data from YouHaveDownloaded.com, a range of downloads have been actioned from the Palace including a cam copy of Tower Heist, a telesync copy of Arthur Christmas, and music from The Beach Boys.' I love this. The data is, of course, filled with potential inaccuracies -- and that's the point
bittorrent surveillance downloading internet privacy france hadopi
SiliconRepublic story on CoderDojo : 'it's both incredible and poignant that a voluntary movement that was born in Ireland during the summer is about to go international. Coder Dojo, the brainchild of 19-year-old entrepreneur and programmer James Whelton from Cork and tech entrepreneur Bill Liao, began as a Saturday morning club for kids to teach each other software programming. It has grown into a national movement up and down Ireland, a place where kids and their parents can go and learn to write software code in a friendly environment. The first UK Coder Dojo was held in London only last week and other countries in Europe are clamouring to get the initiative started there, too.' Good on them!
coderdojo programming coding kids children teaching education tech ireland
Air France 447 Flight-Data Recorder Transcript - What Really Happened Aboard Air France 447 - Popular Mechanics : The (comp.)risks of overautomation strike again. "When trouble suddenly springs up and the computer decides that it can no longer cope—on a dark night, perhaps, in turbulence, far from land -- the humans might find themselves with a very incomplete notion of what's going on. They'll wonder: What instruments are reliable, and which can't be trusted?"
aviation crash flight flying autopilot stalls warnings alarms ui af447 risks automation
spark : sparklines in your terminal window. Simply give it a comma or space-separated list of data values, and it'll generate an ANSI-graphics sparkline chart. Brilliant! (via mjd)
via:mjdominus sparklines charts graphs bash shell terminal cli ansi
Script used by Instagram to duplicate the paged-in VM pages to another host : as seen in their "Instagram Engineering" post
vmtouch - the Virtual Memory Toucher : 'vmtouch is a tool for learning about and controlling the file system cache of unix and unix-like systems. It is BSD licensed.'
Scott Andreas - Garbage, Garbage Everywhere [slides] : 'GC Strategies for Event Processing Systems on the JVM'
gc java jvm event-streams event-processing tuning slides presentations scott-andreas performance
Large file management with git-annex : 'uses Git to manage files that are larger than Git can easily handle—without checking them into the repository. But git-annex provides ways to track those files using much of the same infrastructure as Git, so that moving or deleting those files can all be tracked in much the same way as committed files. In addition, git-annex allows for branches and distributed clones of its trees.' I may investigate using this to sync my MP3s instead of SVN
collectSPACE : 'The Source for Space History and Artifacts' -- and just in time for xmas too!
space spaaace memorabilia collecting gomi tat artifacts ebay science xmas
747s using VLANs to secure in-flight access to engine management systems : 'I was contracted to test the systems on a Boeing 747. They had added a new video system that ran over IP. They segregated this from the control systems using layer 2 VLANs. We managed to break the VLANs and access other systems and with source routing could access the Engine management systems.' (via Risks)
LINX Public Affairs » Scarlet wins in European Court : 'The Court judgement therefore goes well beyond saying what a court may decide, by means of an injunction: it also sets out the limits of Member States' powers to legislate to draft ISPs as copyright police. It will be a crucial precedent in future arguments about the Digital Economy Act, in the UK, HADOPI in France, various blocking requirements in Italy, and numerous other schemes across the EU. As victories for ISPs in the copyright wars go, this one was comprehensive. It will be seen as a landmark ruling for years to come.' woot
linx scarlet isps hadopi eu privacy filtering copyright irma filesharing
How does LMAX's disruptor pattern work? - Stack Overflow : LMAX's "Disruptor" concurrent-server pattern, claiming to be a higher-throughput, lower-latency, and lock-free alternative to the SEDA pattern using a massive ring buffer. Good discussion here at SO. (via Filippo)
via:filippo servers seda queueing concurrency disruptor patterns latency trading performance ring-buffers
Scrapheap Transhumanism : Lepht Anonym and the 'Grinders'. crazy stuff -- low-end DIY cybernetic augmentation. 'The implants sit in various places under my skin: middle fingertips of my left hand, back of the right hand, right forearm — tiny magnets, five or six millimeters across, coated in gold and then in silicon to isolate the delicate metal from the destructive environment of your body. They're something of an investment at about thirty euros apiece, and hard to get hold of, but worth pursuing. When implanted, they become technological sensory organs. There's an entire world of electromagnetic radiation out there, invisible to most. Our cities are saturated with it. A radio, for instance, gives off a field that's bigger than the device itself. So do power supplies and wires in the walls. The implants pick up on the fields, and because they're magnets, they fizz with gentle electricity, telling you this hard drive is currently active, that one is turned off, there's the main line in the wall. Holding a mobile phone, you can feel the signals it sends and receives. You know it's ringing before it starts to play any sounds, and when you answer it, you stick the touchscreen stylus to the back of your hand to hold it, then to your finger to type.'
diy augmentation cybernetics transhumanism lepht-anonym grinders biohacking cyberpunk medicine
Apache considered harmful : ouch
the legend of St. Columba, patron saint of copyright infringers : 'At this point IPKat team member Jeremy dons his old academic hat and excitedly draws attention to some research he did on the St Columba case. The goodly saint was given access to a psalter that was in the possession of Abbot Finian in around the year 560. A psalter is a book of psalms -- definitely public domain stuff, having been compiled during the reign of King David, who is generally reckoned to have died around 970 years before the common era. Even on a life + 70 year basis, copyright would have expired around getting on for 1,500 years before Columba came on to the scene. Having illicitly copied the psalter he refused to deliver it up to King Dermot of Tara, who famously said "to every cow its calf, to every book its copy" -- not "to every cow its calf, to every author his work". Anyway, to cut a long story short, Columba refused to hand it over, fled the country for the safety of England (like the founder of Wikileaks), converted the Picts to Christianity, settled in Iona and became a saint. You can read this all in "St Columba the Copyright Infringer" [1985] 12 European Intellectual Property Review 350-353.' (via Eoin O'Dell). Someone fill in the misquoting High Court judges....
st-columba books via:cearta ireland law history filesharing copyright
eclim (eclipse + vim) : 'Eclim is less of an application and more of an integration of two great projects. The first, Vim, is arguably one of the best text editors in existence. The second, Eclipse, provides many great tools for development in various languages. Each provides many features that can increase developer productivity, but both still leave something to be desired. Vim lacks native Java support and many of the advanced features available in Eclipse. Eclipse, on the other hand, still requires the use of the mouse for many things, and when compared to Vim, provides a less than ideal interface for editing text. That is where eclim comes into play. Instead of trying to write an IDE in Vim or a Vim editor in Eclipse, eclim provides an Eclipse plug-in that exposes Eclipse features through a server interface, and a set of Vim plug-ins that communicate with Eclipse over that interface. This functionality can be leveraged in three primary ways, as illustrated below.'
Determining response times with tcprstat : 'Tcprstat is a free, open-source TCP analysis tool that watches network traffic and computes the delay between requests and responses. From this it derives response-time statistics and prints them out.' Computes percentiles, too
tcp tcprstat tcp-ip networking measurement statistics performance instrumentation linux unix tools cli
DTrace and Erlang : from Basho, via istvan. DTrace is becoming more compelling as a deep instrumentation/monitoring API -- I didn't realise disabled DTrace probes were virtually 0-overhead (a "2 NOOP instruction placeholder", apparently), that's nifty. Wonder if they've fixed the licensing mess, though
dtrace monitoring instrumentation debugging tracing unix erlang via:istvan
Benchmarking Cassandra Scalability on AWS - Over a million writes per second : NetFlix' benchmarks -- impressively detailed. '48, 96, 144 and 288 instances', across 3 EC2 AZs in us-east, successfully scaling linearly
Inside the mind of the octopus : "Researchers who study octopuses are convinced that these boneless, alien animals—creatures whose ancestors diverged from the lineage that would lead to ours roughly 500 to 700 million years ago—have developed intelligence, emotions, and individual personalities. Their findings are challenging our understanding of consciousness itself."
Dark Sky - Weather Prediction, Reinvented by Adam Grossman & Jack Turner — Kickstarter : Yes! short-term weather prediction and dataviz. I've been vaguely considering ideas along these lines recently, but these guys have gone much further. US residents, fund it -- I really hope this gets made and makes it to Ireland...
Studded Bicycle Tires : Thinking of the winter commute again. This page has a load of fantastic info on winter tyres
Linux SS Utility To Investigate Sockets / Network Connections : 'When amount of sockets is enough large, netstat or even plain cat /proc/net/tcp/ cause nothing but pains and curses. In linux-2.4 the desease [sic] became worse: even if amount of sockets is small reading /proc/net/tcp/ is slow enough. This utility presents a new approach, which is supposed to scale well.' via scanlan
Storage Infrastructure Behind Facebook Messages : HBase and Haystack; all data LZO-compressed; very interesting approach to testing -- they 'shadow the real production workload into the test cluster to test before going into production'. This catches a 'high percentage' of issues before production. nice
testing shadowing haystack hbase facebook scalability lzo messaging sms via:james-hamilton
Avoiding Full GCs in HBase with MemStore-Local Allocation Buffers : Fascinating. Evading the Java GC by reimplementing a slab allocator, basically
memory allocation java gc jvm hbase memstore via:dehora slab-allocator
How to beat the CAP theorem : Nathan "Storm" Marz on building a dual realtime/batch stack. This lines up with something I've been building in work, so I'm happy ;)
peak6/scala-ssh-shell - GitHub : 'Backdoor that gives you a scala shell over ssh on your jvm. The shell is not sandboxed, anyone access the shell can touch anything in the jvm and do anything the jvm can do including modifying and deleting files, etc.' nifty!
Anrealage pixelled shop : 'Experimental japanese designer Kunihiko Morinaga has been inspired by pixel for his latest FW 2011 collection of his brand Anrealage. The best part is that this pixelled style has been applied also to his brand showroom in Harajuku, Tokyo.' Love the rug
anrealage pixelation pixel-art via:mlkshk rugs shops tokyo kunihiko-morinaga
the etymology of the anatomical term "Thagomizer" : 'The term was coined by Gary Larson in a 1982 Far Side comic strip, in which a group of cavemen in a faux-modern lecture hall are taught by their caveman professor that the spikes were named "after the late Thag Simmons". The term was picked up initially by Ken Carpenter, a palaeontologist at the Denver Museum of Nature and Science, who used the term when describing a fossil at the Society of Vertebrate Paleontology Annual Meeting in 1993. Thagomizer has since been adopted as an informal anatomical term, and is used by the Smithsonian Institution, the Dinosaur National Monument in Utah, the book The Complete Dinosaur and the BBC documentary series Planet Dinosaur.' (via John Looney)
via:john-looney thagomizer the-far-side comics til dinosaurs funny
Computer Virus Hits U.S. Drone Fleet : 'Predator and Reaper crews use removable hard drives to load map updates and transport mission videos from one computer to another. The virus is believed to have spread through these removable drives.' hmm, not quite sure how that air gap is supposed to work
Bayes' theorem ruled inadmissible in UK law courts : Bayes' theorem, and 'similar statistical analysis', ruled inadmissible in UK law courts (via Tony Finch)
Amazon hiring embedded OS developers : hey, I know a few of those! 'I need more help on a project I'm driving at Amazon where we continue to make big changes in our datacenter network to improve customer experience and drive down costs while, at the same time, deploying more gear into production each day than all of Amazon.com used back in 2000. It's an exciting time and we have big changes happening in networking. If you enjoy and have experience in operating systems, networking protocol stacks, or embedded systems and you would like to work on one of the biggest networks in the world, [get in touch].' -- James Hamilton
feedback loop n-gram analyzer : 'a simple parser of ARF compliant FBL complaints, which normalizes the email complaints and generates a 6-tuple n-gram version of the message. These n-grams are stored in a Redis database, keyed by the file in which they can be found. An inverse index also exists that allow you to find all messages containing a particular n-gram word.'
anti-spam spam fbl feedback filtering n-grams similarity hashing redis searching
Lovelace's Leap : a great observation from jgc. 'Lovelace realized that even though a computer was, at its heart, a mathematical machine, it wasn't restricted to doing mathematics. She realized that a computer could be used to process other types of 'information' by having numbers represent anything else. She realized that a computer could handle text, or music, or practically anything. That's Lovelace's Leap.'
jgc history ada-lovelace computing software information code babbage
Rectangular subdivisions of the world : 'Eric Fischer, who continues his string of mapping fun and doesn't even do it for his day job, maps the world in binary subdivisions. Each bounding box contains an equal number of geotagged tweets.' via Nelson
maps mapping bounding-boxes world earth geodata geotagging twitter
Hikaru Dorodango : 'Hikaru dorodango are balls of mud, molded by hand into perfect spheres, dried, and polished to an unbelievable luster. The process is simple, but the result makes it seem like alchemy. A traditional pastime among the children of Japan, the exact origin of hikaru dorodango is unknown.'
mud dirt dorodango japan art howto sculpture hands craft play children
Storm : 'The past decade has seen a revolution in data processing. MapReduce, Hadoop, and related technologies have made it possible to store and process data at scales previously unthinkable. Unfortunately, these data processing technologies are not realtime systems, nor are they meant to be. There's no hack that will turn Hadoop into a realtime system; realtime data processing has a fundamentally different set of requirements than batch processing. However, realtime data processing at massive scale is becoming more and more of a requirement for businesses. The lack of a "Hadoop of realtime" has become the biggest hole in the data processing ecosystem. Storm fills that hole.'
Storm: distributed and fault-tolerant realtime computation : intro slideshow to this really nifty-looking distcomp platform
Hacker News thread on Storm : lots of good questions and answers in here
Computer gamers solve problem in AIDS research that puzzled scientists for years : "This is the first instance that we are aware of in which online gamers solved a longstanding scientific problem," writes Khatib. "These results indicate the potential for integrating video games [like FoldIt] into the real-world scientific process: the ingenuity of game players is a formidable force that, if properly directed, can be used to solve a wide range of scientific problems."
foldit gaming games science biology aids viruses protease protein-folding proteins vr
Black Hat: Insulin pumps can be hacked : "Everything has an embedded processor and computer in it," he said. "Every time you hide behind [security by] obscurity, it is going to fail." Brad Smith, a researcher and Black Hat conference staffer who also is a registered nurse, said the medical field largely looks the other way when it comes to securing patient devices. "I lecture at all the medical conferences," he said during the press conference. "They just hide it. Pay attention to what [Radcliffe] is saying. His life is in this pump." (via Risks Digest)
via:risks insulin pump medicine security hacking health wireless
A few git tips you didn't know about : 'git checkout -t' alone is worth the bookmark
Conor O'Neill on his freesat/DTT system : 'Our replacement for Sky TV cost €99. Ariva 120. No monthly fees!' -- sounds very intriguing, that's a good price point
Golomb-coded sets : 'a probabilistic data structure conceptually similar to a Bloom filter, but with a more compact in-memory representation, and a slower query time.' could come in handy
gcs bloom-filters probabilistic data-structures memory algorithms
The Best Science Fiction Books (According to Reddit) : contains a surprisingly-large number which I haven't read
Dutch grepping Facebook for welfare fraud : 'The [Dutch] councils are working with a specialist Amsterdam research firm, using the type of computer software previously deployed only in counterterrorism, monitoring [LinkedIn, Facebook and Twitter] traffic for keywords and cross-referencing any suspicious information with digital lists of social welfare recipients. Among the giveaway terms, apparently, are "holiday" and "new car". If the automated software finds a match between one of these terms and a person claiming social welfare payments, the information is passed on to investigators to gather real-life evidence.' With a 30% false positive rate, apparently -- let's hope those investigations aren't too intrusive!
grep dutch holland via:tjmcintyre privacy facebook twitter linkedin welfare dole fraud false-positives searching
The Monkeysphere Project : OpenPGP's web of trust extending further. 'Everyone who has used a web browser has been interrupted by the "Are you sure you want to connect?" warning message, which occurs when the browser finds the site's certificate unacceptable. But web browser vendors (e.g. Microsoft or Mozilla) should not be responsible for determining whom (or what) the user trusts to certify the authenticity of a website, or the identity of another user online. The user herself should have the final say, and designation of trust should be done on the basis of human interaction. The Monkeysphere project aims to make that possibility a reality.'
Convergence : 'Convergence is a secure replacement for the Certificate Authority System. Rather than employing a traditionally hard-coded list of immutable CAs, Convergence allows you to configure a dynamic set of Notaries which use network perspective to validate your communication. Convergence allows you to choose who you want to trust, rather than having someone else's decision forced on you. You can revise your trust decisions at any time, so that you're not locked in to trusting anyone for longer than you want.'
ssl tls trust security https web via:filippo firefox plugins pki
Dave Neary on The Cost of Going it Alone : 'I'm going to talk about the costs associated with modifying and maintaining free software "out of tree" – that is, when you don't work with the developers of the software to have your changes integrated. But I'm also going to talk about the costs of working with upstream projects. It can be easy for us to forget that working upstream takes time and money – and we ignore that to our peril. It's in our interests as free software developers to make it as cost-effective as possible for people to work with us. Hopefully, if you're a commercial developer, you'll come away from this article with a better idea of when it's worthwhile to work upstream, and when it isn't. And if you're a community developer, perhaps this will give you some ideas about how to make it easier for people to work with you.'
Google App Engine Price Hike Stuns Developers - - Platform as a Service - Informationweek : 'Now that Google has begun offering App Engine users a way to calculate the new rate and compare it with the old rate, developers are realizing their bills will rise, by a factor of 10 or 100 or more in some cases, when the pricing change takes effect in a few months.' - ouch
Through speed of traffic on San Francisco area streets vs. popularity with Flickr and Twitter users : "slower streets" generate more photos/tweets than "faster streets", with a peak around 9 mph
Microsoft's new IE "Ribbon" debunked : 'nobody — almost literally 0% of users — uses the menu bar, and only 10% of users use the command bar. Nearly everybody is using the context menu or hotkeys. So the solution, obviously, is to make both the menu bar and the command bar bigger and more prominent. Right? Microsoft UI has officially entered the realm of self-parody.' (via Nelson)
design hci microsoft ui statistics user-hostile ribbon windows
The Daily Mail's frequent copyright abuse finally catches up with them : This is how you do it -- bravo to Alice Taylor, who got them fair and square as they did their usual trick of lifting copyrighted content without permission
copyright journalism photography daily-mail via:torrentfreak
Real-World Scala: Dependency Injection (DI) : I think I prefer the structural-typing approach, TBH
Bog Body: Committing to Open Source : Oisin Hurley on viable strategies for a commercial software company to handle participation in open source. Shame I've never found anywhere to viably put these into action, but they sound accurate
Fastest sort of fixed length 6 int array - Stack Overflow : huh, I'd never heard of sorting networks before
sorting-networks c algorithms sorting optimization sort stack-overflow
'What Idiot Wrote The Patent That Might Invalidate Software Patents? Oh, Wait, That Was Me' | Techdirt : 'So I was thinking - great they invalidated software patents, lets see what crappy patent written by an idiot they picked to do it - then I realized the idiot in question was me :-) Not sure how I feel about this. John - inventor of the patent in question.'
patents swpats reform usa software-development coding funny techdirt
good taxonomy of memcached use cases : via Jeff Barr's announcement of the Elasticache launch. from 2008, but a better taxonomy than I've seen elsewhere
memcached caching mysql performance scalability via:jeffbarr
Bootstrap, from Twitter : 'a toolkit from Twitter designed to kickstart development of webapps and sites'; 'includes base CSS and HTML for typography, forms, buttons, tables, grids, navigation, and more.' Very, very nice, AL2 licensed (via Mick Twomey)
via:micktwomey twitter ui css design html styling web-apps layout
echolibre & Orchestra : 'In particular, we want to thank the Irish and Dublin web communities. I've met other web communities in Europe and in the USA, and I can say, hand on heart, they don't have a patch on you guys.' awww ;) Congrats, guys
echolibre php web startups ireland paas orchestra.io engine-yard
Building with Legos : Netflix tech blog on how they deploy their services. Notably, they avoid the Puppet/Chef approach, citing these reasons: 'One is that it eliminates a number of dependencies in the production environment: a master control server, package repository and client scripts on the servers, network permissions to talk to all of these. Another is that it guarantees that what we test in the test environment is the EXACT same thing that is deployed in production; there is very little chance of configuration or other creep/bit rot. Finally, it means that there is no way for people to change or install things in the production environment (this may seem like a really harsh restriction, but if you can build a new AMI fast enough it doesn't really make a difference).'
Bog body found in Co Laois could be that of sacrificed king : 'All of the other bog bodies were found on significant boundaries. The idea is that because the goddess is the land, by inserting bodies and other items relating to their inauguration as king along the boundaries, it gives form to the goddess.' things were pretty damn gory back then
nifty Cydia iPhone signal-strength app : very nifty. pity my phone's not jailbroken, or I'd be trying this out
jailbreaking cydia iphone signal-strength 3g wireless mapping
Pavo Real : amazingly detailed peacock print by Argentinian artist Azul de Corso; very nice
Unbound: The Crowdfunding Cargo Cult – Telegraph Blogs : 'why was Unbound set up in the first place? It's because they constructed a cargo cult, believing that if they mimicked the superficial elements of successful crowdfunding, they could enjoy the same success as others – but perhaps even more, thanks to their relationships with publishers, agents, authors, and the media.' They're not the only Kickstarter-cargo-culting company, too. via waxy
via:waxy unbound kickstarter cargo-cult funding crowdfunding books uk
Arthur Recreates Scenes from Classic Movies : bored on maternity leave, this is what happens (via Niamh)
IT expenditure and failure – submission to public expenditure consultation : Antoin lays into the disfunctional Irish IT procurement system. "The status quo isn't just making things expensive and slow, it's asphyxiating the government's ability to serve."
antoin it procurement ireland government civil-service letters
Why we should expel the Vatican's Ambassador, the Papal Nuncio : 'In 2011, we have a new Government, who have stopped making excuses for the Vatican State. The Facebook campaign now has over 5,000 members, who continue to send emails and letters to their TDs and to the Minister for Foreign Affairs expressing the clear message that we want action. Enda Kenny said yesterday that the Vatican downplayed the rape and torture of Irish children to to uphold instead the primacy of the institution, its power, standing and 'reputation'. We should expel the Vatican's Papal Nuncio and send the message that they have destroyed the very things they prized the most.'
vatican papal-nuncio religion catholicism politics diplomacy ireland child-abuse cloyne-report
Paintings by Daniel Castan : really striking oil paintings of Manhattan; looks like small-format canvases can be bought at http://en.carredartistes.com/acheter-un-tableau-peintre-et-vente-peinture-artiste/petit-format/daniel-castan
daniel-castan art via:mlkshk paintings toget cities new-york nyc manhattan
Facebook group: Expel the Irish Papal Nuncio : "The Report of the Commission of Inquiry into Sexual Abuse in the Dublin ArchDiocese clearly states that the Papal Nuncio, the Diplomatic representative of the Vatican, refused to reply to investigators. Now the Cloynes Report has found that the Vatican's Ambassador intervened to undermine the efforts to protect children as recently as 1996. This is not acceptable. The Irish Government has the power to expel diplomats. If we are to take the reports' findings seriously, we must expel the Papal Nuncio."
vatican politics ireland abuse facebook lobbying cloyne-report
zen.org Communal Weblog » Bigger Than His Body : 'My beautiful, smart, funny, geeky, blue-eyed, bearded, amazing husband died last night with me and two of his aunts holding him.' So sorry for Elana and their boys -- Brendan was a nice guy and a great hacker :(
Death Notice Of Brendan KEHOE, Dun Laoghaire, Dublin, Ireland : 'Removal on Saturday from Quinn's of Glasthule to Mount Jerome Crematorium for service at 2pm. No flowers, please. Donations, if desired, to Educate Together.'
Tracking the Trackers: To Catch a History Thief | Stanford Center for Internet and Society : jaysus. the Epic Marketplace online ad network performs a history stealing attack to determine if the viewer has recently visited 'pages about getting pregnant and fertility, including at the Mayo Clinic'. very very scummy -- massive privacy violation (via Adam Shostack)
privacy history browsers history-stealing css attacks security via:adamshostack epic-marketplace nai ads
How we use Redis at Bump : via Simon Willison. some nice ideas here, particularly using a replication slave to handle the potentially latency-impacting disk writes in AOF mode
queueing redis nosql databases storage via:simonw replication bump
SSL perf tip : don't use Diffie-Hellman ciphers, they're slow
stud : 'a network proxy that terminates TLS/SSL connections and forwards the unencrypted traffic to some backend. It's designed to handle 10s of thousands of connections efficiently on multicore machines.'
an ex-Skype employee dishes the dirt on their buyout and acquisition : some incredible stories -- pretty mind-boggling stuff, I'm amazed people stuck around
PSI License [PDF] : the license under which the open data from various councils around Fingal and Dublin (see www.dublinked.ie) is being published
licensing open-data dublin fingal open public government county-councils city-council ireland
Great Hacker News thread on Andy Baio's "Kind Of Screwed" shakedown : full of good commentary on the rather horrific result. here's one: "I wonder how the photographer would feel if the company that manufactured the trumpet played by Miles Davis had claimed that his photograph violated the copyright of their "sculpture" and the tailor Miles got his suit from also protested. Of all art forms, photography has some of the least claim on being an entirely original creation of the artist."
photography miles-davis jay-maisel andy-baio waxy hn discussion copyright copyfight creativity art
_Scaling with MongoDB_, Michael Schurter 2011 [PDF] : presentation with some rather terrifying MongoDB war stories
F.B.I. Seizes Web Servers, Knocking Sites Offline : law enforcement fail. "the agents took entire server racks, perhaps because they mistakenly thought that "one enclosure is = to one server," [DigitalOne's CEO] said in an e-mail."
search-and-seizure law-enforcement fbi fail datacenters racks digitalone usa hosting
Hero orang-utan sparks copyright row - The Irish Times - Thu, Jun 16, 2011 : "They did not have the right to sell it and have infringed his copyright. It is as simple as that." Scummy -- some company called "News Team International" taking YouTube content and passing it off as their own
Hacker News | Ooops. : brilliant thread of epic "OMG WHAT HAVE I DONE" stories
64yourself : Damn. my 2006 hack http://taint.org/c64ize/ reinvented, although with a lot more panache :(
_Spotify: Large Scale, Low Latency, P2P Music-on-Demand Streaming_ : Gunnar Kreitz' paper on its innards! 'Spotify is a music streaming service offering lowlatency access to a library of over 8 million music tracks. Streaming is performed by a combination of client-server access and a peer-to-peer protocol. In this paper, we give an overview of the protocol and peer-to-peer architecture used and provide measurements of service performance and user behavior. The service currently has a user base of over 7 million and has been available in six European countries since October 2008. Data collected indicates that the combination of the client-server and peer-to-peer paradigms can be applied to music streaming with good results. In particular, 8.8% of music data played comes from Spotify's servers while the median playback latency is only 265 ms (including cached tracks). We also discuss the user access patterns observed and how the peer-to-peer network affects the access patterns as they reach the server.'
spotify via:waxy streaming p2p music architecture papers networking
Asciiflow : 'ASCII Flow Diagram Tool'. great web-based ASCII-art drawing app; create diagrams in your browser
Python Idioms and Efficiency Suggestions : will have to run this by our resident Pythonistas in work as a good set of guidelines
Scala: The Static Language that Feels Dynamic : a good intro from Bruce Eckel. We need a good excuse to deploy some Scala ;)
Redditor explains why Apple power cables break frequently : "As with any company, Apple consists of many divisions (Sales, Marketing, Customer Service, etc.) THE most powerful division at Apple is Industrial Design. For those of you unfamiliar with the term industrial design, this is the division that makes the decisions about the overall look and feel of Apple's products. And when I say "the most powerful", I mean that their decisions trump the decisions of any other division at Apple, including Engineering and Customer Service. Now it just so happens that the Industrial Design department HATES how a strain relief looks on a power adapter. They would much prefer to have a nice clean transition between the cable and the plug. Aesthetically, this does look nicer, but from an engineering point of view, it's pretty much committing reliability suicide. Because there is no strain relief, the cables fail at a very high rate because they get bent at very harsh angles. I'm sure that the Engineering division gave every reason in the world why a strain relief should be on an adapter cable, and Customer Service said how bad the customer experience would be if tons of adapters failed, but if industrial design doesn't like a strain relief, guess what, it gets removed."
France To Launch a National Patent Troll : 'The operation, called "France Brevets" will buy up patents from small operation and put the French government in charge of [...] shaking down companies for money.' I think the word is: incroyable
france fail omgwtfbbq patent-trolls swpats patents government innovation software europe
The first Irish case on defamation via autocomplete : Google Instant has picked up people searching for 'Ballymascanlon hotel receivership' and is now offering this as an autocomplete option -- cue defamation lawsuit. Defamation via machine learning
machine-learning defamation google google-instant search ballymascanlon hotels autocomplete law-enforcement
Data Protection Commissioner investigating Eircom's "three strikes" system : Eircom accused customers of piracy using systems that hadn't been updated for DST. 'this appears to show up ineptitude in relation to a very basic aspect of network management - i.e. making sure that the server clock reflects daylight savings time. As a result, it seems that users found themselves being accused on the basis of what somebody else did from the same IP address either an hour earlier or an hour later. Consequently, the users who were wrongfully accused should consider themselves lucky that this incompetence did not lead to their being accused of a serious crime - for example, being arrested and having their homes searched due to the wrong time being used.' As TJ explains, this could have very serious results
dpc ireland eircom fail time dst daylight-savings three-strikes filesharing piracy
Hipster Ipsum : 'Adipisicing do Tumblr fugiat vinyl Pitchfork. Organic tempor laboris, esse Tumblr irure eu nostrud. Dolor Cosby sweater mustache qui consequat incididunt. McSweeney's ullamco occaecat Wes Anderson. Minim aute lomo, duis ea proident enim Carles. Eiusmod culpa photo booth ex. Pariatur incididunt minim qui, dolor Pitchfork wayfarers mollit vinyl fixie.' (via boogah)
via:boogah hipster lorem-ipsum filler text markov-chains funny humour
Apple rips off student's rejected iPhone app : 'Wi-Fi Sync' was rejected from the App Store last May -- and a year later, iOS 5 is released with the same feature. what a coincidence! 'Hughes said Wi-Fi Sync was rejected from the iTunes App Store in May, 2010, one month after he submitted it. He said an iPhone developer relations representative named Steve Rea personally called him prior to sending a formal rejection email to say the app was admirable, but went on to explain there were unspecified security concerns and that it did things not specified in the official iPhone software developers' kit. "They did say that the iPhone engineering team had looked at it and were impressed," Hughes told El Reg. "They asked for my CV as well."'
apple walled-garden protectionism iphone wifi syncing apps ip rip-offs
Why Ryanair The Cookie Monster is just an urban myth : "If the price manipulation allegations were true, we would have expected to see price discrepancies in the results between Firefox and Chrome on day two. What we actually saw were exactly the same prices on both browsers."
Piracy: are we being conned? : The Age with a cynical take on pro-music-biz anti-piracy "reports". "The quality of data and analysis is very weak as its political objective is so clear. It does not use actual ABS data but data taken from Europe. It's an elemental statistical error, it's fudging with numbers to come out with a figure which is 'kinda sorta' plausible."
piracy filesharing copyright australia the-age newspapers ifpi acta
Our 256,000 (and counting) atheists, agnostics, humanists and non-religious - The Irish Times : "The 'non-religious' are the largest group in the State after Catholics, according to the [2006] census." Doubtless higher in this year's, too...
Dr. Neal Krawetz explains perceptual hashing : ie. TinEye and other "images like this one" search engines. nice explanation
algorithm images analysis programming dct hashing perceptual-hash tineye via:hn image
Telehack: May the command line live forever : 'Connected to TELEHACK port 13 / It is 8:16 am on Saturday, April 30, 2011 in Mountain View, California, USA. There are 10 local users. There are 24139 hosts on the network.' via Waxy
via:waxy simulation history telnet arpanet networking unix bbs
irishindoleaks : 'leaking the indo's offline wikileaks coverage online where it belongs' - scans of each article
irish-independent ireland politics wikileaks newspapers scans
_Tim Robinson: Connemara_ : a new documentary, based on the work of Tim Robinson, the great Connemara map-maker. showing this Sunday at 1pm at the IFI in Dublin
bump2babe - The Consumer Guide to Maternity Services in Ireland : wow, they've done a really good job on the statistics collation here
GTA4 Google Map : wow, very impressive -- as far as I can tell, it really _is_ using GMaps infrastructure to some degree
google-maps google maps gta4 grand-theft-auto via:nelson games
Gunnar Kreitz, _Spotify - Behind The Scenes_ : the innards of Spotify's client, server fleet, and P2P layer, from the dev team themselves. good stuff
Kill Screen - Profile: Bennett Foddy : The author of cult web-games QWOP and GIRP is a member of Cut Copy! crazy
"Treasure Map" - Threadless.com : great tee
Today in nose-leech news -- it's a species! : 'The T. rex leech uses its teeth to saw into the tissues of mammals' orifices, including eyes, urethras, rectums, and vaginas.' OH JESUS
nose nose-leech leeches nature horror omgwtf via:jwz nightmare parasites
The Secrets of Building Realtime Big Data Systems : great slides, via HN. recommends a canonical Hadoop long-term store and a quick, realtime, separate datastore for "not yet processed by Hadoop" data
hadoop big-data data scalability datamining realtime slides presentations
Javascript PC Emulator : truly incredible -- quite fast (about 386 speeds) under Chrome, even! from the HN comments: 'I just forkbombed my browser. Nothing is sacred anymore.' more comments at http://news.ycombinator.com/item?id=2555349
browser javascript linux emulation fabrice-bellard hacks amazing cool google-chrome x86
Graham Linehan and Osama Bin Laden : a master-stroke of Twitter-based media hoaxing, very funny. particularly like the assist from @Mike_FTW!
The Hargreaves Report : 'The publication of Digital Opportunity follows a six-month independent review of IP and Growth, led by Professor Ian Hargreaves. He was asked to consider how the national and international IP system can best work to promote innovation and growth.' Some fantastic recommendations here. I hope this provides clear direction to similar Irish efforts...
Netflix Beats BitTorrent's Bandwidth : 'For perhaps the first time in the internet's history, the largest percentage of the net's traffic is content that is paid for.' A great demo of how *good*, legit, for-pay services, can beat out less usable, dodgy, but free ones (via Waxy)
via:waxy piracy bandwidth bittorrent internet netflix filesharing
The MongoDB NoSQL Database Blog - MongoDB live at Craigslist : '>MongoDB is now live at Craigslist, where it is being used to archive [10TB] of [old posts]'. iiiinteresting
Worldtime Buddy : easy timezone conversion -- bookmarkable URLs, sensible levels of AJAX/JS, nicely done
How to make WIFI work at tech conferences : a success story from JSConf. great tips, I'm sure some will be practical at smaller scales ;)
wifi 802.11b 802.11n networking 802.11g conferences jsconf wireless
TwitPic assert ownership over images posted to it, signs licensing deal with sleb-photos agency : scummy. don't use TwitPic if they are planning to monetize your photos, even if it's currently just for a "small number of celebrities". (via my dad)
Cool, but obscure unix tools : these are great - some new ones on me!
Charanjit Singh on how he invented acid house ... by mistake : An interview (of sorts)! 'Cast your mind to the acid house scene and your immediate thought probably doesn't involve an ageing Bollywood session musician. Yet the softly spoken Indian man who greets me at the door of his friend's suburban Acton home on a sunny Sunday morning is credited with creating what some have labelled the first ever acid house record.'
acid-house music via:xxjfg guardian interviews history india bollywood ragas
Indian Street Graphics - a set on Flickr : a great collection (via Bruce Sterling)
TextAid - Google Chrome extension : "It's All Text" for Chrome. annoyingly, Chrome blocks forking of processes by extensions, so a daemon process (provided) needs to be running separately, but otherwise it works nicely. Particularly nice is that the daemon is just written in dependency-hell-free perl rather than Node.JS ;)
Chrome to get HTTPS public key pinning : 'Starting with Chrome 13, we'll have HTTPS pins for most Google properties. This means that certificate chains for, say, https://www.google.com, must include a whitelisted public key. It's a fatal error otherwise.' good anti-MITM protection
Online censorship now bordering on the ridiculous in Turkey - Reporters Without Borders : 'access to websites containing words on the list would in theory be suspended and it would be impossible to create new ones containing them. However, it is not clear how and to what extent the directive will be implemented in practice. The TIB could decide to suppress or block pages for just one blacklisted word. ... The list, which borders on the ridiculous, includes words such as "etek" (skirt), "baldiz" (sister-in-law) and "hayvan" (animals). It poses serious problems for access to online information. If words such as "free" and "pic" are censored, countless references to freedom and everyday photos will be eliminated from the Turkish Internet.' Incredible (via Danny)
via:mala repression internet turkey censorship filtering false-positives
Yahoo Sells Delicious To YouTube Founders : 'Q: Will AVOS maintain the Delicious service with all of its functionality A: Yes, that's the plan. There may be a time of adjustment as AVOS re-launches Delicious, but the company's intention is to add new features and grow the service overall.'
Linux Profiling tools and techniques : great tips for system-level and app-level profiling on Linux from Padraig
SmugMug's Don MacAskill on last week's EBS outage : comme il faut
LRB · James Meek · In the Sorting Office : 'The postwoman is paid a pittance to deliver corporate mail. She hasn't done her job well, yet so few people have complained about missed deliveries that she hasn't been found out. Across the world, postal services are being altered like this: optimised to deliver the maximum amount of unwanted mail at the minimum cost to businesses. In the internet age private citizens are sending less mail than they used to, but that's only part of the story of postal decline. The price of driving down the cost of bulk mailing for a handful of big organisations is being paid for by the replacement of decently paid postmen with casual labour and the erosion of daily deliveries.' (via Tony Finch)
via:fanf post mail postal-service holland dutch postmen work jobs business politics lrb
Amazon EC2 outage: summary and lessons learned : Rightscale CTO on last week's outage; pretty detailed, good round-up of useful commentary from around the web, too
Dropbox dedupe feature allows materialization of any file, if you know its hash : 'allows users to exploit Dropbox's file hashing scheme to copy files into their account without actually having them. Dropship will save the hashes of a file in JSON format. Anyone can then take these hashes and load the original file into their Dropbox account using Dropship.' heh. that sounds very familiar, I seem to recall thinking about this problem on several occasions... ;) Dropbox certainly didn't like it, going by this account
security filesharing dropbox online-backup online-storage p2p hashes sha dmca
Hacker News comments thread on the Dropbox dedupe bug : some good discussion on workarounds
DuoSecurity : well-packaged, well-designed, two-factor auth for SSH from Dug Song. free for small-scale use, too, it looks like. awesome! I've signed up (via Nelson)
via:nelson security authentication authorization two-factor-auth openssh ssh dug-song
Dylan Collins asks: has Ireland peaked as an Internet hub? : based on Twitter's surprise move passing over Dublin for London, and how to fix it: "launch the Internet Visa, an aggressive program that allows all Irish Internet companies to recruit from anywhere in the world. Reduce the red tape (combine all permit and visa documentation), guarantee a turnaround time measured in days (a small number) and avoid all the mistakes the UK has made with its Startup Visa initiative. Bring the talent from everywhere outside the EU to Ireland. Ireland doesn't scale organically. So it needs to in-source. We need to be honest about our shortcomings and tackle them with something which will make HR Directors smile. Imagine a country with all the existing advantages of Ireland plus the ability to hire anyone in the world you wanted. Who in their right mind wouldn't establish their European base there?" He's dead right, this is a massive problem for the Irish tech industry right now
ireland bureaucracy red-tape twitter tech business visas work government dylan-collins
Bug 647959 – Add Honest Achmed's root certificate to Mozilla : 'Honest Achmed is at least more honest than Comodo.' lol
comodo security security-theatre ssl tls certificates funny trust firefox
demerphq on "perl's regexps are slow" : His classic response to the Russ Cox DFA-over-NFA regular expressions paper. 'A general purpose regex engine like that required for perl has to be able to do a lot, and has to balance considerations ranging from memory footprint of a compiled object, construction time, flexibility, rich feature-sets, the ability to accomodate huge character sets, and of course most importantly matching performance. And it turns out that while DFA engines have a very good worst case match time, they dont actually have too many other redeeming features. Construction can be extremely slow, the memory footprint vast, all kinds of trickery is involved to do unicode or capturing properly and they aren't suitable for patterns with backreferences.' -- Also interesting to note that he mentions an approach I've used in several SpamAssassin speedup add-ons, too ;)
performance perl regular-expressions perlmonks demerphq regexps dfa nfa state-machines
This sounds very cool! Nice one, hackerspace ppl.
Ireland's Hackerspaces and Makerspaces (091 Labs - Galway, Belfast Hackerspace, MilkLabs - Limerick, Nexus Cork and TOG - Dublin) have been asked to build and man a temporary hackerspace during the MindField - International Festival of Ideas (http://www.mindfield.ie/). MindField will take place over the weekend of 29 April - 1 May in Merrion Square.
During MindField our temporary hackerspace will provide a range of events where festival participants can learn about diybio, 3D printing, basic electronics and micro controllers, electronic fashion/crafting and open data. These events are included in the festival schedule (http://mindfield.ie/festival-schedul/).
In parallel with these events we have an opportunity run a Hardware Hacking Challenge. In this challenge we will try to engage a group of willing hacker, makers and festival participants in the challenge to create or construct interesting or innovative projects out of recycled hardware. We are trying to source interesting materials, electronic devices or equipment that can be used to based projects off or as sources of components.
We are particularly interested in devices that contain various types of transducers which can then be hooked up to micro controllers and computers. We're not looking for normal computer equipment or servers we've got lots of that, but more unusual stuff that people have lying around.
If you think you've got something they might like, contact Robert Fitzsimons.
diy dublin electronics hackerspaces hacking ireland mindfield
brandnewretro | scans from the past : a mate of mine, scanning Irish cultural artifacts from Ireland in the '70s and '80s. fanzines!
fanzines irish ireland history 1980s 1970s dundalk culture scans
Pound : 'a reverse proxy, load balancer and HTTPS front-end for Web server(s). Pound was developed to enable distributing the load among several Web-servers and to allow for a convenient SSL wrapper for those Web servers that do not offer it natively. Pound is distributed under the GPL'
What is Facebook's architecture? - Quora : nicely detailed summary
How Little Sleep Can You Get Away With? : 'after just a few days, the four- and six-hour group reported that, yes, they were slightly sleepy. But they insisted they had adjusted to their new state. Even 14 days into the study, they said sleepiness was not affecting them. In fact, their performance had tanked. In other words, the sleep-deprived among us are lousy judges of our own sleep needs. We are not nearly as sharp as we think we are.'
Permanent TSB's tracker-mortgage paydown option isn't such a good deal after all : 'it might be in your interest if you have a tracker mortgage and are unable to get a better rate of interest on the €5,000 that you are being tempted to repay PTSB. You can get up to 4.2% from PTSB deposit accounts, 9.7% from 10-year Irish sovereign bonds, 9% from residential property. Yet PTSB is prepared to give you less than a measly 2% over a five year period on your €5,000 repayment.'
ptsb permanent-tsb finance money mortgages tracker-mortgage investment
This Bacteria is Violating Copyright | tor.com | Science fiction and fantasy | Blog posts : the Joyce estate playing their usual role. 'are we now nearing a point where copyright law can result in the retraction of a life form?' (via John Looney)
copyright dna bacteria james-joyce joyce-estate frivolous lawsuits copyfight craig-venter
Virgin and NTL filtering fail : 'Virgin and NTL [in the UK] blocked [del.icio.us] for years' due to a false positive -- joshua
del.icio.us false-positives filtering uk isps virgin ntl fail via:hackernews
Nelson Minar on map tiles : quite a lot of detail into the prevailing state of the art in how online zoomable maps store their tiles
pyflakes.vim - on-the-fly Python code checking in Vim : Vim gets a good IDE feature. 'highlights common Python errors like misspelling a variable name on the fly. It also warns about unused imports, redefined functions, etc.'
Rumor: Google "Disgusted" With Record Labels : 'Once again, Warner is the fly in the ointment, the same company that praises Spotify one day, renews their licenses for the rest of the world and then the next day doesn't want to license them in the US.'
google music cloud licensing music-industry record-labels warner-music streaming
Dublin - Europe's Next Startup Petri Dish? - NYTimes.com : 'Ireland's tech scene continues to expand in spite of the woeful state of the rest of the economy with a plethora of accelerator programs, seed funds and events like Founders and the IBM smartcamp global finals happening there in the last year or two. '
Digital Rights Ireland » Garda plans to introduce web blocking in Ireland : 'Last year we revealed that the Department of Justice was working on secret plans to introduce internet filtering in Ireland. Now, despite a complete lack of any legislation, public consultation or democratic discussion, these plans have moved to the implementation stage.' wtf, this is just appalling lack of oversight
gardai blocking filtering ireland politics legislation oversight isps ispai alto censorship eff
Daragh O'Brien on the Gardai's plans to force ISPs to implement IP filtering : 'Internet blocking is ineffective. The current proposal lacks sufficient checks and balances, and may even require ISPs and telcos to break other laws to comply. It will inevitably result in innocents being tarred as offenders. Data Protection principles (such as "Adequate, Relevant, and Not Excessive" are being blatantly ignored to implement an ineffective solution. Far better is to shut down the shop by removing the images at source and invest time, energy, and resources into a more transparent effort to manage this issue.' well said
internet filtering censorship blocking gardai isps ireland data-protection privacy
RIM: The inmates have taken over the asylum : some notes from Blackberry's slow circle round the toilet, as it's hammered by iPhones and Androids. also: I can't believe QNX is still alive
Improving Linux performance by preserving Buffer Cache State : handy -- a patch to rsync(1) which will not disturb the buffer cache, so that large file transfers and backups will not interfere with what's been cached previously
performance linux caching buffer-cache rsync io cache patches backups
ImperialViolet - Revocation doesn't work : OCSP doesn't work -- the browser vendors have failed to implement it safely
L. MULLIGAN. GROCER.: Beer of the Week: Metalman Pale Ale : *excellent* Irish pale ale, brewed by ex-co-worker Grainne and her partner Tim, now on sale in my favourite pub. yay!
TomatoUSB : 'an alternative Linux-based firmware for powering Broadcom-based ethernet routers. It is a modification of the famous Tomato firmware, with additional built-in support for USB port, wireless-N mode support, support for several newer router models, and various enhancements. Tomato USB supports many Broadcom-based routers from Asus, Linksys, Buffalo, Netgear and other manufacturers.' Looks good -- I've been a Tomato fan for many years -- and jzawodny-approved
Detecting Certificate Authority compromises and web browser collusion | The Tor Blog : 'If I had to make a bet, I'd wager that an attacker was able to issue high value [SSL] certificates, probably by compromising [the USERTRUST SSL certificate authority] in some manner, this was discovered sometime before the revocation date, each certificate was revoked, the vendors notified, the patches were written, and binary builds kicked off - end users are probably still updating and thus many people are vulnerable to the failure that is the CRL and OCSP method for revocation.' It seems addons.mozilla.org was one of the bogus certs acquired. Major ouch. Thanks to EFF/Tor et al for investigating this -- SSL cert revocation is a shambles
security ssl tls certificates ca revocation crypto exploits eff tor comodo usertrust
I'm uncomfortable voting for David Norris for President. Here's why.
In November last year, he was a key voice in a Senate debate on the topic of "Protection of Intellectual Property Rights", where he quoted heavily from the flawed judgement by Mr. Justice Peter Charleton in the Warner, Universal, Sony BMG and EMI vs UPC case. (There are allegations that he called the debate after speaking to Paul McGuinness (U2's manager) and Niall Stokes (of Hot Press).)
In the debate, Norris quotes Mr Justice Charleton, saying:
'In failing to provide legislative provision for blocking, diverting and interrupting internet copyright theft, Ireland is not yet fully in compliance with its obligations under European law.' Norris then says: 'Irish law could be brought into alignment with the intention of the European directive through a simple statutory instrument.' [1]
Now, let me clarify my position -- I'm in favour of some means of resolving the level of piracy of music and movies which is widespread nowadays, and I believe there's a mutually agreeable way to do this. But what Norris and Mr Justice Charleton propose is not it. Here are the problems as I see them.
It Lets The Internet Filtering Genie Out Of The Bottle
The big one.
The problem is that any infrastructure for 'blocking, diverting and interrupting internet copyright theft' is effectively infrastructure for 'blocking, diverting and interrupting' any communication on the net. We have to be very careful about how this is permitted, as it'll very quickly suffer "feature creep" and become a general-purpose censorship system -- the Great Firewall Of Ireland. As Damien Mulley put it:
'first they'll start with the Pirate Bay. Then comes Mininova, IsoHunt, then comes YouTube (they have dodgy stuff, right?), how long before we have Boards.ie because someone quoted a newspaper article or a section of a book? And don't think they'll stop there too, any site that links to The Pirate Bay and the others on the hate list will probably be added to the list too...'
In Australia, the anti-child-porn filtering system was quickly used to block gambling websites, gay and straight porn sites, political parties, Wikipedia entries, Christian sites, Wikileaks, and a dentist; in Thailand, a similar system was used to block criticism of the royal family.
Will It Help? I Don't Think So
Norris:
'As long as Irish law is deficient, Mr. Justice Charleton has found that all creative Irish industries are losing money.'
This is quite a hilariously overblown and sweeping statement. ALL creative Irish industries? What qualifies as a 'creative' industry? I suspect some in this country have been involved in industrial acts of creation that made money. ;)
While they're not Irish, the well-known indie label Beggar's Banquet has gone on the record as stating the opposite where the current music situation is concerned --
"There's fewer gatekeepers now. We don't have to knock on a TV station's door or a radio station's door and it's made us far more competitive. [...] There's a wide highway in front of us we can go speeding down, and it wasn't there even two years ago. It means the majors are looking at a world where only 35 Gold Albums a year are certified compared to ten times that recently. But going above Gold in the US is not a problem for us."
So it appears a 'creative' industry (albeit in the UK) is finding things not quite so bad.
Norris again:
'the facts were established in the judgment of Mr. Justice Charleton in which he stated: "Between 2005 and 2009 the recording companies experienced a reduction of 40% in the Irish market for the legal sale of recorded music." That is a devastating blow. [...] He went on to state: "Some 675,000 people are likely to be engaged in some form of illegal downloading from time to time."'
Without quite lining up one statement with the other, this reinforces the impression that the only reason the recording companies have seen these drops in revenues is due to internet-borne piracy. However, quoting the brilliant Mumblin' Deaf Ro on the topic of lies, damn lies, and music biz statistics:
'The drop in the value of Irish retail music sales was 11.7% between 2008 and 2009, which is significantly less than the 18% overall drop in retail sales for the economy that year. Digital album sales have increased by 30% since 2007 both in terms of volume and market value.'
So in other words, between 2008 and 2009, Irish retail music sales outperformed the retail sales economy as a whole!
In addition, Ro provides the following BPI figures for UK market volumes over the 2005-2009 period:
Year Albums Singles
2005 159.0m 47.9m
2006 154.7m 66.9m
2007 138.1m 86.6m
2008 133.6m 115.1m
2009 128.9m 152.7m
It's clear that singles sales went through the roof, more than tripling. Album sales did drop however, but nowhere near by 40% -- and this coincided with the general drop in the prevailing global economy around that time. He also notes that digital sales in the UK went through the roof globally on a number of metrics in 2009.
While this does not provide figures for the Irish market, I'm at a loss as to how it could be radically different -- Irish and UK consumers have pretty similar musical tastes and consumption habits, I would guess.
Here's a theory: perhaps the issue could be that "Irish" music sales are associated with bricks-and-mortar music shops selling the physical product, whereas digital music sales are associated with online services based outside Ireland, and an Irish buyer buying an album at 7digital.co.uk, or on iTunes, isn't counted as an "Irish retail sale"? Could the problem be that we don't have any significant Irish shops selling music online, I wonder?
Bricks-and-mortar music shops, such as ex-Senator Donie Cassidy's "Celtic Note" (who coincidentally was quite vociferous in that Seanad debate), are indeed hurting in this new model of music consumption -- and that's a problem. But given that good, working digital music sales systems are in operation, it doesn't necessarily appear to be due to massive volumes of internet-borne piracy, going by these figures.
Essentially, internet piracy is a convenient bogeyman, especially for the technophobic old guard, but may have little bearing on the current woes of the Irish record industry and bricks-and-mortar music shops.
(Update: a couple of days after this was posted, a pair of economists at the LSE have said basically the same thing.)
Audible Magic Won't Work For Long Anyway
Audible Magic, which Norris suggests is IRMA's favoured filtering system, received the following verdict from the EFF back in 2004:
'Should Audible Magic's technology be widely adopted, it is likely that P2P file-sharing applications would be revised to implement encryption. Accordingly, network administrators will want to ask Audible Magic tough questions before investing in the company's technology, lest the investment be rendered worthless by the next P2P "upgrade."'
Naturally, encryption is widespread nowadays, so this may already be the case.
Internet Censorship Harms Our Global Image
'do we really want to send out the message that, digitally, we're the new France? Come to think of it, do we want to tell Google, Facebook, Apple and Twitter that, digitally, we're the new Britain?'
Right now, more than ever, we need to put out an image that we're ready to do business on our end of the internet. Mandatory censorship systems don't exactly support this.
In Summary
So in summary, I would hope to see a more balanced approach to the issue from Norris. Most of the problematic statements in his speech were directly sourced from Mr. Justice Charleton's flawed judgement, but some critical thinking would be vital, I would have thought. The fact that this was lacking, particularly given the allegations of heavy music-biz lobbying beforehand, leaves me feeling less inclined to vote for him than I would have been before, particularly since I haven't heard any clarification on these issues.
([1]: Funnily enough, an SI similar to this was nearly sneaked through a couple of weeks ago, according to reports.)
ireland filesharing politics filtering internet law irma piracy
HBGary planned to "BLOW THE BALLS OFF OF NMAP" : 'I would like to call it "B.E.S.T. Scanner" so people kind of get stuck calling it "the best scanner". We can figure out what BEST means later.' omgwtf. Is this guy 12 years old?
funny security humor anonymous scanner nmap hbgary open-source fail idiots
Ireland's new coalition on media, IT & IP law | Lex Ferenda : 'some first thoughts on how the just-published coalition agreement (Fine Gael and Labour) in Ireland proposes to deal with issues of interest to cyberlaw and media law.'
lex-ferenda law ireland ip content internet fair-use copyright tv
Nuclear energy: Inside the black box : What's going on inside the Fukushima nuclear reactor, and how it is hoped meltdown can be averted
nuclear-power meltdown disasters japan fukushima power electricity nuclear
Backdoor legislation is no way to tackle thorny issue of copyright - The Irish Times - Fri, Mar 11, 2011 : good article by Karlin Lillington on the attempted sneaking-through of an SI to 'deal with' filesharing. agreed on all counts
filesharing piracy ireland law karlin-lillington legislation fianna-fail
The Remarkable Notability Of Old Man Murray | Rock, Paper, Shotgun : wow, the *entire games industry* (basically) comes out to praise Old Man Murray -- the influential satire site. I'd forgotten about their Time-To-Crate game rating system (which I still apply)
Old Man Murray Deleted From Wikipedia | Rock, Paper, Shotgun : more idiotic deletionism from Wikipedia. when will someone fork WP with a saner community?
wikipedia deletionpedia deletion gaming history old-man-murray web community asshats
The Telegraph - Calcutta (Kolkata) | Frontpage | A lesson for colony cousin : So much win in one article. (a) the Bengali equivalent of "craic" is, roughly, "phatiphati"; (b) "In Irish pubs, amid the tiddly-doo music, this is a craic"; (c) wtf Gadaffi references; (d) shared post-colonialist glee
craic funny colonialism bengali phatiphati tiddly-doo libya india cricket
O2's page on their new "block 18+ content on mobile internet" policy : O2 UK have just instituted a mandatory block for all "18+" content, which is only removed once the customer pays a UKP1 fee via credit card (which is immediately refunded). Twitter is *full* of angry UK O2 users right now
TechWire: Don't do it, Enda and Eamon : Adrian Weckler with a plea for the incoming govt regarding the attempt to rush through '3 Strikes' by the outgoing one: 'Such a law will have absolutely no effect on the practice of illegal filesharing. None. Zero. It hasn't worked in France. It hasn't worked in Britain. And it certainly won't work in Ireland. On the other hand, it may well send a signal to huge, jobs-creating digital IT companies that Ireland is a place that tries to legislate away personal digital freedoms.'
3-strikes ireland adrian-weckler politics filesharing piracy filtering internet freedom
Notch on piracy: "if a pirated game is a lost sale, should bad reviews be illegal?" | PC Gamer : wish more "piracy = theft" people would think about this viewpoint. mind you, fwiw, I buy my games, and have paid for Minecraft ;)
UK Government Agency wants your spam, but filters the submission address : doh (via Graham Cluley)
BikeDroid : Warren's Android app to track DublinBike availability: 'Use BikeDroid to locate the nearest free bike or stand to you. Get real-time status of all bike stands displayed on a map of your city.'
How to block retweets : in Twitter, obvs. This is incredibly handy, and very poorly-documented
BallotBox.ie Posts Emigrant-Vote Results : FG 63 seats, Labour 51, SF 23, Greens 10, Ind 11, FF 2. interesting to see SF's strong showing among emigrants -- something for electoral reformers to think about ;)
Dublin Bikes 2 Go! : 'an [unofficial] mobile web application that the public can use to find 'Dublin Bikes' stations and information about bike availability'
Frank Zappa proposed EMusic in 1983 : incredlble -- way ahead of his time on this one
It's pretty common for apps to require "configuration" -- external files which can contain settings to customise their behaviour. Ideally, apps shouldn't require configuration, and this is always a good aim. But in some situations, it's unavoidable.
In the abstract, it may seem attractive to use a fully-fledged programming language as the language to express configuration in. However, I think this is not a good idea. Here are some reasons why configuration files should not be expressed in a programming language (and yes, I include "Ruby without parentheses" in that bucket):
Provability
If a configuration language is Turing-incomplete, configuration files written in it can be validated "offline", ie. without executing the program it configures. All programming languages are, by definition, Turing-complete, meaning that the program must be executed in full before its configuration can be considered valid.
Offline validation is a useful feature for operational usability, as we've found with "spamassassin --lint".
Security
Some configuration settings may be insecure in certain circumstances; for example, in SpamAssassin, we allow certain classes of settings like whitelist/blacklists to be set in a users ~/.spamassassin/user_prefs file, while disallowing rule definitions (which can cause poor performance if poorly written).
If your configuration file is simply an evaluated chunk of code, it becomes more difficult to protect against an attacker introspecting the interpreter and overriding the security limitations. It's not impossible, since you can, for instance, use a sandboxed interpreter, but this is typically not particularly easy to implement.
Usability
Here's a rather hairy configuration file I've concocted.
#! /usr/bin/somelanguage
!$ app.status load html
!c = []
;c['sources'] = < >
;c['sources'].append(
NewConfigurationThingy("foo_bar",
baz="flargle"))
;c['builders'] = < >
;c['bots'] = < >
!$ app.steps load source, shell
;bf_mc_generic = factory.SomethingFactory( <
woo(source.SVN, svnurl="http://example.com/foo/bar"),
woo(shell.Configure, command="/bar/baz start"),
woo(shell.Test, command="/bar/baz test"),
woo(shell.Configure, command="/bar/baz stop")
> );
;b1 = < "name": "mc-fast", "slavename": "mc-fast",
"builddir": "mc-fast", "factory": ;bf_mc_generic >
;c['builders'].append(;b1)
;SomethingOrOther = ;c
This isn't actually entirely concocted from thin air -- it's actually bits of our BuildBot configuration file, from before we switched to using Hudson. I've replaced the familiar Python syntax with deliberately-unfamiliar made-up syntax, to emulate the user experience I had attempting to configure BuildBot with no pre-existing Python knowledge. ;)
Compare with this re-stating of the same configuration data in a simplified, "configuration-oriented" imaginary DSL:
add_source NewConfigurationThingy foo_bar baz=flargle
buildfactory bf_mc_generic source.SVN http://example.com/foo/bar
buildfactory bf_mc_generic shell.Configure /bar/baz start
buildfactory bf_mc_generic shell.Test /bar/baz test
buildfactory bf_mc_generic shell.Configure /bar/baz stop
add_builder name=mc-fast slavename=mc-fast
builddir=mc-fast factory=bf_mc_generic
Essentially, I've extracted the useful configuration data from the hairy example, discarded the symbology used to indicate types, function calls, data structure construction, and let the configuration domain knowledge imply what's necessary. Not only is this easier to comprehend for the casual reader, it also reduces the risk of syntax errors, by simply minimising the number of syntactical components.
See Also
The Wikipedia page on DSLs is quite good on the topic, with a succinct list of pros and cons.
This StackOverflow thread has some good comments -- I particularly like this point:
When you need your application to be very "configurable" in ways that you cannot imagine today, then what you really need is a plugins system. You need to develop your application in a way that someone else can code a new plugin and hook it into your application in the future.
+1.
This seems to be a controversial topic -- as you can see, that page has people on both sides of the issue. Maybe it fundamentally comes down to a matter of taste. Anyway -- my $.02.
Update: discussions elsewhere: HackerNews
Another Update, 2012-04-06: Robey Pointer wrote a post called Why Config?, in which he describes a Scala-based configuration language in use at Twitter, which uses Scala's runtime code evaluation, and a Scala trait, to express configuration succinctly in a Scala source file and load it at runtime. The downside? It's a Scala source file, executed at runtime, containing configuration. :(
However, this comment in the comments section is worth a read:
At Netli (now part of Akamai) we had a configuration framework very similar in spirit and appearance to Configgy. It was in early 2000-s, we open sourced it since. (http://ncnf.sourceforge.net/). It would provide on-the-fly reload for the C-based programs (the ncnf if a C library). It also had some perks like attribute inheritance and a concept of block references. Most importantly though, it contained a separate schema language and a validator to allow configuration be checked before pushing in production. At Netli we used it to configure 1200 services on over 400 hardware boxes, the configuration becoming about 20+mb in length (assembled from several pieces by the CPP, then M4 templating library).
Naturally, it wasn't Netli's first attempt at doing configuration. One of the first attempts failed since it was Turing-complete. That approach was to specify the configuration as a Perl data specification. In a very short time the lure of unused expressiveness of such Turing-complete environment prevailed and people started to write for-loops around data pieces and doing other tricks to remove redundancy from the configuration. It turned out to be a disaster in the end, with configuration becoming unmaintainable and flaky.
One principle I got out out of that exercise is that configuration shall not be Turing-complete. We've got burned specifically by that property far too many times. Yet I do agree with you that a validation facility is a must-have, which is something not usually part of the simple text-based frameworks. C-based NCNF had it almost from the very beginning though, and it proved to be a very useful harness.
+1. There's lots more info on that system at this post at lionet.livejournal.com.
Another Update, 2017-05-09: casio_juarez on Twitter:
Dev: I'll use a declarative language for config this time.
— 0x0DEADA55 (@casio_juarez) May 8, 2017
6 months later: Let's add variables.
12 mos: And conditionals.
18 mos: Fuck.
Also related: The Configuration Complexity Clock.
(Image credit: Turn The Dial by VERY URGENT Photography)
Tom Morris - Request for comment: a 'Good API' checklist and committee : Sane suggestions for good HTTP APIs
Votomatic : Brilliant! "find out which political parties are compatible with you." The app asks a few questions, you furnish survey-style responses, and it figures out which party is closest in published policy. It works quite well, determining that my optimum is Labour (correct)
How a Remote Town in Romania Has Become Cybercrime Central | Magazine : the story of Ramnicu Valcea -- Romania's Silicon Valley of phishing
U.S. Government Shuts Down 84,000 Websites, 'By Mistake' | TorrentFreak : DHS/ICE domain seizures suffer a serious false positive problem, resulting in the seizure and shutting down of 84,000 subdomains of a free DNS provider, replacing them with a banner accusing the site of trafficking in child porn. whoops!
dhs ice censorship internet domains dns seizure false-positives child-porn
Israeli general claims Stuxnet attacks as one of his successes : 'Haaretz reports [on a] video that was played at a party organized for General Gabi Ashkenazi's last day on the job. The video contained references to the successes he achieved during his stint as chief of staff, [including] the Stuxnet worm attack on Iran's uranium enrichment facility at Natanz and and the nuclear reactor at Bushehr.'
Gerrit, Git and Jenkins : This is the future of code review. Commit directly from your git checkout to the Gerrit code-review system; change is immediately web-visible and enters the review workflow; at the same time, Jenkins checks out the proposed change and runs the test suite; once it's approved, it automatically gets checked in. Brilliant!
git coding code-review workflows jenkins gerrit c-i testing automation
FareBot: Read data from public transit cards with your NFC-equipped Android phone - codebutler : 'When demonstrating FareBot, many people are surprised to learn that much of the data on their ORCA card is not encrypted or protected. This fact is published by ORCA, but is not commonly known and may be of concern to some people who would rather not broadcast where they've been to anyone who can brush against the outside of their wallet. Transit agencies across the board should do a better job explaining to riders how the cards work and what the privacy implications are.' (via Boing Boing)
via:boingboing privacy android rfid security transit mobile encryption mifare desfire farebot
Storymap : great UI for a little Dublin oral-history site -- just a GMaps mashup with links to YouTube, but it works very well
dublin ireland storymap stories oral-history people google-maps mashups youtube video
Spotify Second Largest Source Of Revenue In Europe For Labels : wow. the WinAmp guys were right -- 'on a European level, Spotify is the second single largest source of revenue for record labels. This means that 2010 saw dramatic increase in its usage as well as payouts to record labels and artists themselves.' this via an IFPI report
ifpi music spotify streaming revenue record-labels europe sweden isps mp3
Zero stroke - Wikipedia, the free encyclopedia : 'With the price of bread running into billions a loaf the German people [...] had to get used to counting in thousands of billions. This, according to some German physicians, brought on a new nervous disease known as "zero stroke," or "cipher stroke" [...] The persons afflicted with the malady are perfectly normal, except "for a desire to write endless rows of ciphers and engage in computations more involved than the most difficult problems in logarithms."' (via Joe Drumgoole)
germany zero hyperinflation inflation via:jdrumgoole money brain mental-illness
Fine Gael's Facebook spam campaign : jesus. Not only do they coin the cramp-inducing neologism "twolicy", they then have the temerity to suggest that people should "donate" their Facebook status so that FG can spam their social group. awful
No Sleep 'Til Brooklands: A True Story Of Daily Mail Lies (guest post) : how the Daily Mail (UK) works, via b3ta. mind-boggling misuse of one woman's comments to concoct a story, according to this
daily-mail journalism libel media newspapers law uk via:b3ta
Using Git to manage a web site : simple, basic demo of a git post-receive hook to auto-check-out every rev committed to a git repository
If you visit the Irish Times at all frequently, you'll probably have noticed a nifty "wisdom of crowds" feature in the right sidebar: the list of "most read" articles. It's quite good, since they're often very interesting articles. Unfortunately, there's no RSS feed for this feature.
Well, now there is:
Gamasutra - News - Opinion: Minecraft And The Question Of Luck : 'Notch's luck was that he came across the idea of doing a first-person fortress building game. His alignment was that the game that he wanted to make was culturally connected to [he PC gamer] tribe. While the game may appear ugly, and its purchase process etc seem naive to many a gaming professional, all of those decisions that Notch made along the road to releasing his game were from the point of view of a particular perspective of what games are, what matters and what were the things that he could trust the tribe to figure out for themselves.'
tribes viral minecraft gaming analysis games culture gamasutra via:nelson future software marketing
Spamwiki : good wiki tracking spam operations, their current campaigns, who's doing it etc.
Spammers Are Now Using Verified By Visa : Visa's atrociously-designed "security" program is now being used by criminals to process their credit-card payments, allegedly
Michael "Liar's Poker" Lewis on Ireland's economic collapse : PDF of the 15-page Vanity Fair article -- from interviews I've read in advance, this seems pretty good
michael-lewis vanity-fair articles pdf toread economy ireland disaster collapse
Dublin bikes revisited : Fantastic comparative number crunching on the JC Decaux Dublin Bikes scheme, compared to their other European cities (Brussels, Lyons, Paris, Seville), times of day, busiest stations, rainfall, etc.
bikes dublin-bikes cycling dublin ireland jc-decaux number-crunching analysis statistics
Wired: how a Toronto statistician cracked the state lottery : 'The tic-tac-toe lottery was seriously flawed. It took a few hours of studying his tickets and some statistical sleuthing, but he discovered a defect in the game: The visible numbers turned out to reveal essential information about the digits hidden under the latex coating. Nothing needed to be scratched off—the ticket could be cracked if you knew the secret code.'
toronto hacks money statistics probability wired tic-tac-toe singleton
Google: Bing Is Cheating, Copying Our Search Results : laaaame, Microsoft
Java Hangs When Converting 2.2250738585072012e-308 : ie. the same value as the PHP bug. 'Konstantin [Pressier] reported this problem to Oracle three weeks ago, but is still waiting for a reply.' good job, Oracle!
Keeping Track of Electioneering | Election Leaflets 2011 : 'See or post leaflets shoved through your door by parties and candidates across the land. RSS feeds and email alerts available by constituency. Add new leaflets through a web form or by email.'
election leaflets pamphlets ge11 ireland politics fianna-fail
Data Protection Commissioner warns the parties not to spam in advance of the coming election : Any teeth though?
The worst week for the worst Taoiseach in the State's history : incredible insider account of Cowen's final ineptitudes as FF leader. Beyond GUBU
gubu funny inept ireland brian-cowen fianna-fail dail crazy politics
where Fine Gael got their new poster source images : "Google Image 'People' = Ethnic Diversity". bwahahahaha
funny fg fine-gael inept design lame google stock-photos people enda-kenny boards
gist: 782263 - How to redirect a running process' output to a file and logout : a nifty gdb hack; essentially dup()s a couple of files in /tmp in place of fd 1 and 2, then uses the bashism "detach" to nohup the running process
gdb hacks linux process shell unix via:hn nifty dup detach bash
apenwarr/sshuttle - GitHub : 'Any TCP session you initiate to one of the proxied IP addresses [specified on the command line] will be captured by sshuttle and sent over an ssh session to the remote copy of sshuttle, which will then regenerate the connection on that end, and funnel the data back and forth through ssh. Fun, right? A poor man's instant VPN, and you don't even have to have admin access on the server.'
vpn ssh security linux opensource tcp networking tunnelling port-forwarding
Why djb redo won't be the Git of build systems : A counter-argument: "so, redo, from a conceptual point of view, has a really good and simple approach (very djb-y), and I'm sure it's an excellent tool for new projects, but for existing projects that already use make in a non-recursive fashion, it would a maintenance PITA. And that's why I conclude that redo in its current conceptual state will never be the Git of build systems. make is still more flexible, and even though it has its flaws, it's still good enough for most people, and also a de-facto standard."
The things make got right (and how to make it better) : jgc provides a good demonstration of how a general-purpose programming language tends to make a crappy DSL -- specifically Rakefiles
dsl build make coding jgc languages configuration makefiles rake ruby
good Hacker News thread on djb's "redo" : YA make-replacement build system. the thread is better than the linked article, btw
hacker-news via:fanf make build djb redo compilation building coding open-source
Hudson's future : renaming to "Jenkins" due to Oracle asshattery
Bicycle Safety: How to Not Get Hit by Cars : really quite good advice -- all except for "take the whole lane", which in my experience aggravates drivers and causes road rage and risky behaviour. avoid
tvrenamer.pl : Another TV file-renaming script. looks a little fragile/hacky at a glance though
Boxee TV Show Download Automation : organise downloaded TV shows into the directory format Boxee (and by extension, XBMC) wants; some votes for Sickbeard here
Reset Ireland : 'Greater access to information on the workings of government empower the public in making informed decisions on the direction they want the country to take. Open Government and media reforms can make this a reality. The key inspiration for Reset Ireland comes from projects such as the Icelandic Modern Media Initiative along with various other projects promoting openness, transparency and accountability in Government around the world.'
Rules of SCRAM : 'GOATS just stand around during this phase and stare at each other, rolling their eyes frequently at howlers (such as using serialization to SOAP for storage, or databases as RPC mechanisms). It is often useful for GOATS — or anybody, really — to take notes for the monthly BACKSTABBING drill.'
Tunisian government harvesting usernames and passwords : injects JS onto Google, Facebook, Yahoo! non-encrypted login pages to submit the typed username and password against nonexistent http URLs, e.g. 'http://www.google.com/wo0dh3ad', presumably so that DPI logging can collect them. apparently the HTTPS login pages are blocked to force use of HTTP
tunisia via:pjakma security snooping surveillance https javascript
Hacker Culture: A Response to Bruce Sterling on WikiLeaks : good article from Gabriella Coleman in The Atlantic
hackers bruce-sterling wikileaks julian-assange politics blackhat hacking