Skip to content

Justin's Linklog Posts

Links for 2013-12-04

Published December 4, 2013

  • wrk

    a modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU. It combines a multithreaded design with scalable event notification systems such as epoll and kqueue.  An optional LuaJIT script can perform HTTP request generation, response processing, and custom reporting.
    Written in C, ASL2 licensed.

    (tags: wrk benchmarking http performance testing lua load-testing load-generation)

  • Removing DRM Boosts Music Sales by 10%

    Based on a working paper from University of Toronto researcher Laurina Zhang

    Comparing album sales of four major labels before and after the removal of DRM reveals that digital music revenue increases by 10% when restrictions are removed. The effect goes up to 30% for long tail content, while top-selling albums show no significant jump. The findings suggest that dropping technical restrictions can benefit both artists and the major labels.
    more details: http://inside.rotman.utoronto.ca/laurinazhang/files/2013/11/laurina_zhang_jmp_nov4.pdf , “Intellectual Property Strategy and the Long Tail: Evidence from the Recorded Music Industry”, Laurina Zhang, November 4, 2013

    (tags: ip copyright drm mp3 music laurina-zhang research long-tail albums rights-management piracy)

  • 100 Years of Breed “Improvement” | Science of Dogs

    The English bulldog has come to symbolize all that is wrong with the dog fancy and not without good reason; they suffer from almost every possible disease. A 2004 survey by the Kennel Club found that they die at the median age of 6.25 years (n=180). There really is no such thing as a healthy bulldog. The bulldog’s monstrous proportions makes them virtually incapable of mating or birthing without medical intervention.
    (via Bryan)

    (tags: dogs eugenics breeding horror science genetics traits animals pets bulldog pedigree)

  • SkyJack – autonomous drone hacking

    Samy Kamkar strikes again. ‘Using a Parrot AR.Drone 2, a Raspberry Pi, a USB battery, an Alfa AWUS036H wireless transmitter, aircrack-ng, node-ar-drone, node.js, and my SkyJack software, I developed a drone that flies around, seeks the wireless signal of any other drone in the area, forcefully disconnects the wireless connection of the true owner of the target drone, then authenticates with the target drone pretending to be its owner, then feeds commands to it and all other possessed zombie drones at my will.’

    (tags: drones amazon hacking security samy-kamkar aircrack node raspberry-pi airborne-zombies)

  • Why Did 9,000 Porny Spambots Descend on This San Diego High Schooler? – Alexis C. Madrigal – The Atlantic

    Good article about emergent behaviour from networked malware: ‘The metabot, therefore, is viral. You get followed because of who follows you. This tendency explains the strange geographical cluster among San Diego high school students. Perhaps one of those kids was being followed by a really popular account (like @Interscope records, perhaps, which follows hundreds of thousands of people), and through that link, the bot stumbled into this little circle of San Diego teens. All of this activity would have remained under the radar, of course, all part of the silent non-human web. Except something went awry. For some reason, Olivia got stuck in a weird loop, and the metabot kept spawning spambots that chose to follow her over and over, relentlessly. Maybe once the metabot reached the San Diego kids, a bug kicked in. Instead of negative feedback keeping her (and everyone else) from being followed too often, we got runaway positive feedback. The bots followed her because other bots followed her. And on and on. Which is, perhaps a kind of reasoning that we can understand: It’s the core logic of fame and celebrity itself. Attention flows to Snooki because attention flowed to Snooki. Attention flows to Olivia because attention flowed to Olivia. Olivia and her friends weren’t wrong when they thought she’d become suddenly famous. Her audience just wasn’t human.’

    (tags: socialnetworking spam twitter bots fame alexis-madrigal)

Links for 2013-12-03

Published December 3, 2013

Links for 2013-12-02

Published December 2, 2013

Links for 2013-11-28

Published November 28, 2013

Links for 2013-11-26

Published November 26, 2013

Links for 2013-11-25

Published November 25, 2013

Links for 2013-11-21

Published November 21, 2013

  • Shadows in the Woods

    beautiful German boardgame, suitable for playing with kids — an adult moves a tealight candle around the board, while kids take turns moving gnomes around in the shadows behind tall “trees”. recommended by JK

    (tags: games boardgames german kids candles light)

  • ‘No basis in law’ : Gardai probe Ballyphehane group after raid

    Freemen wackiness in Cork.

    The house of one member of the group was raided by gardaí last week, but it is not thought that any arrests were made, according to an eyewitness. Gardaí broke down the front door of the house. The group, which appears to be part of the Freemen of the Land movement, which does not recognise the State, has attempted to hold ‘trials’ in Ballyphehane Community Centre. It attempted to summon HSE staff, gardaí, social workers, solicitors and others to appear to be tried by a self-selected jury earlier this month. The group handed out documents purporting to be a summons to HSE staff and garda stations, demanding that named people attend a trial by ‘éire court’ on Tuesday 5 November at 9am “to stand trial for their acts of terrorism against mothers, their offspring and others in our community”, according to the group’s literature. This week the group has begun posting about UCC, saying the college is “a private for profit corporation, and a business partner of and partly owned by Pfizers and Bank of Ireland”. The group suggest that UCC bases its “authority” on Maritime Law. UCC has yet to respond to the group’s allegations.

    (tags: freemen crazy cork politics ireland hse gardai ucc law)

Links for 2013-11-20

Published November 20, 2013

Links for 2013-11-19

Published November 19, 2013

  • Software Detection of Currency

    Steven J. Murdoch presents some interesting results indicating that the EURion constellation may have been obsoleted:

    Recent printers, scanners and image manipulation software identify images of currency, will not process the image and display an error message linking to www.rulesforuse.org. The detection algorithm is not disclosed, however it is possible to test sample images as to whether they are identified as currency. This webpage shows an initial analysis of the algorithm’s properties, based on results from the automated generation and testing of images. […] Initially it was thought that the “Eurion constellation” was used to identify banknotes in the newly deployed software based system, since this has been confirmed to be the technique used by colour photocopiers, and was both necessary and sufficient to prevent an item being duplicated using the photocopier tested. However further investigation showed that the detection performed by software is different from the system used in colour photocopiers, and the Eurion constellation is neither necessary nor sufficent, and in fact it probably is not even a factor.

    (tags: eurion algorithms photoshop security currency money euro copying obscurity reversing)

  • Factual/drake

    a simple-to-use, extensible, text-based data workflow tool that organizes command execution around data and its dependencies. Data processing steps are defined along with their inputs and outputs and Drake automatically resolves their dependencies. […] Drake is similar to GNU Make, but designed especially for data workflow management. It has HDFS [and S3] support, allows multiple inputs and outputs, and includes a host of features designed to help you bring sanity to your otherwise chaotic data processing workflows.
    Via Nelson. Looks interesting, although I’d like to see more features around retries, single-executor locking, parallelism, alerting/metrics, and unattended cron-like operation — those are always the hard part when I wind up coding up a data pump.

    (tags: make data data-pump drake via:nelson pipelines workflow)

  • AK at re:Invent 2013: Getting Maximum Performance from Redshift

    good Redshift tips

    (tags: redshift aws amazon performance scaling s3 rdbms sql ops analytics)

  • Tintin And The Copyright Sharks – Falkvinge on Infopolicy

    A rather sordid tale of IP acquisition and exploitation, from the sounds of it

    (tags: tintin moulinsart belgium history herge ip copyright royalties rick-falkvinge)

Links for 2013-11-16

Published November 16, 2013

  • IPSO representative trivialising impact of the Loyaltybuild data breach

    A very worrying quote from Una Dillon of the Irish Payment Services Organisation in regard to the Loyaltybuild incident:

    “I wouldn’t be overly concerned if one of my cards was caught up in this,” Dillon says. “Even in the worst-case scenario – one in which my card was used fraudulently – my card provider will refund me everything that is taken”.
    This reflects a deep lack of understanding of (a) how identity fraud works, and (b) how card-fraud refunds in Ireland appear to work. (a): Direct misuse of credit card data is not always the result. Fraudsters may prefer to instead obtain separate credit through identity theft, ie. using other personal identifying data. (b): Visa debit cards have no credit limit — your bank account can be cleared out in its entirety, and refunds can take a long time. For instance, http://www.askaboutmoney.com/showthread.php?t=174482 describes several cases, including one customer who waited 21 days for a refund. All in all it’s trivialising a major risk for consumers. As I understand it, a separate statement from IPSO recommended that all customers of Loyaltybuild schemes need to monitor their bank accounts daily to keep an eye out for fraud, which is pretty absurd. Not impressive at all.

    (tags: loyaltybuild ipso money cards credit-cards visa debit-cards payment fraud identity-theft ireland)

  • Why GitHub is not your CV

    There is really astonishingly little value in looking at someone’s GitHub projects out of context. For a start, GitHub has no way of customising your profile page, and what is shown by default is the projects with the most stars, and the projects you’ve recently pushed to. That is, GitHub picks your most popular repos and puts those at the top. You have no say about what you consider important, or worthwhile, or interesting, or well-engineered, or valuable. You just get what other people think is useful. Aside from which, GitHub displays a lot of useless stats about how many followers you have, and some completely psychologically manipulative stats about how often you commit and how many days it is since you had a day off. So really, your GitHub profile displays two things: how ‘influential’ you are, and how easily you can be coerced into constantly working. It’s honestly about as relevant to a decent hiring decision as your Klout score.

    (tags: cv github open-source hiring career meritocracy work via:apyhr)

  • An Empirical Evaluation of TCP Performance in Online Games

    In this paper, we have analyzed the performance of TCP in of ShenZhou Online, a commercial, mid-sized MMORPG. Our study indicates that, though TCP is full-fledged and robust, simply transmitting game data over TCP could cause unexpected performance problems. This is due to the following distinctive characteristics of game traffic: 1) tiny packets, 2) low packet rate, 3) application-limited traffic generation, and 4) bi-directional traffic. We have shown that because TCP was originally designed for unidirectional and network-limited bulk data transfers, it cannot adapt well to MMORPG traffic. In particular, the window-based congestion control mechanism and the fast retransmit algorithm for loss recovery are ineffective. This suggests that the selective acknowledgement option should be enabled whenever TCP is used, as it significantly enhances the loss recovery process. Furthermore, TCP is overkill, as not every game packet needs to be transmitted reliably and processed in an orderly manner. We have also shown that the degraded network performance did impact users’ willingness to continue a game. Finally, a number of design guidelines have been proposed by exploiting the unique characteristics of game traffic.
    via Nelson

    (tags: tcp games udp protocols networking internet mmos retransmit mmorpgs)

  • Column: The Loyaltybuild breach shows it’s time to take data protection seriously

    What is afoot here is a rerun of the Celtic Tiger era “light touch regulation” of financial services. Ireland has again made a Faustian pact whereby we lure employers here on the understanding that they will not subject to too-stringent a regulatory system. As the Loyaltybuild breach has shown, this is a bargain that will probably end badly. And as with the financial services boom, it is making the Germans nervous. Perhaps we will listen to them this time.

    (tags: fergal-crehan loyaltybuild celtic-tiger ireland dpa regulation data-protection privacy credit-cards)

  • mgodave/barge

    Looks very alpha, but one to watch.

    A JVM Implementation of the Raft Consensus Protocol

    (tags: via:sbtourist raft jvm java consensus distributed-computing)

Links for 2013-11-15

Published November 15, 2013

  • RocksDB

    ‘ A persistent key-value store for fast storage environments’, ie. BerkeleyDB/LevelDB competitor, from Facebook.

    RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation. We benchmarked LevelDB and found that it was unsuitable for our server workloads. Thebenchmark results look awesome at first sight, but we quickly realized that those results were for a database whose size was smaller than the size of RAM on the test machine – where the entire database could fit in the OS page cache. When we performed the same benchmarks on a database that was at least 5 times larger than main memory, the performance results were dismal. By contrast, we’ve published the RocksDB benchmark results for server side workloads on Flash. We also measured the performance of LevelDB on these server-workload benchmarks and found that RocksDB solidly outperforms LevelDB for these IO bound workloads. We found that LevelDB’s single-threaded compaction process was insufficient to drive server workloads. We saw frequent write-stalls with LevelDB that caused 99-percentile latency to be tremendously large. We found that mmap-ing a file into the OS cache introduced performance bottlenecks for reads. We could not make LevelDB consume all the IOs offered by the underlying Flash storage.
    Lots of good discussion at https://news.ycombinator.com/item?id=6736900 too.

    (tags: flash ssd rocksdb databases storage nosql facebook bdb disk key-value-stores lsm leveldb)

  • Amazon Route 53 Infima

    Colm McCarthaigh has open sourced Infima, ‘a library for managing service-level fault isolation using Amazon Route 53’.

    Infima provides a Lattice container framework that allows you to categorize each endpoint along one or more fault-isolation dimensions such as availability-zone, software implementation, underlying datastore or any other common point of dependency endpoints may share. Infima also introduces a new ShuffleShard sharding type that can exponentially increase the endpoint-level isolation between customer/object access patterns or any other identifier you choose to shard on. Both Infima Lattices and ShuffleShards can also be automatically expressed in Route 53 DNS failover configurations using AnswerSet and RubberTree.

    (tags: infima colmmacc dns route-53 fault-tolerance failover multi-az sharding service-discovery)

Links for 2013-11-13

Published November 13, 2013

Links for 2013-11-12

Published November 12, 2013

  • Reactor hits GA

    ‘It can’t just be Big Data, it has to be Fast Data: Reactor 1.0 goes GA’:

    Reactor provides the necessary abstractions to build high-throughput, low-latency–what we now call “fast data”–applications that absolutely must work with thousands, tens of thousands, or even millions of concurrent requests per second. Modern JVM applications must be built on a solid foundation of asynchronous and reactive components that efficiently manage the execution of a very large number of tasks on a very small number of system threads. Reactor is specifically designed to help you build these kinds of applications without getting in your way or forcing you to work within an opinionated pattern.
    Featuring the LMAX Disruptor ringbuffer, the JavaChronicle fast persistent message-passing queue, Groovy closures, and Netty 4.0. This looks very handy indeed….

    (tags: disruptor reactive-programming reactor async libraries java jvm frameworks spring netty fast-data)

  • Backblaze Blog » How long do disk drives last?

    According to Backblaze’s data, 80% of drives last 4 years, and the median lifespan is projected to be 6 years

    (tags: backblaze storage disk ops mtbf hardware failure lifespan)

  • Heirloom Chemistry Set by John Farrell Kuhns — Kickstarter

    This is a beauty. I wonder if they can ship to Ireland?

    To tell our story for this Kickstarter project, we really have to start in Christmas of 1959. Like many young scientists of the time, I received a Gilbert Chemistry set. This chemistry set provided me hours of great fun and learning as well as laying the foundation for my future as a research chemist. As I became an adult I wanted to share these types of experiences with my daughter, my nephews and nieces, and friends. But soon I became aware real chemistry sets were no longer available. Without real chemistry sets and opportunities for students to learn and explore, where would our future chemists come from? So …. I set out on a mission.

    (tags: chemistry science chemistry-sets education play kickstarter)

  • Philippe Flajolet’s contribution to streaming algorithms [preso]

    Nice deck covering HyperLogLog and its origins, plus a slide at the end covering the Flajolet/Wegman Adaptive Sampling algorithm (“how do you count the number of elements which appear only once in stream using constant size memory?”)

    (tags: algorithms sketching hyperloglog flajolet wegman adaptive-sampling sampling presentations slides)

  • 3 Tacos or 4 Flautas Per Order Make a Healthy Diet in Greatest Scientific Study Ever

    “In reality, [tacos and flautas] aren’t bad meals,” the report argues. “The error that many of us Mexicans [Gustavo note: and gabachos] commit is including these types of dishes in our regular diet without an appropriate balance of them and falling into excessively eating them; accompanied by a lack of physical activity, it creates bad eating habits.” The good docs go on to note that people can eat tacos and flautas without negatively affecting their health, but “the key resides in controlling the quantity and frequency of eating these types of meals.” They also make the point that overall, tacos and flautas have less grease than doughnuts, french fries and even some health bars, although they didn’t specify which brands in the latter. In a subsequent blog post, the scientists go on to describe flautas as an “energy food” due to their composition, and conclude by recommending that a healthy diet can include three tacos al pastor or four flautas per order, “controlling the frequency of intake.” So have at it, boyos, but in moderation. And I can already hear the skeptics: What about tacos de chicharrones? Why not focus on carne asada? Did they take into consideration chiles de mordida? Did they factor in horchata? And whither the burrito variable?

    (tags: science tacos flautas mexican-food food eating yay)

Links for 2013-11-11

Published November 11, 2013

Links for 2013-11-08

Published November 8, 2013

  • Where your “full Irish” really comes from

    This is really disappointing; many meats labelled as “Irish” are anything but. The only trustworthy mark is the Bord Bia “Origin Ireland” stamp — I’ll be avoiding any products without this in future.

    Under European labelling law, country of origin is mandatory for beef, fish, olive oil, honey and fresh fruit and vegetables. Next month the EU will make it law to specify country of origin for the meat of pigs, chicken, sheep and goats, with a lead-in time of anywhere up to three years for food companies to comply. The pork rule, however, will only apply to fresh pork and not to processed meat, so consumers still won’t get a country-of-origin label on rashers, sausages or ham. In the meantime, the Bord Bia Origin-Ireland stamp is a guarantee that your Irish breakfast ingredients are indeed Irish.

    (tags: bord-bia labelling eu country-of-origin meat pork food quality)

  • Killing Freedom of Information in Ireland

    TheStory.ie will, in all likelihood, cease all FOI requests. And we will not seek funding from the public to support an immoral, cynical, unjustified and probably illegal FOI fee regime. We will not pay for information that the public already pays for. We will not support a system that perpetuates an outrageous infringement of citizen rights. The legislation was gutted in 2003 and it is being gutted again. More generally the number of requests from journalists from all news organisations in Ireland will fall as a result of these amendments, and the resulting efforts to shine a light on the administration of the State will certainly deteriorate. And secrecy will prevail.

    (tags: ireland politics foi information secrecy law)

Links for 2013-11-07

Published November 7, 2013

  • 10 Things You Should Know About AWS

    Some decent tips in here, mainly EC2-focussed

    (tags: amazon ec2 aws ops rds)

  • Tracing Brazil’s Guy Fawkes Masks

    really fascinating, from Ethan Zuckerman:

    The photo of workers making Guy Fawkes masks is something of a Rorschach test. If you’re primed to see the exploitative nature of global capitalism when you see people making a plastic mask, it’s there in the image. if you’re looking for the global spread of a protest movement, it’s there too, with a Brazilian factory making a local knock-off of a global icon to cash in on a national protest. Because the internet is a copying machine, it’s very bad at context. It’s easier to encounter the image of masks being manufactured devoid of accompanying details than it is to find the story behind the images. And given our tendency to ignore information in languages we don’t read, it’s easy to see how the masks come detached from their accompanying story. For me, the image is more powerful with context behind it. It’s possible to reflect on the irony of a Hollywood prop becoming an activist trope, the tensions between mass-production and anonymity and the individuality of one’s identity and grievance, the tensions between local and global, Warner Bros and Condal, intellectual property and piracy, all in the same image.

    (tags: anonymous globalization manufacturing piracy knock-offs brazil ethan-zuckerman global local hollywood capitalism)

  • ReCreate Ireland – Creativity through Reuse

    Great idea.

    For creative groups, we aim to offer easy access to a rich and varied selection of textures, colours and shapes. Members are also be able to participate in creativity workshops facilitated by fully trained professional artists either in-house or on your own premises. We intend to be the first choice of teachers, early childhood educators and arts animators in the community. For businesses, ReCreate reduces the costs of moving on end-of-line materials. We are a professional, credible and reliable partner organisation and our aim is to divert approximately 115 metric tonnes of clean materials from landfill annually. All collections are free of charge.

    (tags: recreate diy make-and-do recycling landfill art play scrap)

  • 3D-Print Your Own 20-Million-Year-Old Fossils

    When I get my hands on a 3-D printer, this will be high up my list of things to fabricate: a replica of a 20-million year old hominid skull.

    With over 40 digitized fossils in their collection, you can explore 3D renders of fossils representing prehistoric animals, human ancestors, and even ancient tools. Captured using Autodesk software, an SLR camera, and often the original specimen (rather than a cast replica), these renderings bring us closer than most will ever get to holding ancient artifacts. And if you’ve got an additive manufacturing device at your disposal, you can even download Sketchfab plans to generate your own.

    (tags: 3d-printing fossils africa history hominids replication fabrication sketchfab)

  • Makers & Brothers & Others

    ‘A Tiny Seasonal Department Store’, featuring the amazing cakes of Wildflour Bakery among others, at 5 Dame Lane, D2.

    The tiny department store will be a wonderful seasonal gathering of Makers & Brothers favourite local and international brands. The Others in this project are a carefully considered bunch of partners from the worlds of flowers, food, fashion, beauty, homeware, gifts and more.  Makers & Brothers & Others, the tiny department store, promises to be a unique, exciting and engaging retail environment. A place to explore, a seasonal store alive with wonder and served by experts. Kindly hosted by the Fumbally Exchange.

    (tags: dublin shopping food cakes wildflour-bakery makers-and-brothers xmas)

Links for 2013-11-06

Published November 6, 2013

Links for 2013-11-05

Published November 5, 2013

Links for 2013-11-04

Published November 4, 2013

Links for 2013-11-03

Published November 3, 2013

  • There is NO spare capacity for Dublin’s water supply

    The problem in a nutshell is that for an uncomfortable amount of the year the demand outstrips what the system can comfortably supply. In the graph below you’ll see the red line (demand for water) matches and regularly exceeds the blue line (what’s produced).

    (tags: drought water dublin mismanagement capacity dcc dublin-council graphs)

  • Old Fashioned 101

    Circa 1800, the Cocktail was a “hair of the dog” morning drink that tamed spirits with water, sugar and bitters (patent medicine). The late 19th Century expanded the use of the word “cocktail” to encompass just about any mixed drink. Since then, the Old Fashioned—literally, the old-fashioned way of making a cocktail—has been our contemporary expression of the original drink. During the 20th Century, various bad ideas encrusted the Old Fashioned. Here we will strip off those barnacles to expose the amazingly simple and sublime drink beneath.
    thanks to Ben for this one…

    (tags: recipe alcohol drinks cocktails old-fashioned bourbon bitters)

  • Metropolitan police detained David Miranda for promoting ‘political’ causes | World news | The Observer

    “We assess that Miranda is knowingly carrying material […] the disclosure or threat of disclosure is designed to influence a government, and is made for the purpose of promoting a political or ideological cause. This therefore falls within the definition of terrorism.”

    (tags: security david-miranda journalism censorship terrorism the-guardian)

  • A Brief Tour of FLP Impossibility

    One of the most important results in distributed systems theory was published in April 1985 by Fischer, Lynch and Patterson. Their short paper ‘Impossibility of Distributed Consensus with One Faulty Process’, which eventually won the Dijkstra award given to the most influential papers in distributed computing, definitively placed an upper bound on what it is possible to achieve with distributed processes in an asynchronous environment. This particular result, known as the ‘FLP result’, settled a dispute that had been ongoing in distributed systems for the previous five to ten years. The problem of consensus – that is, getting a distributed network of processors to agree on a common value – was known to be solvable in a synchronous setting, where processes could proceed in simultaneous steps. In particular, the synchronous solution was resilient to faults, where processors crash and take no further part in the computation. Informally, synchronous models allow failures to be detected by waiting one entire step length for a reply from a processor, and presuming that it has crashed if no reply is received. This kind of failure detection is impossible in an asynchronous setting, where there are no bounds on the amount of time a processor might take to complete its work and then respond with a message. Therefore it’s not possible to say whether a processor has crashed or is simply taking a long time to respond. The FLP result shows that in an asynchronous setting, where only one processor might crash, there is no distributed algorithm that solves the consensus problem.

    (tags: distributed-systems flp consensus-algorithms algorithms distcomp papers proofs)

  • Find a separating hyperplane with this One Weird Kernel Trick

    Terrible internet ad-spam recast as machine-learning spam

    ’37-year-old patriot discovers “weird” trick to end slavery to the Bayesian monopoly. Discover the underground trick she used to slash her empirical risk by 75% in less than 30 days… before they shut her down. Click here to watch the shocking video! Get the Shocking Free Report!’

    (tags: funny via:hmason machine-learning spam wtf svms bayesian)

Links for 2013-11-01

Published November 1, 2013

  • It’s time for Silicon Valley to ask: Is it worth it?

    These companies and their technologies are built on data, and the data is us. If we are to have any faith in the Internet, we have to trust them to protect it. That’s a relationship dynamic that will become only more intertwined as the Internet finds its way into more aspects of our daily existences, from phones that talk to us to cars that drive themselves. The US’s surveillance programs threaten to destroy that trust permanently. America’s tech companies must stand up to this pervasive and corrosive surveillance system. They must ask that difficult question: “Is it worth it?”

    (tags: silicon-valley tech nsa gchq spying surveillance internet privacy data-protection)

  • Serf

    ‘a service discovery and orchestration tool that is decentralized, highly available, and fault tolerant. Serf runs on every major platform: Linux, Mac OS X, and Windows. It is extremely lightweight: it uses 5 to 10 MB of resident memory and primarily communicates using infrequent UDP messages [and an] efficient gossip protocol.’

    (tags: clustering service-discovery ops linux gossip broadcast clusters)

  • “Effective Computation of Biased Quantiles over Data Streams” [paper]

    Skew is prevalent in many data sources such as IP traffic streams.To continually summarize the distribution of such data, a high-biased set of quantiles (e.g., 50th, 90th and 99th percentiles) with finer error guarantees at higher ranks (e.g., errors of 5, 1 and 0.1 percent, respectively) is more useful than uniformly distributed quantiles (e.g., 25th, 50th and 75th percentiles) with uniform error guarantees. In this paper, we address the following two prob-lems. First, can we compute quantiles with finer error guarantees for the higher ranks of the data distribution effectively, using less space and computation time than computing all quantiles uniformly at the finest error? Second, if specific quantiles and their error bounds are requested a priori, can the necessary space usage and computation time be reduced? We answer both questions in the affirmative by formalizing them as the “high-biased” quantiles and the “targeted” quantiles problems, respectively, and presenting algorithms with provable guarantees, that perform significantly better than previously known solutions for these problems. We implemented our algorithms in the Gigascope data stream management system, and evaluated alternate approaches for maintaining the relevant summary structures.Our experimental results on real and synthetic IP data streams complement our theoretical analyses, and highlight the importance of lightweight, non-blocking implementations when maintaining summary structures over high-speed data streams.
    Implemented as a timer-histogram storage system in http://armon.github.io/statsite/ .

    (tags: statistics quantiles percentiles stream-processing skew papers histograms latency algorithms)

  • Statsite

    A C reimplementation of Etsy’s statsd, with some interesting memory optimizations.

    Statsite is designed to be both highly performant, and very flexible. To achieve this, it implements the stats collection and aggregation in pure C, using libev to be extremely fast. This allows it to handle hundreds of connections, and millions of metrics. After each flush interval expires, statsite performs a fork/exec to start a new stream handler invoking a specified application. Statsite then streams the aggregated metrics over stdin to the application, which is free to handle the metrics as it sees fit. This allows statsite to aggregate metrics and then ship metrics to any number of sinks (Graphite, SQL databases, etc). There is an included Python script that ships metrics to graphite.

    (tags: statsd graphite statsite performance statistics service-metrics metrics ops)

  • 34 Irish pubs listed in Michelin good food guide

    if Linnane’s and Cronin’s are anything to go by, these will be worth a visit

    (tags: pubs ireland tourism food holidays michelin)

  • Fax vs Twilio

    A fax machine called my #twilio voice number, this is how @twilio transcribed it…. http://pic.twitter.com/RYh19Pg2pG
    This is amazing. Machine talking to machine, with hilarious results

    (tags: twilio transcription machine audio fax hey-hey-hey you-know-its-hey funny)

Links for 2013-10-31

Published October 31, 2013

  • Dark Mail Alliance

    Founded by Silent Circle and Lavabit. this is promising….

    To bring the world our unique end-to-end encrypted protocol and architecture that is the ‘next-generation’ of private and secure email. As founding partners of The Dark Mail Alliance, both Silent Circle and Lavabit will work to bring other members into the alliance, assist them in implementing the new protocol and jointly work to proliferate the worlds first end-to-end encrypted ‘Email 3.0’ throughout the world’s email providers. Our goal is to open source the protocol and architecture and help others implement this new technology to address privacy concerns against surveillance and back door threats of any kind.

    (tags: privacy surveillance email smtp silent-circle lavabit dark-mail open-source standards crypto)

  • Ponies by Kij Johnson | Tor.com

    A rather dark short story about little girls, peer pressure, and childhood. no fun for this dad of 3 girls :( (via Tatu Saloranta)

    (tags: via:cowtowncoder writing fiction sf childhood peer-pressure tor ponies)

  • HdrHistogram by giltene

    A Histogram that supports recording and analyzing sampled data value counts across a configurable integer value range with configurable value precision within the range. Value precision is expressed as the number of significant digits in the value recording, and provides control over value quantization behavior across the value range and the subsequent value resolution at any given level.

    (tags: hdr histogram data-structures coding gil-tene sampling measuring)

Links for 2013-10-30

Published October 30, 2013

  • Counterfactual Thinking, Rules, and The Knight Capital Accident

    John Allspaw with an interesting post on the Knight Capital disaster

    (tags: john-allspaw ops safety post-mortems engineering procedures)

  • Toyota’s killer firmware: Bad design and its consequences

    This is exactly what you do NOT want to read about embedded systems controlling acceleration in your car:

    The Camry electronic throttle control system code was found to have 11,000 global variables. Barr described the code as “spaghetti.” Using the Cyclomatic Complexity metric, 67 functions were rated untestable (meaning they scored more than 50). The throttle angle function scored more than 100 (unmaintainable). Toyota loosely followed the widely adopted MISRA-C coding rules but Barr’s group found 80,000 rule violations. Toyota’s own internal standards make use of only 11 MISRA-C rules, and five of those were violated in the actual code. MISRA-C:1998, in effect when the code was originally written, has 93 required and 34 advisory rules. Toyota nailed six of them. Barr also discovered inadequate and untracked peer code reviews and the absence of any bug-tracking system at Toyota.
    On top of this, there was no error-correcting RAM in use; stack-killing recursive code; a quoted 94% stack usage; risks of unintentional RTOS task shutdown; buffer overflows; unsafe casting; race conditions; unchecked error code return values; and a trivial watchdog timer check. Crappy, unsafe coding.

    (tags: firmware horror embedded-systems toyota camry safety acceleration misra-c coding code-verification spaghetti-code cyclomatic-complexity realtime rtos c code-reviews bug-tracking quality)

  • Forensic Topology

    The sounds were not, however, caused by ghosts but by a group of three or four men at least to some degree professionally trained, the FBI now believes, in tunneling: a close-knit and highly disciplined team, perhaps from the construction industry, perhaps even a disgruntled public works crew who decided to put their knowledge of the city’s underside to more lucrative work. After all, Rehder explained, their route into the bank was as much brute-force excavation as it was a retracing of the region’s buried waterways, accessing the neighborhood by way of the city’s complicated storm-sewer network, itself built along old creek beds that no longer appear on city maps. As LAPD lieutenant Doug Collisson, one of the men present on the day of the tunnel’s discovery, explained to the Los Angeles Times back in 1987, the crew behind the burglary “would have had to require some knowledge of soil composition and technical engineering. … The way the shaft itself was constructed, it was obviously well-researched and extremely sophisticated.” Rehder actually goes further, remarking that when Detective Dennis Pagenkopp “showed crime scene photos of the core bit holes” produced by the burglars’ boring upward into the vault “to guys who were in the concrete-coring business, they whistled with professional admiration.”

    (tags: cities crime architecture digging tunnels subterranean la lapd banks via:bldgblog sewers)

Links for 2013-10-29

Published October 29, 2013

Links for 2013-10-28

Published October 28, 2013

  • 14 Apple hacks from sugru

    I like the impromptu docking station hack

    (tags: apple sugru hacks hardware fixing repair diy)

  • Bruce Schneier On The Feudal Internet And How To Fight It

    This is very well-put.

    In its early days, there was a lot of talk about the “natural laws of the Internet” and how it would empower the masses, upend traditional power blocks, and spread freedom throughout the world. The international nature of the Internet made a mockery of national laws. Anonymity was easy. Censorship was impossible. Police were clueless about cybercrime. And bigger changes were inevitable. Digital cash would undermine national sovereignty. Citizen journalism would undermine the media, corporate PR, and political parties. Easy copying would destroy the traditional movie and music industries. Web marketing would allow even the smallest companies to compete against corporate giants. It really would be a new world order. Unfortunately, as we know, that’s not how it worked out. Instead, we have seen the rise of the feudal Internet: Feudal security consolidates power in the hands of the few. These companies [like Google, Apple, Microsoft, Facebook etc.] act in their own self-interest. They use their relationship with us to increase their profits, sometimes at our expense. They act arbitrarily. They make mistakes. They’re deliberately changing social norms. Medieval feudalism gave the lords vast powers over the landless peasants; we’re seeing the same thing on the Internet.

    (tags: bruce-schneier politics internet feudal-internet google apple microsoft facebook government)

  • Russia: Hidden chips ‘launch malware attacks from irons’

    Cyber criminals are planting chips in electric irons and kettles to launch spam [jm: actually, malware] attacks, reports in Russia suggest. State-owned channel Rossiya 24 even showed footage of a technician opening up an iron included in a batch of Chinese imports to find a “spy chip” with what he called “a little microphone”. Its correspondent said the hidden devices were mostly being used to spread viruses, by connecting to any computer within a 200m (656ft) radius which were using unprotected Wi-Fi networks. Other products found to have rogue components reportedly included mobile phones and car dashboard cameras.

    (tags: wifi viruses spam malware security russia china toasters kettles appliances)

  • Asteroid “mining” with Linux and FOSS

    Planetary Resources is a company with a sky-high (some might claim “pie in the sky”) goal: to find and mine asteroids for useful minerals and other compounds. It is also a company that uses Linux and lots of free software. So two of the engineers from Planetary Resources, Ray Ramadorai and Marc Allen, gave a presentation at LinuxCon North America to describe how and why the company uses FOSS—along with a bit about what it is trying to do overall.

    (tags: lwn mining planets asteroids space linux foss open-source)

Links for 2013-10-26

Published October 26, 2013

  • Mac OS 10.9 – Infinity times your spam

    a pretty stupid Mail.app IMAP bug hoses Fastmail:

    Yes you read that right. It’s copying all the email from the Junk Folder back into the Junk Folder again!. This is legal IMAP, so our server proceeds to create a new copy of each message in the folder. It then expunges the old copies of the messages, but it’s happening so often that the current UID on that folder is up to over 3 million. It was just over 2 million a few days ago when I first emailed the user to alert them to the situation, so it’s grown by another million since. The only way I can think this escaped QA was that they used a server which (like gmail) automatically suppresses duplicates for all their testing, because this is a massively bad problem.

    (tags: osx bugs mail.app mail imap fastmail fail)

  • Google: Our Robot Cars Are Better Drivers Than Puny Humans | MIT Technology Review

    One of those analyses showed that when a human was behind the wheel, Google’s cars accelerated and braked significantly more sharply than they did when piloting themselves. Another showed that the cars’ software was much better at maintaining a safe distance from the vehicle ahead than the human drivers were. “We’re spending less time in near-collision states,” said Urmson. “Our car is driving more smoothly and more safely than our trained professional drivers.”

    (tags: google cars driving safety roads humans robots automation)

Links for 2013-10-25

Published October 25, 2013

Links for 2013-10-24

Published October 24, 2013

  • Roma, Racism And Tabloid Policing: Interview With Gary Younge : rabble

    [This case] shows the link between the popular and the state. This is tabloid journalism followed by tabloid policing. It’s also completely ignorant. I wrote my article on the Roma after covering the community for a week. I thought, “that’s interesting – there’s a range of phenotypes, ways of looking, that include Roma.” I mentioned two blonde kids by chance. I mentioned that Roma are more likely to speak the language of the country they’re in than Romani, more likely to have the religion of the country they’re in. But they have the basic aspect that is true for all identities – they know each other and other people know them. It’s not like I’m an expert on the Roma. I was covering them for a week and after the second day I knew Roma children had blonde hair and blue eyes. These people who took that kid away knew nothing. And on that basis they abducted a child.

    (tags: roma racism ireland gary-younge tabloid journalist children hse gardai)

  • Experian Sold Consumer Data to ID Theft Service

    This is what happens when you don’t have strong controls on data protection/data privacy — the US experience.

    While [posing as a US-based private investigator] may have gotten the [Vietnam-based gang operating the massive identity fraud site Superget.info] past Experian and/or CourtVentures’ screening process, according to Martin there were other signs that should have alerted Experian to potential fraud associated with the account. For example, Martin said the Secret Service told him that the alleged proprietor of Superget.info had paid Experian for his monthly data access charges using wire transfers sent from Singapore. “The issue in my mind was the fact that this went on for almost a year after Experian did their due diligence and purchased” Court Ventures, Martin said. “Why didn’t they question cash wires coming in every month? Experian portrays themselves as the data-breach experts, and they sell identity theft protection services. How this could go on without them detecting it I don’t know. Our agreement with them was that our information was to be used for fraud prevention and ID verification, and was only to be sold to licensed and credentialed U.S. businesses, not to someone overseas.”
    via Simon McGarr

    (tags: via:tupp_ed privacy security crime data-protection data-privacy experian data-breaches courtventures superget scams fraud identity identity-theft)

Links for 2013-10-23

Published October 23, 2013

Links for 2013-10-22

Published October 22, 2013

Links for 2013-10-21

Published October 21, 2013

Links for 2013-10-20

Published October 20, 2013

  • WISH: A Monumental 11-Acre Portrait in Belfast by Jorge Rodríguez-Gerada

    Must go up and visit this.

    Unveiled several days ago in Belfast, Northern Ireland as part of the Belfast Festival, WISH is the latest public art project by Cuban-American artist Jorge Rodriguez-Gerada. The image depicted is of an anonymous Belfast girl and is so large it can only be viewed from the highest points in Belfast or an airplane. Several years in the making, WISH was first plotted on a grid using state-of-the-art Topcon GPS technology and 30,000 manually placed wooden stakes in Belfast’s Titanic Quarter. The portrait was then “drawn” with aid of volunteers who helped place nearly 8 million pounds of natural materials including soil, sand, and rock over a period of four weeks.

    (tags: belfast ireland art portraits jorge-rodriguez-gerada land soil)

Links for 2013-10-19

Published October 19, 2013

Links for 2013-10-15

Published October 15, 2013

  • How to Read a Scientific Paper (About That Researcher With a Nematode in His Mouth) – Wired Science

    Let’s rewind to September 2012. It was about then- according to this recently published report (paywall) in The American Journal of Tropical Medicine – that an “otherwise healthy, 36-year-old man” felt a rough patch in his mouth, a scaly little area his right cheek. It didn’t hurt. But then it didn’t stay there either. He started testing for it with his tongue. It traveled. It moved to the back of his mouth, then forward, coiled backwards again. In the language of science: “These rough patches would appear and disappear on a daily basis, giving the patient the indirect sense that there was an organism moving within the oral cavity.”

    (tags: nematodes parasites biology medicine paper gross funny wired mouth)

  • “High Performance Browser Networking”, by Ilya Grigorik, read online for free

    Wow, this looks excellent. A must-read for people working on systems with high-volume, low-latency phone-to-server communications — and free!

    How prepared are you to build fast and efficient web applications? This eloquent book provides what every web developer should know about the network, from fundamental limitations that affect performance to major innovations for building even more powerful browser applications—including HTTP 2.0 and XHR improvements, Server-Sent Events (SSE), WebSocket, and WebRTC. Author Ilya Grigorik, a web performance engineer at Google, demonstrates performance optimization best practices for TCP, UDP, and TLS protocols, and explains unique wireless and mobile network optimization requirements. You’ll then dive into performance characteristics of technologies such as HTTP 2.0, client-side network scripting with XHR, real-time streaming with SSE and WebSocket, and P2P communication with WebRTC. Deliver optimal TCP, UDP, and TLS performance; Optimize network performance over 3G/4G mobile networks; Develop fast and energy-efficient mobile applications; Address bottlenecks in HTTP 1.x and other browser protocols; Plan for and deliver the best HTTP 2.0 performance; Enable efficient real-time streaming in the browser; Create efficient peer-to-peer videoconferencing and low-latency applications with real-time WebRTC transports
    Via Eoin Brazil.

    (tags: book browser networking performance phones mobile 3g 4g hsdpa http udp tls ssl latency webrtc websockets ebooks via:eoin-brazil google http2 sse xhr ilya-grigorik)

  • Even the NSA is finding it hard to cope with spam

    3 new Snowden leaks, covering acquisition of Yahoo address books, buddy lists, and email account activity, and how spammer activity required intervention to avoid losing useful data in the noise

    (tags: spam spammers nsa snowden leaks anti-spam yahoo im mail)

  • “What Should I Monitor?”

    slides (lots of slides) from Baron Schwartz’ talk at Velocity in NYC.

    (tags: slides monitoring metrics ops devops baron-schwartz pdf capacity)

Links for 2013-10-14

Published October 14, 2013

  • The trouble with timestamps

    Timestamps, as implemented in Riak, Cassandra, et al, are fundamentally unsafe ordering constructs. In order to guarantee consistency you, the user, must ensure locally monotonic and, to some extent, globally monotonic clocks. This is a hard problem, and NTP does not solve it for you. When wall clocks are not properly coupled to the operations in the system, causal constraints can be violated. To ensure safety properties hold all the time, rather than probabilistically, you need logical clocks.

    (tags: clocks time distributed databases distcomp ntp via:fanf aphyr vector-clocks last-write-wins lww cassandra riak)