Skip to content

Year: 2014

Links for 2014-03-20

Published March 20, 2014

Links for 2014-03-19

Published March 19, 2014

  • No, Nate, brogrammers may not be macho, but that’s not all there is to it

    Great essay on sexism in tech, "brogrammer" culture, "clubhouse chemistry", outsiders, wierd nerds and exclusion:

    Every group, including the excluded and disadvantaged, create cultural capital and behave in ways that simultaneously create a sense of belonging for them in their existing social circle while also potentially denying them entry into another one, often at the expense of economic capital. It’s easy to see that wearing baggy, sagging pants to a job interview, or having large and visible tattoos in a corporate setting, might limit someone’s access. These are some of the markers of belonging used in social groups that are often denied opportunities. By embracing these markers, members of the group create real barriers to acceptance outside their circle even as they deepen their peer relationships. The group chooses to adopt values that are rejected by the society that’s rejecting them. And that’s what happens to “weird nerd” men as well—they create ways of being that allow for internal bonding against a largely exclusionary backdrop.
    (via Bryan O'Sullivan)

    (tags: nerds outsiders exclusion society nate-silver brogrammers sexism racism tech culture silicon-valley essays via:bos31337)

  • Impact of large primitive arrays (BLOBS) on JVM Garbage Collection

    some nice graphs and data on CMS performance, with/without -XX:ParGCCardsPerStrideChunk

    (tags: cms java jvm performance optimization tuning off-heap-storage memory)

  • Anatomical Collages by Travis Bedel

    these are fantastic

    (tags: collage anatomy art prints)

  • htcat/htcat

    a utility to perform parallel, pipelined execution of a single HTTP GET. htcat is intended for the purpose of incantations like: htcat https://host.net/file.tar.gz | tar -zx It is tuned (and only really useful) for faster interconnects: [....] 109MB/s on a gigabit network, between an AWS EC2 instance and S3. This represents 91% use of the theoretical maximum of gigabit (119.2 MiB/s).

    (tags: go cli http file-transfer ops tools)

Links for 2014-03-18

Published March 18, 2014

  • Analyzing Citibike Usage

    Abe Stanway crunches the stats on Citibike usage in NYC, compared to the weather data from Wunderground.

    (tags: data correlation statistics citibike cycling nyc data-science weather)

  • NSA surveillance recording every single voice call in at least 1 country

    Storing them in a 30-day rolling buffer, allowing retrospective targeting weeks after the call. 100% of all voice calls in that country, although it's unclear which country that is

    (tags: nsa surveillance gchq telephones phone bugging)

  • S3QL

    a file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X. S3QL is a standard conforming, full featured UNIX file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees and snapshotting which make it especially suitable for online backup and archival.

    (tags: s3 s3ql backup aws filesystems linux freebsd osx ops)

  • What's New in Java 8

    good explanation of all the new features -- I'm really looking forward to fixing up all the crappy over-verbose interface-as-lambdas we have scattered throughout our code

    (tags: java java8 lambdas fp functional-programming currying joda-time)

  • FM-index

    a compressed full-text substring index based on the Burrows-Wheeler transform, with some similarities to the suffix array. It was created by Paolo Ferragina and Giovanni Manzini,[1] who describe it as an opportunistic data structure as it allows compression of the input text while still permitting fast substring queries. The name stands for 'Full-text index in Minute space'. It can be used to efficiently find the number of occurrences of a pattern within the compressed text, as well as locate the position of each occurrence. Both the query time and storage space requirements are sublinear with respect to the size of the input data.
    kragen notes 'gene sequencing is using [them] in production'.

    (tags: sequencing bioinformatics algorithms bowtie fm-index indexing compression search burrows-wheeler bwt full-text-search)

Links for 2014-03-14

Published March 14, 2014

  • Health privacy: formal complaint to ICO

    'Light Blue Touchpaper' notes:

    Three NGOs have lodged a formal complaint to the Information Commissioner about the fact that PA Consulting uploaded over a decade of UK hospital records to a US-based cloud service. This appears to have involved serious breaches of the UK Data Protection Act 1998 and of multiple NHS regulations about the security of personal health information.
    Let's see if ICO can ever do anything useful.... not holding my breath

    (tags: ico privacy data-protection dpa nhs health data ross-anderson)

  • Why Google Flu Trends Can't Track the Flu (Yet)

    It's admittedly hard for outsiders to analyze Google Flu Trends, because the company doesn't make public the specific search terms it uses as raw data, or the particular algorithm it uses to convert the frequency of these terms into flu assessments. But the researchers did their best to infer the terms by using Google Correlate, a service that allows you to look at the rates of particular search terms over time. When the researchers did this for a variety of flu-related queries over the past few years, they found that a couple key searches (those for flu treatments, and those asking how to differentiate the flu from the cold) tracked more closely with Google Flu Trends' estimates than with actual flu rates, especially when Google overestimated the prevalence of the ailment. These particular searches, it seems, could be a huge part of the inaccuracy problem. There's another good reason to suspect this might be the case. In 2011, as part of one of its regular search algorithm tweaks, Google began recommending related search terms for many queries (including listing a search for flu treatments after someone Googled many flu-related terms) and in 2012, the company began providing potential diagnoses in response to symptoms in searches (including listing both "flu" and "cold" after a search that included the phrase "sore throat," for instance, perhaps prompting a user to search for how to distinguish between the two). These tweaks, the researchers argue, likely artificially drove up the rates of the searches they identified as responsible for Google's overestimates.
    via Boing Boing

    (tags: google flu trends feedback side-effects colds health google-flu-trends)

Links for 2014-03-13

Published March 13, 2014

Links for 2014-03-12

Published March 12, 2014

  • Sacked Google worker says staff ratings fixed to fit template

    Allegations of fixing to fit the stack-ranking curve: 'someone at Google always had to get a low score “of 2.9”, so the unit could match the bell curve. She said senior staff “calibrated” the ratings supplied by line managers to ensure conformity with the template and these calibrations could reduce a line manager’s assessment of an employee, in effect giving them the poisoned score of less than three.'

    (tags: stack-ranking google ireland employment work bell-curve statistics eric-schmidt)

  • Corporate Tax 2014: Irish Government's "flawed premise" on Apple's avoidance

    According to our calculation about €40bn or over 40% of Irish services exports of €90bn in 2012 and related national output, resulted from global tax avoidance schemes. It is true that Ireland gains little from tax cheating but at some point, the US tax system will be reformed and a territorial system where companies are only liable in the US on US profits, would only be viable if there was a disincentive to shift profits to non-tax or low tax countries. The risk for Ireland is that a minimum foreign tax would be introduced that would be greater than the Irish headline rate of 12.5%. It's also likely that US investment in Ireland would not have been jeopardized if Irish politicians had not been so eager as supplicants to doff the cap. Nevertheless today it would be taboo to admit the reality of participation in massive tax avoidance and the Captain Renaults of Merrion Street will continue with their version of the Dance of the Seven Veils.

    (tags: apple tax double-irish tax-avoidance google investment itax tax-evasion ireland)

  • An online Magna Carta: Berners-Lee calls for bill of rights for web

    TimBL backing the "web we want" campaign -- https://webwewant.org/

    (tags: freedom gchq nsa censorship internet privacy web-we-want human-rights timbl tim-berners-lee)

  • How the search for flight AF447 used Bayesian inference

    Via jgc, the search for the downed Air France flight was optimized using this technique: 'Metron’s approach to this search planning problem is rooted in classical Bayesian inference, which allows organization of available data with associated uncertainties and computation of the Probability Distribution Function (PDF) for target location given these data. In following this approach, the first step was to gather the available information about the location of the impact site of the aircraft. This information was sometimes contradictory and filled with ambiguities and uncertainties. Using a Bayesian approach we organized this material into consistent scenarios, quantified the uncertainties with probability distributions, weighted the relative likelihood of each scenario, and performed a simulation to produce a prior PDF for the location of the wreck.'

    (tags: metron bayes bayesian-inference machine-learning statistics via:jgc air-france disasters probability inference searching)

  • How the NSA Plans to Infect 'Millions' of Computers with Malware - The Intercept

    The implants being deployed were once reserved for a few hundred hard-to-reach targets, whose communications could not be monitored through traditional wiretaps. But the documents analyzed by The Intercept show how the NSA has aggressively accelerated its hacking initiatives in the past decade by computerizing some processes previously handled by humans. The automated system – codenamed TURBINE – is designed to “allow the current implant network to scale to large size (millions of implants) by creating a system that does automated control implants by groups instead of individually.” In a top-secret presentation, dated August 2009, the NSA describes a pre-programmed part of the covert infrastructure called the “Expert System,” which is designed to operate “like the brain.”
    Great. Automated malware deployment to millions of random victims. See also the "I hunt sysadmins" section further down...

    (tags: malware gchq nsa oversight infection expert-systems turbine false-positives the-intercept surveillance)

Links for 2014-03-11

Published March 11, 2014

Links for 2014-03-10

Published March 10, 2014

Links for 2014-03-06

Published March 6, 2014

Links for 2014-03-05

Published March 5, 2014

  • A cautionary tale about building large-scale polyglot systems

    'a fucking nightmare':

    Cascading requires a compilation step, yet since you're writing Ruby code, you get get none of the benefits of static type checking. It was standard to discover a type issue only after kicking off a job on, oh, 10 EC2 machines, only to have it fail because of a type mismatch. And user code embedded in strings would regularly fail to compile – which you again wouldn't discover until after your job was running. Each of these were bad individually, together, they were a fucking nightmare. The interaction between the code in strings and the type system was the worst of all possible worlds. No type checking, yet incredibly brittle, finicky and incomprehensible type errors at run time. I will never forget when one of my friends at Etsy was learning Cascading.JRuby and he couldn't get a type cast to work. I happened to know what would work: a triple cast. You had to cast the value to the type you wanted, not once, not twice, but THREE times.

    (tags: etsy scalding cascading adtuitive war-stories languages polyglot ruby java strong-typing jruby types hadoop)

  • It’s So Easy

    Attempting to cash out of Bitcoins turns out to be absurdly difficult:

    Trying to sell the coins in person, and basically saying he ether wants Cash, or a Cashiers check (since it can be handed over right then and there), has apparently been a hilarious clusterfuck. Today he met some guy infront of his bank, and apparently as soon as he mentioned that he needs to get the cash checked to make sure it is not counterfeit, the guy freaked out and basically walked away. Stuff like this has been happening all week, and he apparently so far has only sold a single coin of several hundred.

    (tags: bitcoin fail funny mtgox fraud cash fiat-currency via:rsynnott buttcoin)

  • Florida cops used IMSI catchers over 200 times without a warrant

    Harris is the leading maker of [IMSI catchers aka "stingrays"] in the U.S., and the ACLU has long suspected that the company has been loaning the devices to police departments throughout the state for product testing and promotional purposes. As the court document notes in the 2008 case, “the Tallahassee Police Department is not the owner of the equipment.” The ACLU now suspects these police departments may have all signed non-disclosure agreements with the vendor and used the agreement to avoid disclosing their use of the equipment to courts. “The police seem to have interpreted the agreement to bar them even from revealing their use of Stingrays to judges, who we usually rely on to provide oversight of police investigations,” the ACLU writes.

    (tags: aclu police stingrays imsi-catchers privacy cellphones mobile-phones security wired)

Links for 2014-03-04

Published March 4, 2014

Links for 2014-03-02

Published March 2, 2014

  • Answer to How many topics (queues) can be created in Apache Kafka? - Quora

    Good to know:

    'As far as I understand (this was true as of 2013, when I last looked into this issue) there's at least one Apache ZooKeeper znode per topic in Kafka. While there is no hard limitation in Kafka itself (Kafka is linearly scalable), it does mean that the maximum number of znodes comfortable supported by ZooKeeper (on the order of about ten thousand) is the upper limit of Kafka's scalability as far as the number of topics goes.'

    (tags: kafka queues zookeeper znodes architecture)

Links for 2014-03-01

Published March 1, 2014

  • Care.data is in chaos. It breaks my heart | Ben Goldacre

    There are people in my profession who think they can ignore this problem. Some are murmuring that this mess is like MMR, a public misunderstanding to be corrected with better PR. They are wrong: it's like nuclear power. Medical data, rarefied and condensed, presents huge power to do good, but it also presents huge risks. When leaked, it cannot be unleaked; when lost, public trust will take decades to regain. This breaks my heart. I love big medical datasets, I work on them in my day job, and I can think of a hundred life-saving uses for better ones. But patients' medical records contain secrets, and we owe them our highest protection. Where we use them – and we have used them, as researchers, for decades without a leak – this must be done safely, accountably, and transparently. New primary legislation, governing who has access to what, must be written: but that's not enough. We also need vicious penalties for anyone leaking medical records; and HSCIC needs to regain trust, by releasing all documentation on all past releases, urgently. Care.data needs to work: in medicine, data saves lives.

    (tags: hscic nhs care.data data privacy data-protection medicine hospitals pr)

Links for 2014-02-27

Published February 27, 2014

Links for 2014-02-23

Published February 23, 2014

Links for 2014-02-21

Published February 21, 2014

Links for 2014-02-20

Published February 20, 2014

Links for 2014-02-19

Published February 19, 2014

  • Belkin managed to put their firmware update private key in the distribution

    'The firmware updates are encrypted using GPG, which is intended to prevent this issue. Unfortunately, Belkin misuses the GPG asymmetric encryption functionality, forcing it to distribute the firmware-signing key within the WeMo firmware image. Most likely, Belkin intended to use the symmetric encryption with a signature and a shared public key ring. Attackers could leverage the current implementation to easily sign firmware images.' Using GPG to sign your firmware updates: yay. Accidentally leaving the private key in the distribution: sad trombone.

    (tags: fail wemo belkin firmware embedded-systems security updates distribution gpg crypto public-key pki home-automation ioactive)

  • Video Processing at Dropbox

    On-the-fly video transcoding during live streaming. They've done a great job of this!

    At the beginning of the development of this feature, we entertained the idea to simply pre-transcode all the videos in Dropbox to all possible target devices. Soon enough we realized that this simple approach would be too expensive at our scale, so we decided to build a system that allows us to trigger a transcoding process only upon user request and cache the results for subsequent fetches. This on-demand approach: adapts to heterogeneous devices and network conditions, is relatively cheap (everything is relative at our scale), guarantees low latency startup time.

    (tags: ffmpeg dropbox streaming video cdn ec2 hls http mp4 nginx haproxy aws h264)

Links for 2014-02-18

Published February 18, 2014

  • GPLv2 being tested in US court

    The case is still ongoing, so one to watch.

    Plaintiff wrote an XML parser and made it available as open source software under the GPLv2. Defendant acquired from another vendor software that included the code, and allegedly distributed that software to parties outside the organization. According to plaintiff, defendant did not comply with the conditions of the GPL, so plaintiff sued for copyright infringement. Defendants moved to dismiss for failure to state a claim. The court denied the motion.

    (tags: gpl open-source licensing software law legal via:fplogue)

  • Latest Snowden leak: GCHQ spying on Wikileaks users

    “How could targeting an entire website’s user base be necessary or proportionate?” says Gus Hosein, executive director of the London-based human rights group Privacy International. “These are innocent people who are turned into suspects based on their reading habits. Surely becoming a target of a state’s intelligence and security apparatus should require more than a mere click on a link.” The agency’s covert targeting of WikiLeaks, Hosein adds, call into question the entire legal rationale underpinning the state’s system of surveillance. “We may be tempted to see GCHQ as a rogue agency, ungoverned in its use of unprecedented powers generated by new technologies,” he says. “But GCHQ’s actions are authorized by [government] ministers. The fact that ministers are ordering the monitoring of political interests of Internet users shows a systemic failure in the rule of law."

    (tags: gchq wikileaks snowden privacy spying surveillance politics)

  • "Hackers" unsubscribed a former Mayor from concerned citizen's emails

    "The dog ate my homework, er, I mean, hackers hacked my account."

    Former Mayor of Kildare, Cllr. Michael Nolan, has denied a claim he asked a local campaigner to stop e-mailing him. Cllr. Michael Nolan from Newbridge said his site was hacked and wrong e-mails were sent out to a number of people, including Leixlip based campaigner, John Weigel. Mr. Weigel has been campaigning, along with others, about the danger of electromagnetic radiation to humans and the proximity of communications masts to homes and, in particular schools. He regularly updates local politicians on news items relating to the issue. Recently, he said that he had received an e-mail from Cllr. Nolan asking to be removed from Mr. Weigel’s e-mail list. The Leader asked Cllr. Nolan why he had done this. But the Fine Gael councillors said that “his e-mail account was hacked and on one particular day a number of mails a were sent from my account pertaining to be from me.”

    (tags: dog-ate-my-homework hackers funny kildare newbridge fine-gael michael-nolan email politics ireland excuses)

  • Making Remote Work Work

    very good, workable tips on how to remote-work effectively (both in the comments of this thread and the original article)

    (tags: tips productivity collaboration hn via:lhl remote-working telecommuting work)

  • Disgraced Scientist Granted U.S. Patent for Work Found to be Fraudulent - NYTimes.com

    Korean researcher Hwang Woo-suk electrified the science world 10 years ago with his claim that he had created the world’s first cloned human embryos and had extracted stem cells from them. But the work was later found to be fraudulent, and Dr. Hwang was fired from his university and convicted of crimes. Despite all that, Dr. Hwang has just been awarded an American patent covering the disputed work, leaving some scientists dumbfounded and providing fodder to critics who say the Patent Office is too lax. “Shocked, that’s all I can say,” said Shoukhrat Mitalipov, a professor at Oregon Health and Science University who appears to have actually accomplished what Dr. Hwang claims to have done. “I thought somebody was kidding, but I guess they were not.” Jeanne F. Loring, a stem cell scientist at the Scripps Research Institute in San Diego, said her first reaction was “You can’t patent something that doesn’t exist.” But, she said, she later realized that “you can.”

    (tags: patents absurd hwang-woo-suk cloning stem-cells science biology uspto)

Links for 2014-02-17

Published February 17, 2014

Links for 2014-02-16

Published February 16, 2014

  • About Ultima Ratio Regum

    This sounds amazing. I hope it makes it to some kind of "semi-finished".

    A semi-roguelike game inspired by Jorge Borges, Umberto Eco, Neal Stephenson, Shadow of the Colossus, Europa Universalis and Civilization. Although currently in its early stages, URR aims to explore several philosophical and sociological issues that both arose during the sixteenth and seventeenth century (when the game is approximately set), and in the present day, whilst almost being a deep, complex and highly challenging roguelike. To do this the game seeks to generate realistic world histories, though ones containing a few unusual happenings and anomalous experiences. The traditional roguelike staple of combat will be rare and deadly – whilst these mechanics will be modeled in detail, exploration, trade and diplomacy factors will have just as much effort put into them.

    (tags: games ultima-ratio-regum roguelikes borges umberto-eco worlds ascii-art)

Links for 2014-02-14

Published February 14, 2014

  • Beirtear na IMSIs: Ireland's GSOC surveillance inquiry reveals use of mobile phone interception systems | Privacy International

    It is interesting to note that the fake UK network was the only one detected by Verrimus. However, given that IMSI Catchers operate multiple fake towers simultaneously, it is highly likely that one or more Irish networks were also being intercepted. Very often a misconfiguration, such as an incorrect country code, is the only evidence available of an IMSI Catcher being deployed when forensic tools are not being used to look for one.

    (tags: privacy imsi-catchers surveillance bugging spying gsocgate gsoc ireland mobile-phones)

  • TCP incast vs Riak

    An extremely congested local network segment causes the "TCP incast" throughput collapse problem -- packet loss occurs, and TCP throughput collapses as a side effect. So far, this is pretty unsurprising, and anyone designing a service needs to keep bandwidth requirements in mind. However it gets worse with Riak. Due to a bug, this becomes a serious issue for all clients: the Erlang network distribution port buffers fill up in turn, and the Riak KV vnode process (in its entirety) will be descheduled and 'cannot answer any more queries until the A-to-B network link becomes uncongested.' This is where EC2's fully-uncontended-1:1-network compute cluster instances come in handy, btw. ;)

    (tags: incast tcp networking bandwidth riak architecture erlang buffering queueing)

  • Irish Law Society takes a stand for "brand owners IP rights"

    The Law Society will attend a meeting of the Oireachtas Health Committee today to outline its strong opposition to the Government proposals to introduce legislation that will require tobacco products to use plain packaging. The society’s director general Ken Murphy will be its principal representative at the meeting today to discuss its submission on the legislation, and to discuss its concerns that a plain packaging regime will undermine registered trade mark, and design, systems and will amount to an “expropriation of brand owners intellectual property rights’. Speaking ahead of the meeting, Mr Murphy told The Irish Times the views contained in it represent those of the Law Society as a whole, and its 10,000 members, and have been endorsed by the society as a whole, rather than the committee. Mr Murphy also said the purpose of the Law Society submission was not to protect the tobacco industry, rather the wider effect and impact such a law would have on intellectual property rights, trade marks, in other areas. “There is a real concern also that plain packaging in the tobacco industry is just the beginning of a trend that will severely undermine intellectual property owners’ rights in other sectors such as alcohol, soft drinks and fast foods.”
    Judging by some reactions on Twitter, "endorsed by the society as a whole" may be over-egging it a little.

    (tags: law-society gubu law ireland ip packaging branding trademarks cigarettes health tobacco)

  • British American Tobacco - Plain packaging of tobacco products

    Compare and contrast with the Law Society's comments:

    We believe we are entitled to use our packs to distinguish our products from those of our competitors. Our brands are our intellectual property which we have created and invested in. Plain packaging would deny us the right to use brands. But also, a brand is also an important tool for consumers. As the British Brands Group has stated  , plain packaging legislation "ignores the crucial role that branding plays in providing consumers with high quality, consistent products they can trust". The restriction of valuable corporate brands by any government would risk placing it in breach of legal obligations relating to intellectual property rights and, in most cases, international trade.

    (tags: law-society branding ip ireland tobacco cigarettes law trademarks)

  • Why dispute resolution is hard

    Good stuff (as usual) from Ross Anderson and Stephen Murdoch. 'Today we release a paper on security protocols and evidence which analyses why dispute resolution mechanisms in electronic systems often don’t work very well. On this blog we’ve noted many many problems with EMV (Chip and PIN), as well as other systems from curfew tags to digital tachographs. Time and again we find that electronic systems are truly awful for courts to deal with. Why? The main reason, we observed, is that their dispute resolution aspects were never properly designed, built and tested. The firms that delivered the main production systems assumed, or hoped, that because some audit data were available, lawyers would be able to use them somehow. As you’d expect, all sorts of things go wrong. We derive some principles, and show how these are also violated by new systems ranging from phone banking through overlay payments to Bitcoin. We also propose some enhancements to the EMV protocol which would make it easier to resolve disputes over Chip and PIN transactions.'

    (tags: finance security ross-anderson emv bitcoin chip-and-pin banking architecture verification vvat logging)

Links for 2014-02-13

Published February 13, 2014

Links for 2014-02-12

Published February 12, 2014

  • Migrating from MongoDB to Cassandra

    Interesting side-effect of using LUKS for full-disk encryption: 'For every disk read, we were pulling in 3MB of data (RA is sectors, SSZ is sector size, 6144*512=3145728 bytes) into cache. Oops. Not only were we doing tons of extra work, but we were trashing our page cache too. The default for the device-mapper used by LUKS under Ubuntu 12.04LTS is incredibly sub-optimal for database usage, especially our usage of Cassandra (more small random reads vs. large rows). We turned this down to 128 sectors — 64KB.'

    (tags: cassandra luks raid linux tuning ops blockdev disks sdd)

  • SpamAssassin 3.4.0 released

    Good to see the guys cracking on without me ;) '2014-02-11: SpamAssassin 3.4.0 has been released adding native support for IPv6, improved DNS Blocklist technology and support for massively-scalable Bayesian filtering using the Redis backend.'

    (tags: antispam open-source spamassassin apache)

  • 193_Cellxion_Brochure_UGX Series 330

    The Cellxion UGX Series 330 is a 'transportable Dual GSM/Triple UMTS Firewall and Analysis Tool' -- ie. an IMSI catcher in a briefcase, capable of catching IMSI/IMEIs in 3G. It even supports configurable signal strength. Made in the UK

    (tags: cellxion imsi-catchers imei surveillance gsocgate gsm 3g mobile-phones security spying)

Links for 2014-02-11

Published February 11, 2014

  • Trousseau

    'an interesting approach to a common problem, that of securely passing secrets around an infrastructure. It uses GPG signed files under the hood and nicely integrates with both version control systems and S3.' I like this as an approach to securely distributing secrets across a stack of services during deployment. Check in the file of keys, gpg keygen on the server, and add it to the keyfile's ACL during deployment. To simplify, shared or pre-generated GPG keys could also be used. (via the Devops Weekly newsletter)

    (tags: gpg encryption crypto secrets key-distribution pki devops deployment)

  • java - Why not use Double or Float to represent currency?

    A good canonical URL for this piece of coding guidance.

    For example, suppose you have $1.03 and you spend 42c. How much money do you have left? System.out.println(1.03 - .42); => prints out 0.6100000000000001.

    (tags: coding tips floating-point float java money currency bugs)

  • "I'm Sorry for what I said when I was Hungry" tee-shirt

    I can relate to this

    (tags: tee-shirts apparel etsy hangry)

  • "IMSI Catcher" used in London

    'One case involved Julian Assange's current home at the Ecuadorian Embassy in London, where visitors were surprised to receive welcome messages from a Ugandan telephone company. It turned out the messages were coming from a foreign base station device installed on the roof, masquerading as a cell tower for surveillance purposes. Appelbaum suspects the GCHQ simply forgot to reformat the device from an earlier Ugandan operation.'
    via T.J. McIntyre.

    (tags: surveillance nsa privacy imsi-catchers gchq london uganda mobile-phones julian-assange ecuador embassies)

  • The Spyware That Enables Mobile-Phone Snooping - Bloomberg

    More background on IMSI catchers -- looking likely to have been the "government-level technology" used to snoop on the Garda Ombudsman's offices, particularly given the 'detection of an unexpected UK 3G network near the GSOC offices':

    The technology involved is called cellular interception. The active variety of this, the “IMSI catcher,” is a portable device that masquerades as a mobile phone tower. Any phone within range (a mile for a low-grade IMSI catcher; as much as 100 miles for a passive interception device with a very large antenna, such as those used in India) automatically checks to see if the device is a tower operated by its carrier, and the false “tower” indicates that it is. It then logs the phone’s International Mobile Subscriber Identity number -- and begins listening in on its calls, texts and data communications. No assistance from any wireless carrier is needed; the phone has been tricked. [...] “network extender” devices -- personal mobile-phone towers -- sold by the carriers themselves, often called femtocells, can be turned into IMSI catchers.
    Via T.J. McIntyre

    (tags: via:tjmcintyre imsi-catchers surveillance privacy gsocgate mobile-phones spying imsi)

Links for 2014-02-10

Published February 10, 2014

  • Git is not scalable with too many refs/*

    Mailing list thread from 2011; git starts to keel over if you tag too much

    (tags: git tags coding version-control bugs scaling refs)

  • Survey results of EU teens using the internet

    A lot of unsupervised use:

    Just under half of children said they access the internet from their own bedroom on a daily basis with 22pc saying they do so several times a day.

    (tags: surveys eu ireland politics filtering internet social-media facebook children teens cyber-bullying)

  • Girls and Software

    a pretty thought-provoking article from Linux Journal on women in computing, and how we're doing it all wrong

    (tags: feminism community programming coding women computing software society work linux-journal children teaching)

  • Why Mt. Gox is full of shit

    leading Bitcoin exchange "Magic The Gatherine Online Exchange" turns out to suffer from crappy code, surprise:

    why does Mt. Gox experience this issue? They run a custom Bitcoin daemon, with a custom implementation of the Bitcoin protocol. Their implementation, against all advice, does rely on the transaction ID, which makes this attack possible. They have actually been warned about it months ago by gmaxwell, and have apparently decided to ignore this warning. In other words, this is not a vulnerability in the Bitcoin protocol, but an implementation error in Mt. Gox' custom Bitcoin software.
    The rest of the article is eyeopening, including the MySQL injection vulnerabilities and failure to correctly secure a Prolexic-defended server. https://news.ycombinator.com/item?id=7211286 has some other shocking reports of Bitcoin operators being incompetent, including 'Bitomat, the incompetent exchange that deleted their own [sole] amazon instance accidentally which contained all their keys, and thus customer funds'. wtfbbq

    (tags: mtgox security bitcoin standards omgwtfbbq via:hn bitomat)

  • Death by Metadata

    The side-effects of algorithmic false-positives get worse and worse.

    What’s more, he adds, the NSA often locates drone targets by analyzing the activity of a SIM card, rather than the actual content of the calls. Based on his experience, he has come to believe that the drone program amounts to little more than death by unreliable metadata. “People get hung up that there’s a targeted list of people,” he says. “It’s really like we’re targeting a cell phone. We’re not going after people – we’re going after their phones, in the hopes that the person on the other end of that missile is the bad guy.”

    (tags: false-positives glenn-greenwald drones nsa death-by-metadata us-politics terrorism sim-cards phones mobile-phones)

  • IBM's creepy AI cyberstalking plans

    'let's say that you tweet that you've gotten a job offer to move to San Francisco. Using IBM's linguistic analysis technologies, your bank would analyze your Twitter feed and not only tailor services it could offer you ahead of the move--for example, helping you move your account to another branch, or offering you a loan for a new house -- but also judge your psychological profile based upon the tone of your messages about the move, giving advice to your bank's representatives about the best way to contact you.'
    Ugh. Here's hoping they've patented this shit so we don't actually have to suffer through it. Creeeepy. (via Adam Shostack)

    (tags: datamining ai ibm stupid-ideas creepy stalking twitter via:adamshostack)

  • "A reason to hang him": how mass surveillance, secret courts, confirmation bias and the FBI can ruin your life - Boing Boing

    This is bananas. Confirmation bias running amok.

    Brandon Mayfield was a US Army veteran and an attorney in Portland, OR. After the 2004 Madrid train bombing, his fingerprint was partially matched to one belonging to one of the suspected bombers, but the match was a poor one. But by this point, the FBI was already convinced they had their man, so they rationalized away the non-matching elements of the print, and set in motion a train of events that led to Mayfield being jailed without charge; his home and office burgled by the FBI; his client-attorney privilege violated; his life upended.

    (tags: confirmation-bias bias law brandon-mayfield terrorism fingerprints false-positives fbi scary)

Links for 2014-02-09

Published February 9, 2014

  • A patent on 'Birth of a Child By Centrifugal Force'

    On November 9 1965, the Blonskys were granted US Patent 3,216,423, for an Apparatus for Facilitating the Birth of a Child by Centrifugal Force. The drawings, as well as the text, are a revelation. The Patent Office has them online at http://tinyurl.com/jd4ra and I urge you - if you have any shred of curiosity in your body - to look them up. For conceiving what appears to be the greatest labour-saving device ever invented, George and Charlotte Blonsky won the 1999 Ig Nobel Prize in the field of Managed Health Care.
    This is utterly bananas. (via christ)

    (tags: via:christ crazy patents 1960s centrifuge birth medicine ignobels)

Links for 2014-02-07

Published February 7, 2014

Links for 2014-02-06

Published February 6, 2014

  • Realtime water level data across Ireland

    Some very nice Dygraph-based time-series graphs in here, along with open CSV data. Good job!

    (tags: open-data water-levels time-series data rivers ireland csv)

  • The Gardai haven't requested info on any Twitter accounts in the past 6 months

    This seems to imply they haven't been investigating any allegations of cyber-bullying/harassment from "anonymous" Twitter handles, despite having the legal standing to do so. Enforcement is needed, not new laws

    (tags: cyber-bullying twitter social-media enforcement gardai policing harassment online society law government)

  • QuakeNet IRC Network- Article - PRESS RELEASE: IRC NETWORKS UNDER SYSTEMATIC ATTACK FROM GOVERNMENTS

    QuakeNet are not happy about GCHQ's DDoS attacks against them.

    Yesterday we learned ... that GCHQ, the British intelligence agency, are performing persistent social and technological attacks against IRC networks. These attacks are performed without informing the networks and are targeted at users associated with politically motivated movements such as "Anonymous". While QuakeNet does not condone or endorse and actively forbids any illegal activity on its servers we encourage discussion on all topics including political and social commentary. It is apparent now that engaging in such topics with an opinion contrary to that of the intelligence agencies is sufficient to make people a target for monitoring, coercion and denial of access to communications platforms. The ... documents depict GCHQ operatives engaging in social engineering of IRC users to entrap themselves by encouraging the target to leak details about their location as well as wholesale attacks on the IRC servers hosting the network. These attacks bring down the IRC network entirely affecting every user on the network as well as the company hosting the server. The collateral damage and numbers of innocent people and companies affected by these forms of attack can be huge and it is highly illegal in many jurisdictions including the UK under the Computer Misuse Act.

    (tags: quakenet ddos security gchq irc anonymous)

  • PID controller

    Good to know; this generic anti-flap damping algorithm has a name.

    A proportional-integral-derivative controller (PID controller) is a generic control loop feedback mechanism (controller) widely used in industrial control systems. A PID controller calculates an "error" value as the difference between a measured process variable and a desired setpoint. The controller attempts to minimize the error by adjusting the process control outputs.

    (tags: control damping flapping pid-controller industrial error algorithms)

Links for 2014-02-05

Published February 5, 2014

Links for 2014-02-04

Published February 4, 2014

  • Yahoo! moving EMEA operations to Dublin

    Like many companies, the structure of Yahoo's business is driven by the needs of the business. There are a number of factors which influence decisions about the locations in which the business operates. To encourage more collaboration and innovation, we’re increasing our headcount in Dublin, thus continuing to bring more Yahoos together in fewer locations. Dublin is already the European home to many of the world’s leading global technology brands and has been a home for Yahoo for over a decade already.
    Via Conor O'Neill

    (tags: via:conoro yahoo emea dublin ireland jobs tech)

Links for 2014-02-03

Published February 3, 2014

  • appear.in

    zero-install, one-click video chat, using WebRTC. nifty

    (tags: conference webrtc chat collaboration video google-chrome conferencing)

  • Opinion: How can we get over ‘Pantigate’?

    The fact that RTÉ had agreed to pay damages (€80,000 in total, according to reports yesterday) to the ‘injured parties’, only came to light in an email from the [far-right Catholic lobby group Iona Institute] to its members last Tuesday. Given the ramifications of the decision to make any kind of payment – regardless of the amount – both for the TV licence payer and those who voice contrarian opinions, the lack of coverage in print media as soon as the Iona email came to light marked a low point for print journalism in Ireland. Aside from a lead story on the damages printed in this paper last Wednesday and ongoing debate online, the media has been glacially slow with commentary and even reportage of the affair. The debacle has untold ramifications for public life in this country. That many liberal commentators may now baulk at the opportunity to speak and write openly and honestly about homophobia is the most obvious issue here. Most worrying of all, however, is the question that with a referendum on the introduction of gay marriage on the horizon, how can we expect the national broadcaster to facilitate even-handed debate on the subject when they’ve already found themselves cowed before reaching the first hurdle?

    (tags: homophobia politics ireland libel dissent lobbying defamation law gay-marriage iona-institute journalism newspapers)

  • Home · linkedin/rest.li Wiki

    Rest.li is a REST+JSON framework for building robust, scalable service architectures using dynamic discovery and simple asynchronous APIs. Rest.li fills a niche for building RESTful service architectures at scale, offering a developer workflow for defining data and REST APIs that promotes uniform interfaces, consistent data modeling, type-safety, and compatibility checked API evolution.
    The new underlying comms layer for Voldemort, it seems.

    (tags: voldemort d2 rest.li linkedin json rest http api frameworks java)

  • Hardened SSL Ciphers Using ELB and HAProxy

    ELBs support the PROXY protocol

    (tags: elb security proxying ssl tls https haproxy perfect-forward-secrecy aws ec2)

Links for 2014-02-01

Published February 1, 2014

Links for 2014-01-30

Published January 30, 2014

Links for 2014-01-29

Published January 29, 2014

Links for 2014-01-28

Published January 28, 2014

Links for 2014-01-27

Published January 27, 2014

  • Extending graphite’s mileage

    Ad company InMobi are using graphite heavily (albeit not as heavily as $work are), ran into the usual scaling issues, and chose to fix it in code by switching from a filesystem full of whisper files to a LevelDB per carbon-cache:

    The carbon server is now able to run without breaking a sweat even when 500K metrics per minute is being pumped into it. This has been in production since late August 2013 in every datacenter that we operate from.
    Very nice. I hope this gets merged/supported.

    (tags: graphite scalability metrics leveldb storage inmobi whisper carbon open-source)

  • BBC News - Pair jailed over abusive tweets to feminist campaigner

    When a producer from BBC Two's Newsnight programme tracked Nimmo down after he had sent the abuse, the former call centre worker told him: "The police will do nothing, it's only Twitter."

    (tags: bbc bullying social-media twitter society uk trolls trolling abuse feminism cyberbullying)

  • If You Used This Secure Webmail Site, the FBI Has Your Inbox

    TorMail was a Tor-based webmail system, and apparently its drives have been imaged and seized by the FBI. More info on the Freedom Hosting seizure:

    The connection, if any, between the FBI obtaining Freedom Hosting’s data and apparently launching the malware campaign through TorMail and the other sites isn’t spelled out in the new document. The bureau could have had the cooperation of the French hosting company that Marques leased his servers from. Or it might have set up its own Tor hidden services using the private keys obtained from the seizure, which would allow it to adopt the same .onion addresses used by the original sites. The French company also hasn’t been identified. But France’s largest hosting company, OVH, announced on July 29, in the middle of the FBI’s then-secret Freedom Hosting seizure, that it would no longer allow Tor software on its servers. A spokesman for the company says he can’t comment on specific cases, and declined to say whether Freedom Hosting was a customer. “Wherever the data center is located, we conduct our activities in conformity with applicable laws, and as a hosting company, we obey search warrants or disclosure orders,” OVH spokesman Benjamin Bongoat told WIRED. “This is all we can say as we usually don’t make any comments on hot topics.”

    (tags: fbi freedom-hosting hosting tor tormail seizures ovh colo servers)

  • Sky parental controls break many JQuery-using websites

    An 11 hour outage caused by a false positive in Sky's anti-phishing filter; all sites using the code.jquery.com CDN for JQuery would have seen errors.

    Sky still appears to be blocking code.jquery.com and all files served via the site, and more worryingly is that if you try to report the incorrect category, once signing in on the Sky website you an error page. We suspect the site was blocked due to being linked to by a properly malicious website, i.e. code.jquery.com and some javascript files were being used on a dodgy website and every domain mentioned was subsequently added to a block list.
    (via Tony Finch)

    (tags: via:fanf sky filtering internet uk anti-phishing phish jquery javascript http web fps false-positives)

  • Coders performing code reviews of scientific projects: pilot study

    'PLOS and Mozilla conducted a month-long pilot study in which professional developers performed code reviews on software associated with papers published in PLOS Computational Biology. While the developers felt the reviews were limited by (a) lack of familiarity with the domain and (b) lack of two-way contact with authors, the scientists appreciated the reviews, and both sides were enthusiastic about repeating the experiment. ' Actually sounds like it was more successful than this summary implies.

    (tags: plos mozilla code-reviews coding science computational-biology biology studies)

  • Caught with our Pantis down

    The views expressed by [the Iona Institute] – especially in relation to gay people – are very much at odds with the liberal secular society that Ireland has become. Indeed, Rory O’Neill suggested that the only time he experiences homophobia is online or at the hands of Iona and Waters. When they’re done with that, they can ask why Iona is given so much room in the media. In any other country in the world, an organisation as litigious as Iona would never be asked to participate in anything.

    (tags: homophobia ireland john-waters iona-institute politics catholicism religion libel defamation rte the-irish-times)

  • Scotch Whiskey flavour wheels

    mine's a Smoky/Spicy/Medicinal, thanks

    (tags: scotch whiskey whisky alcohol dataviz flavour)

Links for 2014-01-24

Published January 24, 2014

Links for 2014-01-22

Published January 22, 2014

  • Ukrainian government targeting protesters using threatening SMS messages

    The government’s opponents said three recent actions had been intended to incite the more radical protesters and sow doubt in the minds of moderates: the passing of laws last week circumscribing the right of public assembly, the blocking of a protest march past the Parliament building on Sunday, and the sending of cellphone messages on Tuesday to people standing in the vicinity of the fighting that said, “Dear subscriber, you are registered as a participant in a mass disturbance.” [....] The phrasing of the message, about participating in a “mass disturbance,” echoed language in a new law making it a crime to participate in a protest deemed violent. The law took effect on Tuesday. And protesters were concerned that the government seemed to be using cutting-edge technology from the advertising industry to pinpoint people for political profiling. Three cellphone companies in Ukraine — Kyivstar, MTS and Life — denied that they had provided the location data to the government or had sent the text messages, the newspaper Ukrainskaya Pravda reported. Kyivstar suggested that it was instead the work of a “pirate” cellphone tower set up in the area.

    (tags: targeting mobile-phones sms text-messaging via:tjmcintyre geotargeting protest ukraine privacy surveillance tech 1984)

  • UK porn filter blocks game update that contained 'sex' in URL

    Staggeringly inept. The UK national porn filter blocks based on a regexp match of the URL against /.*sex.*/i -- the good old "Scunthorpe problem". Better, it returns a 404 response. This is also a good demonstration of how web filtering has unintended side effects, breaking third-party software updates with its false positives.

    The update to online strategy game League of Legends was disrupted by the internet filter because the software attempted to access files that accidentally include the word “sex” in the middle of their file names. The block resulted in the update failing with “file not found” errors, which are usually created by missing files or broken updates on the part of the developers.

    (tags: uk porn filtering guardian regular-expressions false-positives scunthorpe http web league-of-legends sex)

  • Register article on Amazon's attitude to open source

    This article is frequently on target; this secrecy (both around open source and publishing papers) was one of the reasons I left Amazon.

    Of the sources with whom we spoke, many indicated that Amazon's lack of participation was a key reason for why people left the company – or never joined at all. This is why Amazon's strategy of maintaining secrecy may derail the e-retailer's future if it struggles to hire the best talent. [...] "In many cases in the big companies and all the small startups, your Github profile is your resume," explained another former Amazonian. "When I look at developers that's what I'm looking for, [but] they go to Amazon and that resume stops ... It absolutely affects the quality of their hires." "You had no portfolio you could share with the world," said another insider on life after working at Amazon. "The argument this was necessary to attract talent and to retain talent completely fell on deaf ears."

    (tags: amazon recruitment secrecy open-source hiring work research conferences)

  • Chinese Internet Traffic Redirected to Small Wyoming House

    'That address — which is home to some 2,000 companies on paper — was the subject of a lengthy 2011 Reuters investigation that found that among the entities registered to the address were a shell company controlled by a jailed former Ukraine prime minister; the owner of a company charged with helping online poker operators evade an Internet gambling ban; and one entity that was banned from government contracts after selling counterfeit truck parts to the Pentagon.'

    (tags: china internet great-firewall dns wyoming attacks security not-the-onion)

  • James Friend | PCE.js - Classic Mac OS in the Browser

    This is a demo of PCE's classic Macintosh emulation, running System 7.0.1 with MacPaint, MacDraw, and Kid Pix. If you want to try out more apps and games see this demo.
    Incredible. I remember using this version of MacPaint!

    (tags: javascript browser emulation mac macos macpaint macdraw claris kid-pix history desktop pce)

Links for 2014-01-21

Published January 21, 2014

  • likwid

    'Lightweight performance tools'.

    Likwid stands for 'Like I knew what I am doing'. This project contributes easy to use command line tools for Linux to support programmers in developing high performance multi-threaded programs. It contains the following tools: likwid-topology: Show the thread and cache topology likwid-perfctr: Measure hardware performance counters on Intel and AMD processors likwid-features: Show and Toggle hardware prefetch control bits on Intel Core 2 processors likwid-pin: Pin your threaded application without touching your code (supports pthreads, Intel OpenMP and gcc OpenMP) likwid-bench: Benchmarking framework allowing rapid prototyping of threaded assembly kernels likwid-mpirun: Script enabling simple and flexible pinning of MPI and MPI/threaded hybrid applications likwid-perfscope: Frontend for likwid-perfctr timeline mode. Allows live plotting of performance metrics. likwid-powermeter: Tool for accessing RAPL counters and query Turbo mode steps on Intel processor. likwid-memsweeper: Tool to cleanup ccNUMA memory domains.
    No kernel patching required. (via kellabyte)

    (tags: via:kellabyte linux performance testing perf likwid threading multithreading multicore mpi numa)

  • Backblaze Blog » What Hard Drive Should I Buy?

    Because Backblaze has a history of openness, many readers expected more details in my previous posts. They asked what drive models work best and which last the longest. Given our experience with over 25,000 drives, they asked which ones are good enough that we would buy them again. In this post, I’ll answer those questions.

    (tags: backblaze backup hardware hdds storage disks ops via:fanf)

Links for 2014-01-20

Published January 20, 2014

Links for 2014-01-17

Published January 17, 2014

  • Transport Minister planning to make hi-vis jackets mandatory for cyclists

    The minister also spoke of a number of new transport initiatives, such  as mandatory use of high visibility jackets by cyclists.

    (tags: cycling safety law ireland leo-varadkar)

  • The Malware That Duped Target Has Been Found

    a Windows 'RAM scraper' trojan known as Trojan.POSRAM, which was used to attack the Windows-based point-of-sales systems which the POS terminals are connected to. part of an operation called Kaptoxa. 'The code is based on a previous malicious tool known as BlackPOS that is believed to have been developed in 2013 in Russia, though the new variant was highly customized to prevent antivirus programs from detecting it' ... 'The tool monitors memory address spaces used by specific programs, such as payment application programs like pos.exe and PosW32.exe that process the data embossed in the magnetic strip of credit and debit cards data. The tool grabs the data from memory.' ... 'The siphoned data is stored on the system, and then every seven hours the malware checks the local time on the compromised system to see if it’s between the hours of 10 a.m. and 5 p.m. If so, it attempts to send the data over a temporary NetBIOS share to an internal host inside the compromised network so the attackers can then extract the data over an FTP ... connection.' http://www.pcworld.com/article/2088920/target-credit-card-data-was-sent-to-server-in-russia.html says the data was then transmitted to another US-based server, and from there relayed to Russia, and notes: 'At the time of its discovery, Trojan.POSRAM “had a zero percent antivirus detection rate, which means that fully updated antivirus engines on fully patched computers could not identify the software as malicious,” iSight said.' Massive AV fail.

    (tags: kaptoxa trojans ram-scrapers trojan.posram posram point-of-sale security hacks target credit-cards pin ftp netbios smb)

  • Full iSight report on the Kaptoxa attack on Target

    'POS malware is becoming increasingly available to cyber criminals' ... 'there is growing demand for [this kind of malware]'. Watch your credit cards...

    (tags: debit-cards credit-cards security card-present attacks kaptoxa ram-scrapers trojans point-of-sale pos malware target)

  • The Target hack and PCI-DSS

    Both Heartland Payment Systems and Hannaford Bros. were in fact certified PCI-compliant while the hackers were in their system. In August 2006, Wal-Mart was also certified PCI-compliant while unknown attackers were lurking on its network. [...] “This PCI standard just ain’t working,” says Litan, the Gartner analyst. “I wouldn’t say it’s completely pointless. Because you can’t say security is a bad thing. But they’re trying to patch a really weak [and] insecure payment system [with it].”
    Basically, RAM scrapers have been in use in live attacks, sniffing credentials in the clear, since 2007. Ouch.

    (tags: ram-scrapers trojans pins pci-dss compliance security gartner walmart target)

  • ISPAI responds to TD Patrick O'Donovan's bizarre comments regarding "open source browsers"

    ISPAI is rather dismayed and somewhat confused by the recent press release issued by Deputy Patrick O’Donovan (FG). He appears to be asking the Oireachtas Communications Committee (of which he is a member) to investigate: “the matter of tougher controls on the use of open source internet browsers and payment systems”  which he claims “allow users to remain anonymous for illegal trade of drugs weapons and pornography.” Deputy O’Donovan would do well to ask the advice of industry experts on these matters given that legislating to curtail the use of such legitimate software or services, which may be misused by some, is neither practical nor logical. Whether or not a browser is open source bears no relevance to its ability to be the subject of anonymous use. Indeed, Deputy O’Donovan must surely be confusing and conflating different technical concepts? In tracing illegal activities, Law Enforcement Agencies and co-operating parties will use IP addresses – users’ choice of browser has little relevance to an investigation of criminal activity. Equally, it may be that the Deputy is uncomfortable with the concept of electronic payment systems but these underpin the digital economy which is bringing enormous benefit to Ireland. Yes, these may be misused by criminals but so are cash and traditional banking services. Restricting the growth of innovative financial services is not the solution to tackling cyber criminals who might be operating what he describes as “online supermarkets for illegal goods.” Tackling international cybercrime requires more specialist Law Enforcement resources at national level and improved international police cooperation supported by revision of EU legislation relating to obtaining server log evidence existing in other jurisdictions.

    (tags: ispai open-source patrick-o-donovan fine-gael press-releases tor darknet crime)

Links for 2014-01-16

Published January 16, 2014

Don’t use Timers with exponentially-decaying reservoirs in Graphite

Published January 16, 2014

A common error when using the Metrics library is to record Timer metrics on things like API calls, using the default settings, then to publish those to a time-series store like Graphite. Here's why this is a problem.

By default, a Timer uses an Exponentially Decaying Reservoir. The docs say:

'A histogram with an exponentially decaying reservoir produces quantiles which are representative of (roughly) the last five minutes of data. It does so by using a forward-decaying priority reservoir with an exponential weighting towards newer data. Unlike the uniform reservoir, an exponentially decaying reservoir represents recent data, allowing you to know very quickly if the distribution of the data has changed.'

This is more-or-less correct -- but the key phrase is 'roughly'. In reality, if the frequency of updates to such a timer drops off, it could take a lot longer, and if you stop updating a timer which uses this reservoir type, it'll never decay at all. The GraphiteReporter will dutifully capture the percentiles, min, max, etc. from that timer's reservoir every minute thereafter, and record those to Graphite using the current timestamp -- even though the data it was derived from is becoming more and more ancient.

Here's a demo. Note the long stretch of 800ms 99th-percentile latencies on the green line in the middle of this chart:

However, the blue line displays the number of events. As you can see, there were no calls to this API for that 8-hour period -- this one was a test system, and the user population was safely at home, in bed. So while Graphite is claiming that there's an 800ms latency at 7am, in reality the 800ms-latency event occurred 8 hours previously.

I observed the same thing in our production systems for various APIs which suffered variable invocation rates; if rates dropped off during normal operation, the high-percentile latencies hung around for far longer than they should have. This is quite misleading when you're looking at a graph for 10pm and seeing a high 99th-percentile latency, when the actual high-latency event occurred hours earlier. On several occasions, this caused lots of user confusion and FUD with our production monitoring, so we needed to fix it.

Here are some potential fixes.

  • Modify ExponentiallyDecayingReservoir to also call rescaleIfNeeded() inside getSnapshot() -- but based on this discussion, it appears the current behaviour is intended (at least for the mean measurement), so that may not be acceptable. Another risk of this is that it leaves us in a position where the percentiles displayed for time T may actually have occurred several minutes prior to that, which is still misleading (albeit less so).

  • Switch to sliding time window reservoirs, but those are unbounded in size -- so a timer on an unexpectedly-popular API could create GC pressure and out-of-memory scenarios. It's also the slowest reservoir type, according to the docs. That made it too risky for us to adopt in our production code as a general-purpose Timer implementation.

  • Update, Dec 2017: as of version 3.2.3 of Dropwizard Metrics, there is a new SlidingTimeWindowArrayReservoir reservoir implementation, which is a drop-in replacement for SlidingTimeWindowReservoir, with much more acceptable memory footprint and GC impact. It costs roughly 128 bits per stored measurement, and is therefore judged to be 'comparable with ExponentiallyDecayingReservoir in terms of GC overhead and performance'. (thanks to Bogdan Storozhuk for the tip)

  • What we eventually did in our code was to use this Reporter class instead of GraphiteReporter; it clears all Timer metrics' reservoirs after each write to Graphite. This is dumb and dirty, reaching across logical class boundaries, but at the same time it's simple and comprehensible behaviour: with this, we can guarantee that the percentile/min/max data recorded at timestamp T is measuring events in that timestamp's 1-minute window -- not any time before that. This is exactly what you want to see in a time-series graph like those in Graphite, so is a very valuable feature for our metrics, and one that others have noted to be important in comparable scenarios elsewhere.

Here's an example of what a graph like the above should look like (captured from our current staging stack):

Note that when there are no invocations, the reported 99th-percentile latency is 0, and each measurement doesn't stick around after its 1-minute slot.

Another potential bug fix for a related issue, would be to add support to Metrics so that it can use Gil Tene's LatencyUtils package, and its HdrHistogram class, as a reservoir. (Update: however, I don't think this would address the "old data leaking into newer datapoints" problem as fully.) This would address some other bugs in the Exponentially Decaying Reservoir, as Gil describes:

'In your example of a system logging 10K operations/sec with the histogram being sampled every second, you'll be missing 9 out of each 10 actual outliers. You can have an outlier every second and think you have one roughly every 10. You can have a huge business affecting outlier happening every hour, and think that they are only occurring once a day.'

Eek.

Links for 2014-01-15

Published January 15, 2014

Links for 2014-01-14

Published January 14, 2014

Links for 2014-01-13

Published January 13, 2014

Links for 2014-01-11

Published January 11, 2014

  • Growing up unvaccinated: A healthy lifestyle couldn’t prevent many childhood illnesses.

    I understand, to a point, where the anti-vaccine parents are coming from. Back in the ’90s, when I was a concerned, 19-year-old mother, frightened by the world I was bringing my child into, I was studying homeopathy, herbalism, and aromatherapy; I believed in angels, witchcraft, clairvoyants, crop circles, aliens at Nazca, giant ginger mariners spreading their knowledge to the Aztecs, the Incas, and the Egyptians, and that I was somehow personally blessed by the Holy Spirit with healing abilities. I was having my aura read at a hefty price and filtering the fluoride out of my water. I was choosing to have past life regressions instead of taking antidepressants. I was taking my daily advice from tarot cards. I grew all my own veg and made my own herbal remedies. I was so freaking crunchy that I literally crumbled. It was only when I took control of those paranoid thoughts and fears about the world around me and became an objective critical thinker that I got well. It was when I stopped taking sugar pills for everything and started seeing medical professionals that I began to thrive physically and mentally.

    (tags: health medicine science vaccination disease slate)

Links for 2014-01-09

Published January 9, 2014

Links for 2014-01-07

Published January 7, 2014

Links for 2014-01-06

Published January 6, 2014

Links for 2014-01-02

Published January 2, 2014

  • Dogs like to excrete in alignment with the Earth's magnetic field

    Dogs preferred to excrete with the body being aligned along the North-south axis under calm magnetic field conditions.

    (tags: dogs poo excrement shit magnetic-field earth zoology papers)

  • Paul Graham and the Manic Pixie Dream Hacker

    Under Graham’s influence, Mark [Zuckerberg], like many in Silicon Valley, subscribes to the Manic Pixie Dream Hacker ideal, making self-started teenage hackers Facebook’s most desired recruiting targets, not even so much for their coding ability as their ability to serve as the faces of hacking culture. “Culture fit”, in this sense, is one’s ability to conform to the Valley’s boyish hacker fantasy, which is easier, obviously, the closer you are to a teenage boy. Like the Manic Pixie Dream Girl’s role of existing to serve the male film protagonist’s personal growth, the Manic Pixie Dream Hacker’s job is to embody the dream hacker role while growing the VC’s portfolio. This is why the dream hacker never ages, never visibly develops interests beyond hardware and code, and doesn’t question why nearly all the other people receiving funding look like him. Like the actress playing the pixie dream girl, the pixie dream boy isn’t being paid to question the role for which he has been cast. In this way, for all his supposed “disruptiveness”, the hacker pixie actually does exactly what he is told: to embody, while he can, the ideal hacker, until he is no longer young, mono-focused, and boyish-seeming enough to qualify for the role (at that point, vested equity may allow him to retire). And like in Hollywood, VCs will have already recruited newer, younger ones to play him.

    (tags: hackers manic-pixie-dream-girl culture-fit silicon-valley mark-zuckerberg paul-graham y-combinator vc work investment technology recruitment facebook ageism equality sexism)

  • The How and Why of Flapjack

    Flapjack aims to be a flexible notification system that handles: Alert routing (determining who should receive alerts based on interest, time of day, scheduled maintenance, etc); Alert summarisation (with per-user, per media summary thresholds); Your standard operational tasks (setting scheduled maintenance, acknowledgements, etc). Flapjack sits downstream of your check execution engine (like Nagios, Sensu, Icinga, or cron), processing events to determine if a problem has been detected, who should know about the problem, and how they should be told.

    (tags: flapjack notification alerts ops nagios paging sensu)