Skip to content

Category: Uncategorized

Links for 2015-04-06

Published April 6, 2015

Links for 2015-04-03

Published April 3, 2015

  • Twitter’s new anti-harassment filter

    Twitter is calling it a “quality filter,” and it’s been rolling out to verified users running Twitter’s iOS app since last week. It appears to work much like a spam filter, except instead of hiding bots and copy-paste marketers, it screens “threats, offensive language, [and] duplicate content” out of your notifications feed.
    via Nelson

    (tags: via:nelson harassment spam twitter gamergame abuse ml)

  • 5% of Google visitors have ad-injecting malware installed

    Ad injectors were detected on all operating systems (Mac and Windows), and web browsers (Chrome, Firefox, IE) that were included in our test. More than 5% of people visiting Google sites have at least one ad injector installed. Within that group, half have at least two injectors installed and nearly one-third have at least four installed.
    via Nelson.

    (tags: via:nelson ads google chrome ad-injectors malware scummy)

  • On Ruby

    The horrors of monkey-patching:

    I call out the Honeybadger gem specifically because was the most recent time I'd been bit by a seemingly good thing promoted in the community: monkey patching third party code. Now I don't fault Honeybadger for making their product this way. It provides their customers with direct business value: "just require 'honeybadger' and you're done!" I don't agree with this sort of practice. [....] I distrust everything [in Ruby] but a small set of libraries I've personally vetted or are authored by people I respect. Why is this important? Without a certain level of scrutiny you will introduce odd and hard to reproduce bugs. This is especially important because Ruby offers you absolutely zero guarantee whatever the state your program is when a given method is dispatched. Constants are not constants. Methods can be redefined at run time. Someone could have written a time sensitive monkey patch to randomly undefined methods from anything in ObjectSpace because they can. This example is so horribly bad that no one should every do, but the programming language allows this. Much worse, this code be arbitrarily inject by some transitive dependency (do you even know what yours are?).

    (tags: ruby monkey-patching coding reliability bugs dependencies libraries honeybadger sinatra)

Links for 2015-04-02

Published April 2, 2015

Links for 2015-04-01

Published April 1, 2015

Links for 2015-03-31

Published March 31, 2015

Links for 2015-03-30

Published March 30, 2015

Links for 2015-03-26

Published March 26, 2015

Links for 2015-03-25

Published March 25, 2015

Links for 2015-03-24

Published March 24, 2015

Links for 2015-03-23

Published March 23, 2015

Links for 2015-03-22

Published March 22, 2015

Links for 2015-03-19

Published March 19, 2015

  • Stairs to nowhere, trap streets, and other Toronto oddities

    'There’s a set of stairs on Greenwood Avenue that lead nowhere. At the top, a wooden fence at the end of someone’s back yard blocks any further movement, forcing the climber to turn around and descend back to the street. What’s remarkable about the pointless Greenwood stairs, which were built in 1959 as a shortcut to a now-demolished brickyard, is that someone still routinely maintains them: in winter, some kindly soul deposits a scattering of salt lest one of the stairs’ phantom users slip; in summer someone comes with a broom to sweep away leaves. These urban leftovers are lovingly called “Thomassons” after Gary Thomasson, a former slugger for the San Francisco Giants, Oakland As, Yankees, Dodgers, and, most fatefully, the Yomiuri Giants in Tokyo.'

    (tags: trap-streets maps ip google via:bldgblog mapping copyright thomassons orphaned-roads)

Links for 2015-03-18

Published March 18, 2015

  • President's message gets lost in (automated) translation

    In a series of bizarre translations, YouTube’s automated translation service took artistic licence with the [President's] words of warmth. When the head of state sent St Patrick’s Day greetings to viewers, the video sharing site said US comedian Tina Fey was being “particular with me head”. As President Higgins spoke of his admiration for Irish emigrants starting new communities abroad, YouTube said the President referenced blackjack and how he “just couldn’t put the new iPhone” down. And, in perhaps the most unusual moment, as he talked of people whose hearts have sympathy, the President “explained” he was once on a show “that will bar a gift card”.
    (via Daragh O'Brien)

    (tags: lol president ireland michael-d-higgins automation translation machine-learning via:daraghobrien funny blackjack iphone tina-fey st-patrick fail)

  • Irish government under fire for turning its back on basic research : Nature News & Comment

    Pretty much ALL of Ireland's research scientists have put their names to an open letter to the Irish government, decrying the state of science funding, published this week in "Nature". 'Although total spending on research and development grew through the recession, helped by foreign investments, Ireland’s government has cut state spending on research (see ‘Celtic tiger tamed’). It also prioritized grants in 14 narrow areas — ones in which either large global markets exist, or in which Irish companies are competitive. These include marine renewable energy, smart grids, medical devices and computing. The effect has been to asphyxiate the many areas of fundamental science — including astrophysics, particle physics and areas of the life sciences — that have been deprived of funding, several researchers in Ireland told Nature. “The current policies are having a very significant detrimental effect on the health and viability of the Irish scientific ecosystem,” says Kevin Mitchell, a geneticist who studies the basis of neurological disorders at Trinity College Dublin. “Research that cannot be shoehorned into one of the 14 prioritized areas has been ineligible for most funding,” he says.' That's another fine mess Sean Sherlock has gotten us into :(

    (tags: sean-sherlock fail ireland research government funding grants science tcd kevin-mitchell life-sciences nature)

  • Mars One finalist Dr. Joseph Roche rips into the project

    So, here are the facts as we understand them: Mars One has almost no money. Mars One has no contracts with private aerospace suppliers who are building technology for future deep-space missions. Mars One has no TV production partner. Mars One has no publicly known investment partnerships with major brands. Mars One has no plans for a training facility where its candidates would prepare themselves. Mars One’s candidates have been vetted by a single person, in a 10-minute Skype interview. “My nightmare about it is that people continue to support it and give it money and attention, and it then gets to the point where it inevitably falls on its face,” said Roche. If, as a result, “people lose faith in NASA and possibly even in scientists, then that’s the polar opposite of what I’m about. If I was somehow linked to something that could do damage to the public perception of science, that is my nightmare scenario.”

    (tags: science space mars-one tcd joseph-roche nasa mars exploration scams)

Links for 2015-03-17

Published March 17, 2015

Links for 2015-03-13

Published March 13, 2015

  • demonstration of the importance of server-side request timeouts

    from MongoDB, but similar issues often apply in many other TCP/HTTP-based systems

    (tags: tcp http requests timeout mongodb reliability safety)

  • Heka

    an open source stream processing software system developed by Mozilla. Heka is a “Swiss Army Knife” type tool for data processing, useful for a wide variety of different tasks, such as: Loading and parsing log files from a file system. Accepting statsd type metrics data for aggregation and forwarding to upstream time series data stores such as graphite or InfluxDB. Launching external processes to gather operational data from the local system. Performing real time analysis, graphing, and anomaly detection on any data flowing through the Heka pipeline. Shipping data from one location to another via the use of an external transport (such as AMQP) or directly (via TCP). Delivering processed data to one or more persistent data stores.
    Via feylya on twitter. Looks potentially nifty

    (tags: heka mozilla monitoring metrics via:feylya ops statsd graphite stream-processing)

Links for 2015-03-12

Published March 12, 2015

Links for 2015-03-11

Published March 11, 2015

Links for 2015-03-10

Published March 10, 2015

  • Epsilon Interactive breach the Fukushima of the Email Industry (CAUCE)

    Upon gaining access to an ESP, the criminals then steal subscriber data (PII such as names, addresses, telephone numbers and email addresses, and in one case, Vehicle Identification Numbers). They then use ESPs’ mailing facility to send spam; to monetize their illicit acquisition, the criminals have spammed ads for fake Adobe Acrobat and Skype software. On March 30, the Epsilon Interactive division of Alliance Data Marketing (ADS on NASDAQ) suffered a massive breach that upped the ante, substantially.  Email lists of at least eight financial institutions were stolen.  Thus far, puzzlingly, Epsilon has refused to release the names  of compromised clients. [...] The obvious issue at hand is the ability of the thieves to now undertake targeted spear-phishing problem as critically serious as it could possibly be.

    (tags: cauce epsilon-interactive esp email pii data-protection spear-phishing phishing identity-theft security ads)

  • In Ukraine, Tomorrow’s Drone War Is Alive Today

    Drones, hackerspaces and crowdfunding:

    The most sophisticated UAV that has come out of the Ukrainian side since the start of the conflict is called the PD-1 from developer Igor Korolenko. It has a wingspan of nearly 10 feet, a five-hour flight time, carries electro-optical and infrared sensors as well as a video camera that broadcasts on a 128 bit encrypted channel. Its most important feature is the autopilot software that allows the drone to return home in the event that the global positioning system link is jammed or lost. Drone-based intelligence gathering is often depicted as risk-free compared to manned aircraft or human intelligence gathering, but, says Korolenko, if the drone isn’t secure or the signature is too obvious, the human coasts can be very, very high. “Russian military sometimes track locations of ground control stations,” he wrote Defense One in an email. “Therefore UAV squads have to follow certain security measures - to relocate frequently, to move out antennas and work from shelter, etc. As far as I know, two members of UAV squads were killed from mortar attacks after [their] positions were tracked by Russian electronic warfare equipment.”
    (via bldgblog)

    (tags: via:bldgblog war drones uav future ukraine russia tech aircraft pd-1 crowdfunding)

  • Javascript Acid Machine

    a 303 and an 808 in your browser. this is deadly

    (tags: acid 303 music javascript hacks via:hn techno)

Links for 2015-03-09

Published March 9, 2015

  • Ubuntu To Officially Switch To systemd Next Monday - Slashdot

    Jesus. This is going to be the biggest shitfest in the history of Linux...

    (tags: linux slashdot ubuntu systemd init unix ops)

  • uselessd

    A project to reduce systemd to a base initd, process supervisor and transactional dependency system, while minimizing intrusiveness and isolationism. Basically, it’s systemd with the superfluous stuff cut out, a (relatively) coherent idea of what it wants to be, support for non-glibc platforms and an approach that aims to minimize complicated design. uselessd is still in its early stages and it is not recommended for regular use or system integration.
    This may be the best option to evade the horrors of systemd.

    (tags: init linux systemd unix ops uselessd)

  • Japan's Robot Dogs Get Funerals as Sony Looks Away

    in July 2014, [Sony's] repairs [of Aibo robot dogs] stopped and owners were left to look elsewhere for help. The Sony stiff has led not only to the formation of support groups--where Aibo enthusiasts can share tips and help each other with repairs--but has fed the bionic pet vet industry. “The people who have them feel their presence and personality,” Nobuyuki Narimatsu, director of A-Fun, a repair company for robot dogs, told AFP. “So we think that somehow, they really have souls.” While concerted repair efforts have kept many an Aibo alive, a shortage of spare parts means that some of their lives have come to an end.

    (tags: sony aibo robots japan dogs pets weird future badiotday iot gadgets)

  • "Cuckoo Filter: Practically Better Than Bloom"

    'We propose a new data structure called the cuckoo filter that can replace Bloom filters for approximate set membership tests. Cuckoo filters support adding and removing items dynamically while achieving even higher performance than Bloom filters. For applications that store many items and target moderately low false positive rates, cuckoo filters have lower space overhead than space-optimized Bloom filters. Our experimental results also show that cuckoo filters outperform previous data structures that extend Bloom filters to support deletions substantially in both time and space.'

    (tags: algorithms paper bloom-filters cuckoo-filters cuckoo-hashing data-structures false-positives big-data probabilistic hashing set-membership approximation)

  • Amazing cutting from Vanity Fair, 1896, for International Women's Day

    "The sisters make a pretty picture on the platform ; but it is not women of their type who need to assert themselves over Man. However, it amuses them--and others ; and I doubt if the tyrant has much to fear from their little arrows." Constance Markievicz was one of those sisters, and the other was Eva Gore-Booth.

    (tags: markievicz history ireland sligo vanity-fair 19th-century dismissal sexism iwd women)

  • Anatomy of a Hack

    Authy doesn't come off well here: 'Authy should have been harder to break. It's an app, like Authenticator, and it never left Davis' phone. But Eve simply reset the app on her phone using a mail.com address and a new confirmation code, again sent by a voice call. A few minutes after 3AM, the Authy account moved under Eve's control.'

    (tags: authy security hacking mfa authentication google apps exploits)

  • Ask the Decoder: Did I sign up for a global sleep study?

    How meaningful is this corporate data science, anyway? Given the tech-savvy people in the Bay Area, Jawbone likely had a very dense sample of Jawbone wearers to draw from for its Napa earthquake analysis. That allowed it to look at proximity to the epicenter of the earthquake from location information. Jawbone boasts its sample population of roughly “1 million Up wearers who track their sleep using Up by Jawbone.” But when looking into patterns county by county in the U.S., Jawbone states, it takes certain statistical liberties to show granularity while accounting for places where there may not be many Jawbone users. So while Jawbone data can show us interesting things about sleep patterns across a very large population, we have to remember how selective that population is. Jawbone wearers are people who can afford a $129 wearable fitness gadget and the smartphone or computer to interact with the output from the device. Jawbone is sharing what it learns with the public, but think of all the public health interests or other third parties that might be interested in other research questions from a large scale data set. Yet this data is not collected with scientific processes and controls and is not treated with the rigor and scrutiny that a scientific study requires. Jawbone and other fitness trackers don’t give us the option to use their devices while opting out of contributing to the anonymous data sets they publish. Maybe that ought to change.

    (tags: jawbone privacy data-protection anonymization aggregation data medicine health earthquakes statistics iot wearables)

  • Pinterest's highly-available configuration service

    Stored on S3, update notifications pushed to clients via Zookeeper

    (tags: s3 zookeeper ha pinterest config storage)

  • A Journey into Microservices | Hailo Tech Blog

    Excellent three-parter from Hailo, describing their RabbitMQ+Go-based microservices architecture. Very impressive!

    (tags: hailo go microservices rabbitmq amqp architecture blogs)

  • soundcloud/lhm

    The Large Hadron Migrator is a tool to perform live database migrations in a Rails app without locking.

    The basic idea is to perform the migration online while the system is live, without locking the table. In contrast to OAK and the facebook tool, we only use a copy table and triggers. The Large Hadron is a test driven Ruby solution which can easily be dropped into an ActiveRecord or DataMapper migration. It presumes a single auto incremented numerical primary key called id as per the Rails convention. Unlike the twitter solution, it does not require the presence of an indexed updated_at column.

    (tags: migrations database sql ops mysql rails ruby lhm soundcloud activerecord)

  • Biased Locking in HotSpot (David Dice's Weblog)

    This is pretty nuts. If biased locking in the HotSpot JVM is causing performance issues, it can be turned off:

    You can avoid biased locking on a per-object basis by calling System.identityHashCode(o). If the object is already biased, assigning an identity hashCode will result in revocation, otherwise, the assignment of a hashCode() will make the object ineligible for subsequent biased locking.

    (tags: hashcode jvm java biased-locking locking mutex synchronization locks performance)

Links for 2015-03-07

Published March 7, 2015

  • A Zero-Administration Amazon Redshift Database Loader - AWS Big Data Blog

    nifty!

    (tags: lambda amazon aws redshift etl)

  • Archie Markup Language (ArchieML)

    ArchieML (or "AML") was created at The New York Times to make it easier to write and edit structured text on deadline that could be rendered in web pages, or more specifically, rendered in interactive graphics. One of the main goals was to make it easy to tag text as data, without having type a lot of special characters. Another goal was to allow the document to contain lots of notes and draft text that would not be read into the data. And finally, because we make extensive use of Google Documents's concurrent-editing features — while working on a graphic, we can have several reporters, editors and developers all pouring information into a single document — we wanted to have a format that could survive being edited by users who may never have seen ArchieML or any other markup language at all before.

    (tags: aml archie markup text nytimes archieml writing)

  • California Says Motorcycle Lane-Splitting Is Hella Safe

    A recent yearlong study by the California Office of Traffic Safety has found motorcycle lane-splitting to be a safe practice on public roads. The study looked at collisions involving 7836 motorcyclists reported by 80 police departments between August 2012 and August 2013. “What we learned is, if you lane-split in a safe or prudent manner, it is no more dangerous than motorcycling in any other circumstance,” state spokesman Chris Cochran told the Sacramento Bee. “If you are speeding or have a wide speed differential (with other traffic), that is where the fatalities came about.”

    (tags: lane-splitting cycling motorcycling bikes road-safety driving safety california)

  • Try Server

    Good terminology for this concept:

    The try server runs a similar configuration to the continuous integration server, except that it is triggered not on commits but on "try job request", in order to test code pre-commit.
    See also https://wiki.mozilla.org/ReleaseEngineering/TryServer for the Moz take on it.

    (tags: build ci integration try-server jenkins buildbot chromium development)

  • metrics-sql

    A Dropwizard Metrics extension to instrument JDBC resources and measure SQL execution times.

    (tags: metrics sql jdbc instrumentation dropwizard)

  • HP is trying to patent Continuous Delivery

    This is appalling bollocks from HP:

    On 1st March 2015 I discovered that in 2012 HP had filed a patent (WO2014027990) with the USPO for ‘Performance tests in a continuous deployment pipeline‘ (the patent was granted in 2014). [....] HP has filed several patents covering standard Continuous Delivery (CD) practices. You can help to have these patents revoked by providing ‘prior art’ examples on Stack Exchange.
    In fairness, though, this kind of shit happens in most big tech companies. This is what happens when you have a broken software patenting system, with big rewards for companies who obtain shitty troll patents like these, and in turn have companies who reward the engineers who sell themselves out to write up concepts which they know have prior art. Software patents are broken by design!

    (tags: cd devops hp continuous-deployment testing deployment performance patents swpats prior-art)

Links for 2015-03-05

Published March 5, 2015

Links for 2015-03-04

Published March 4, 2015

Links for 2015-03-02

Published March 2, 2015

  • Glowroot

    "Open source APM for Java" -- profiling in production, with a demo benchmark showing about a 2% performance impact. Wonder about effects on memory/GC, though

    (tags: apm java metrics measurement new-relic profiling glowroot)

  • "Everything you've ever said to Siri/Cortana has been recorded...and I get to listen to it"

    This should be a reminder.

    At first, I though these sound bites were completely random. Then I began to notice a pattern. Soon, I realized that I was hearing peoples commands given to their mobile devices. Guys, I'm telling you, if you've said it to your phone, it's been recorded...and there's a damn good chance a 3rd party is going to hear it.

    (tags: privacy google siri cortana android voice-recognition outsourcing mobile)

  • Halcyon Days

    Fantastic 1997-era book of interviews with the programmers behind some of the greatest games in retrogaming history:

    Halcyon Days: Interviews with Classic Computer and Video Game Programmers was released as a commercial product in March 1997. At the time it was one of the first retrogaming projects to focus on lost history rather than game collecting, and certainly the first entirely devoted to the game authors themselves. Now a good number of the interviewees have their own web sites, but none of them did when I started contacting them in 1995. [...] If you have any of the giddy anticipation that I did whenever I picked up a magazine containing an interview with Mark Turmell or Dan [M.U.L.E.] Bunten, then you want to start reading.

    (tags: book games history coding interviews via:walter)

  • Pub Table Quiz - In Aid of Digital Rights Ireland

    Jason Roe is organising a Table Quiz in Dublin on March 26th to support fundraising efforts by Digital Rights Ireland. We will supply tables, questions and a ready supply of beer and maybe finger food.

    (tags: dri pub-quiz fun dublin quizzes)

  • Why are transhumanists such dicks?

    Good discussion from a transhumanist forum (via Boing Boing):

    "I’ve been around and interviewed quite a lot of self-identified transhumanists in the last couple of years, and I’ve noticed many of them express a fairly stark ideology that is at best libertarian, and at worst Randian. Very much “I want super bionic limbs and screw the rest of the world”. They tend to brush aside the ethical, environmental, social and political ramifications of human augmentation so long as they get to have their toys. There’s also a common expression that if sections of society are harmed by transhumanist progress, then it is unfortunate but necessary for the greater good (the greater good often being bestowed primarily upon those endorsing the transhumanism). That attitude isn’t prevalent on this forum at all – I think the site tends to attract more practical body-modders than theoretical transhumanists – but I wondered if anyone else here had experienced the same attitudes in their own circles? What do you make of it?"

    (tags: transhumanism evolution body-modding surgery philosophy via:boingboing libertarianism society politics)

Links for 2015-02-27

Published February 27, 2015

Links for 2015-02-26

Published February 26, 2015

  • JClarity's Illuminate

    Performance-diagnosis-as-a-service. Cool.

    Users download and install an Illuminate Daemon using a simple installer which starts up a small stand alone Java process. The Daemon sits quietly unless it is asked to start gathering SLA data and/or to trigger a diagnosis. Users can set SLA’s via the dashboard and can opt to collect latency measurements of their transactions manually (using our library) or by asking Illuminate to automatically instrument their code (Servlet and JDBC based transactions are currently supported). SLA latency data for transactions is collected on a short cycle. When the moving average of latency measurements goes above the SLA value (e.g. 150ms), a diagnosis is triggered. The diagnosis is very quick, gathering key data from O/S, JVM(s), virtualisation and other areas of the system. The data is then run through the machine learned algorithm which will quickly narrow down the possible causes and gather a little extra data if needed. Once Illuminate has determined the root cause of the performance problem, the diagnosis report is sent back to the dashboard and an alert is sent to the user. That alert contains a link to the result of the diagnosis which the user can share with colleagues. Illuminate has all sorts of backoff strategies to ensure that users don’t get too many alerts of the same type in rapid succession!

    (tags: illuminate jclarity java jvm scala latency gc tuning performance)

  • grpc.io

    Binary message marshalling, client/server stubs generated by an IDL compiler, bidirectional binary protocol. CORBA is back from the dead! Intro blog post: http://googledevelopers.blogspot.ie/2015/02/introducing-grpc-new-open-source-http2.html Relevant: Steve Vinoski's commentary on protobuf-rpc back in 2008: http://steve.vinoski.net/blog/2008/07/13/protocol-buffers-leaky-rpc/

    (tags: http rpc http2 netty grpc google corba idl messaging)

Links for 2015-02-25

Published February 25, 2015

Links for 2015-02-24

Published February 24, 2015

Links for 2015-02-23

Published February 23, 2015

  • ssls.com

    "Cheap SSL certs from $4.99/yr" -- apparently recommended for cheap, low-end SSL certs

    (tags: ssl certs security https ops)

  • Having Your Cake and Eating It Too: Jointly Optimal Erasure Codes for I/O, Storage, and Network-bandwidth | USENIX

    Erasure codes, such as Reed-Solomon (RS) codes, are increasingly being deployed as an alternative to data-replication for fault tolerance in distributed storage systems. While RS codes provide significant savings in storage space, they can impose a huge burden on the I/O and network resources when reconstructing failed or otherwise unavailable data. A recent class of erasure codes, called minimum-storage-regeneration (MSR) codes, has emerged as a superior alternative to the popular RS codes, in that it minimizes network transfers during reconstruction while also being optimal with respect to storage and reliability. However, existing practical MSR codes do not address the increasingly important problem of I/O overhead incurred during reconstructions, and are, in general, inferior to RS codes in this regard. In this paper, we design erasure codes that are simultaneously optimal in terms of I/O, storage, and network bandwidth. Our design builds on top of a class of powerful practical codes, called the product-matrix-MSR codes. Evaluations show that our proposed design results in a significant reduction the number of I/Os consumed during reconstructions (a 5 reduction for typical parameters), while retaining optimality with respect to storage, reliability, and network bandwidth.

    (tags: erasure-coding reed-solomon compression reliability reconstruction replication fault-tolerance storage bandwidth usenix papers)

  • Everyday I'm Shuffling - Tips for Writing Better Spark Programs [slides]

    Two Spark experts from Databricks provide some good tips

    (tags: spark performance batch ops tips slides emr)

Links for 2015-02-22

Published February 22, 2015

Links for 2015-02-20

Published February 20, 2015

  • 2015-02-19 GCE outage

    40 minutes of multi-zone network outage for majority of instances. 'The internal software system which programs GCE’s virtual network for VM egress traffic stopped issuing updated routing information. The cause of this interruption is still under active investigation. Cached route information provided a defense in depth against missing updates, but GCE VM egress traffic started to be dropped as the cached routes expired.' I wonder if Google Pimms fired the alarms for this ;)

    (tags: google outages gce networking routing pimms multi-az cloud)

  • Listen to a song made from data lost during MP3 conversion

    Ryan McGuire, a PhD student in Composition and Computer Technologies at the University of Virginia Center for Computer Music, has created the project The Ghost In The MP3 [....] For his first trick, McGuire took Suzanne Vega’s ‘Tom’s Diner’ and drained it into a vaporous piece titled ‘moDernisT.” McGuire chose the track he explains on his site because it was famously used as one of the main controls in the listening tests used to develop the MP3 algorithm.

    (tags: mp3 music suzanne-vega compression)

Links for 2015-02-19

Published February 19, 2015

  • pcp2graphite

    A gateway script, now included in PCP

    (tags: pcp2graphite pcp graphite ops metrics system)

  • Performance Co-Pilot

    System performance metrics framework, plugged by Netflix, open-source for ages

    (tags: open-source pcp performance system metrics ops red-hat netflix)

  • Superfish: A History Of Malware Complaints And International Surveillance - Forbes

    Superfish, founded and led by former Intel employee and ex-surveillance boffin Adi Pinhas, has been criticised by users the world over since its inception in 2006.

    (tags: superfish lenovo privacy surveillance ads java windows mac firefox pups ssl tls ad-injection komodia)

  • The Superfish certificate has been cracked, exposing Lenovo users to attack | The Verge

    The cracked certificate exposes Lenovo users to man-in-the-middle attacks, similar to those opened up by Heartbleed. Armed with this password and the right software, a coffee shop owner could potentially spy on any Lenovo user on her network, collecting any passwords that were entered during the session. The evil barista could also insert malware into the data stream at will, disguised as a software update or a trusted site.
    Amazingly stupid.

    (tags: superfish inept ca ssl tls lenovo mitm security)

  • Police have asked Dropcam for video from people's home cameras -- Fusion

    “Like any responsible father, Hugh Morrison had installed cameras in every room in the flat,” is the opening line of Intrusion, a 2012 novel set in the near future. Originally installed so that Hugh and his wife can keep an eye on their kids, the Internet-connected cameras wind up being used later in the novel by police who tap into the feeds to monitor the couple chatting on their couch when they are suspected of anti-societal behavior. As with so many sci-fi scenarios, the novel’s vision was prophetic. People are increasingly putting small Internet-connected cameras into their homes. And law enforcement officials are using the cameras to collect evidence about them.

    (tags: privacy dropcam cameras surveillance law-enforcement)

  • Extracting the SuperFish certificate

    not exactly the most challenging reverse I've ever seen ;)

    (tags: reverse-engineering security crypto hacking tls ssl superfish lenovo)

  • The Great SIM Heist: How Spies Stole the Keys to the Encryption Castle

    Holy shit. Gemalto totally rooted.

    With [Gemalto's] stolen encryption keys, intelligence agencies can monitor mobile communications without seeking or receiving approval from telecom companies and foreign governments. Possessing the keys also sidesteps the need to get a warrant or a wiretap, while leaving no trace on the wireless provider’s network that the communications were intercepted. Bulk key theft additionally enables the intelligence agencies to unlock any previously encrypted communications they had already intercepted, but did not yet have the ability to decrypt. [...] According to one secret GCHQ slide, the British intelligence agency penetrated Gemalto’s internal networks, planting malware on several computers, giving GCHQ secret access. We “believe we have their entire network,” the slide’s author boasted about the operation against Gemalto.

    (tags: encryption security crypto nsa gchq gemalto smartcards sim-cards privacy surveillance spying)

  • One year of InfluxDB and the road to 1.0

    half of the [Monitorama] attendees were employees and entrepreneurs at monitoring, metrics, DevOps, and server analytics companies. Most of them had a story about how their metrics API was their key intellectual property that took them years to develop. The other half of the attendees were developers at larger organizations that were rolling their own DevOps stack from a collection of open source tools. Almost all of them were creating a “time series database” with a bunch of web services code on top of some other database or just using Graphite. When everyone is repeating the same work, it’s not key intellectual property or a differentiator, it’s a barrier to entry. Not only that, it’s something that is hindering innovation in this space since everyone has to spend their first year or two getting to the point where they can start building something real. It’s like building a web company in 1998. You have to spend millions of dollars and a year building infrastructure, racking servers, and getting everything ready before you could run the application. Monitoring and analytics applications should not be like this.

    (tags: graphite monitoring metrics tsd time-series analytics influxdb open-source)

  • Sysdig Cloud’s JMX Metrics

    Sysdig Cloud users have the ability to view and analyze Java Management Extensions (JMX) metrics out of the box with no additional configuration or setup required.

    (tags: sysdig jmx java jvm)

  • Will the madness never end? Komodia SSL certificates are EVERYWHERE

    I think that at this point it is safe to assume that any SSL interception product sold by Komodia or based on the Komodia SDK is going to be using the same method. What does this mean? Well, this means that those dodgy certificates aren’t limited to Lenovo laptops sold over a specific date range. It means that anyone who has come into contact with a Komodia product, or who has had some sort of Parental Control software installed on their computer should probably check to see if they are affected.

    (tags: komodia via:jgc ssl lenovo parental-control censorware mitm)

Links for 2015-02-18

Published February 18, 2015

Links for 2015-02-17

Published February 17, 2015

Links for 2015-02-16

Published February 16, 2015

Links for 2015-02-13

Published February 13, 2015

  • Slack's coming to Dublin

    Butterfield insists that Slack improves on the basic messaging functionality offered by its predecessors. The company plans to expand from 100 employees to 250 this year, open an office in Dublin, and launch a version that supports large companies with multiple teams.

    (tags: slack messaging chat dublin ireland jobs tech)

  • yahoo/kafka-manager

    A tool for managing Apache Kafka. It supports the following : Manage multiple clusters; Easy inspection of cluster state (topics, brokers, replica distribution, partition distribution); Run preferred replica election; Generate partition assignments (based on current state of cluster); Run reassignment of partition (based on generated assignments)

    (tags: yahoo kafka ops tools)

  • Vaurien, the Chaos TCP Proxy — Vaurien 1.8 documentation

    Vaurien is basically a Chaos Monkey for your TCP connections. Vaurien acts as a proxy between your application and any backend. You can use it in your functional tests or even on a real deployment through the command-line. Vaurien is a TCP proxy that simply reads data sent to it and pass it to a backend, and vice-versa. It has built-in protocols: TCP, HTTP, Redis & Memcache. The TCP protocol is the default one and just sucks data on both sides and pass it along. Having higher-level protocols is mandatory in some cases, when Vaurien needs to read a specific amount of data in the sockets, or when you need to be aware of the kind of response you’re waiting for, and so on. Vaurien also has behaviors. A behavior is a class that’s going to be invoked everytime Vaurien proxies a request. That’s how you can impact the behavior of the proxy. For instance, adding a delay or degrading the response can be implemented in a behavior. Both protocols and behaviors are plugins, allowing you to extend Vaurien by adding new ones. Last (but not least), Vaurien provides a couple of APIs you can use to change the behavior of the proxy live. That’s handy when you are doing functional tests against your server: you can for instance start to add big delays and see how your web application reacts.

    (tags: proxy tcp vaurien chaos-monkey testing functional-testing failures sockets redis memcache http)

  • Embed-able Computers are a Thing. — February 12, 2015

    'If it works, a copy of Burgertime for DOS is now in your browser, clickable from my entry. If it doesn’t… well, no Burgertime for you. (Unless you visit the page.) There’s a “share this” link in the new archive.org interface for sharing these in-browser emulations in web pages, weblogs and who knows what else.'

    (tags: sharing embeds html javascript emulation msdos burgertime games archive.org)

  • China’s Internet Censors Now Have Their Own Theme Song, And It Is Glorious - China Real Time Report - WSJ

    According to a report posted Thursday to the website of the state-run China Youth Daily, the Cyberspace Administration of China choral group this week unveiled a new song, “Cyberspace Spirit,” glorifying the cleanliness and clarity of China’s uniquely managed Internet. The song, an orchestral march built around a chorus that proclaims China’s ambition to become an “Internet power,” opens with lyrics describing celestial bodies keeping careful watch over the sky. From there, the lyrics conjure more vivid imagery, comparing the Internet to “a beam of incorruptible sunlight” that unites “the powers of life from all creation.”

    (tags: china great-firewall censorship music songs cyberspace-spirit omgwtfbbq)

Links for 2015-02-12

Published February 12, 2015

Links for 2015-02-11

Published February 11, 2015

  • Automating Tinder with Eigenfaces

    While my friends were getting sucked into "swiping" all day on their phones with Tinder, I eventually got fed up and designed a piece of software that automates everything on Tinder.
    This is awesome. (via waxy)

    (tags: via:waxy tinder eigenfaces machine-learning k-nearest-neighbour algorithms automation ai)

  • RateLimitedLogger

    Our latest open source release from Swrve Labs: an Apache-licensed, SLF4J-compatible, simple, fluent API for rate-limited logging in Java: 'A RateLimitedLog object tracks the rate of log message emission, imposes an internal rate limit, and will efficiently suppress logging if this is exceeded. When a log is suppressed, at the end of the limit period, another log message is output indicating how many log lines were suppressed. This style of rate limiting is the same as the one used by UNIX syslog; this means it should be comprehensible, easy to predict, and familiar to many users, unlike more complex adaptive rate limits.' We've been using this in production for months -- it's pretty nifty ;) Never fear your logs again!

    (tags: logs logging coding java open-source swrve slf4j rate-limiting libraries)

Links for 2015-02-10

Published February 10, 2015

Links for 2015-02-06

Published February 6, 2015

Links for 2015-02-05

Published February 5, 2015

Links for 2015-02-03

Published February 3, 2015

Links for 2015-02-02

Published February 2, 2015

  • A Quiet Defense of Patterns

    Marc Brooker: 'When it comes to building working software in the long term, the emotional pursuit of craft is not as important as the human pursuit of teamwork, or the intellectual pursuit of correctness. Patterns is one of the most powerful ideas we have. The critics may be right that it devalues the craft, but we would all do well to remember that the craft of software is a means, not an end.'

    (tags: marc-brooker design-patterns coding software teamwork)

  • One Hundred Miles of Solitude

    Via Walter, the best description of the appeal of Minecraft I've read:

    Minecraft is exceptionally good at intrinsic narrative. It recognises, preserves and rewards everything you do. It presses you to play frontiersman. A Minecraft world ends up dotted with torchlit paths, menhirs, landmarks, emergency caches. Here’s the hole where you dug stone for your first house. Here’s the causeway you built from your spawn point to a handy woodland. Here’s the crater in the landscape where the exploding monster took out you and your wheatfield at once. And, of course, here’s your enormous castle above a waterfall. There’s no utility in building anything bigger than a hut, but the temptations of architecture are irresistible. Minecraft isn’t so much a world generator as a screenshot-generator and a war-story generator. This is what will get the game the bulk of its critical attention, and deservedly so. That’s why I want to call attention to the extrinsic narrative. It’s minimal, implicit,  accidental and very powerful. It’s this: you wake alone beside an endless sea in a pristine, infinite wilderness. The world is yours. You can literally sculpt mountains, with time and effort. You’ll die and be reborn on the beach where you woke first. You’ll walk across the world forever and never see another face. You can build a whole empire of roads and palaces and beacon towers, and the population of that empire will only ever be you. When you leave, your towers will stand empty forever. I haven’t seen that surfaced in a game before. It’s strong wine.

    (tags: minecraft narrative gaming games story)

  • Backstage Blog - Prometheus: Monitoring at SoundCloud - SoundCloud Developers

    whoa, this is pretty excellent. The major improvement over a graphite-based system would be the multi-dimensional tagging of metrics, which we currently have to do by simply expanding the graphite metric's name to encompass all those dimensions and use searching at query time, inefficiently.

    (tags: monitoring soundcloud prometheus metrics service-metrics graphite alerting)

  • 'Prometheus instrumentation library for JVM applications'

    Good example of a clean java OSS release, from Soundcloud. will be copying bits of this myself soon...

    (tags: prometheus java libraries oss github sonatype maven releases)

  • Google Java Style

    A good set of basic, controversy-free guidelines for clean java code style

    (tags: style java google coding guidelines formatting coding-standards)

  • A Brief History of NSA Backdoors

    from 1946 to present

    (tags: nsa security backdoors sigint actel dual_ec_drbg crypto-ag crypto)

  • Study: You Can't Change an Anti-Vaxxer's Mind

    According to a major new study in the journal 'Pediatrics', trying to [persuade anti-vaxxers to vaccinate] may actually make the problem worse. The paper tested the effectiveness of four separate pro-vaccine messages, three of which were based very closely on how the Centers for Disease Control and Prevention (CDC) itself talks about vaccines. The results can only be called grim: Not a single one of the messages was successful when it came to increasing parents' professed intent to vaccinate their children. And in several cases the messages actually backfired, either increasing the ill-founded belief that vaccines cause autism or even, in one case, apparently reducing parents' intent to vaccinate.

    (tags: vaccination health measles mmr autism facts via:mrneutron stupidity cdc papers vaccines)

  • Coining "Dysguria"

    “dysaguria” is the perfect noun, and “dysagurian” is the perfect adjective, to describe the eponymous company in Dave Eggers’ The Circle. It’s not in the same league as Orwell, or Huxley, or Bradbury, or Burgess. But it does raise very important questions about what could possibly go wrong if one company controlled all the world’s information. In the novel, the company operates according to the motto “all that happens must be known”; and one of its bosses, Eamon Bailey, encourages everywoman employee Mae Holland to live an always-on (clear, transparent) life according the maxims “secrets are lies”, “sharing is caring”, and “privacy is theft”. Eggers’s debts to dystopian fiction are apparent. But, whereas writers like Orwell, Huxley, Bradbury, and Burgess were concerned with totalitarian states, Eggers is concerned with a totalitarian company. However, the noun “dystopia” and the adjective “dystopian” – perfect though they are for the terror of military/security authoritarianism in 1984, or Brave new World, or Farenheit 451, or A Clockwork Orange – do not to my mind encapsulate the nightmare of industrial/corporate tyranny in The Circle. On the other hand, “dysaguria” as a noun and “dysagurian” as an adjective, in my view really do capture the essence of that “frightening company”.

    (tags: dysaguria dystopia future sf authoritarianism surveillance the-circle google facebook)

  • A NetHack bot ascends!

    Via negatendo: 'I would like to share my excitement about the fact that after almost a year of development, an instance of my NetHack bot has finally managed to ascend a game for the first time without human interventions, wizard mode cheats or bones stuffing, and did so at the public server at acehack.de.' The bot is written in Clojure. Apparently 'pudding farming' did the trick...

    (tags: clojure via:negatendo pudding-farming games nethack bots)

Links for 2015-01-30

Published January 30, 2015

  • H.P. Lovecat

    Because there exists no method known to man, more terribly suited to expose the cosmic meaningless of existence than pairing the words of H.P. Lovecraft with seemingly delightful and charming pictures of adorable kittens.

    (tags: lovecraft cthulhu horror funny kittens cats images gif)

  • 8 gdb tricks you should know (Ksplice Blog)

    These are very good -- bookmarking for the next time I'm using gdb, probably about 3 years from now

    (tags: c debugging gdb c++ tips coding)

  • EFF’s Game Plan for Ending Global Mass Surveillance

    For years, we’ve been working on a strategy to end mass surveillance of digital communications of innocent people worldwide. Today we’re laying out the plan, so you can understand how all the pieces fit together—that is, how U.S. advocacy and policy efforts connect to the international fight and vice versa. Decide for yourself where you can get involved to make the biggest difference. This plan isn’t for the next two weeks or three months. It’s a multi-year battle that may need to be revised many times as we better understand the tools and authorities of entities engaged in mass surveillance and as more disclosures by whistleblowers help shine light on surveillance abuses.

    (tags: eff privacy nsa surveillance gchq law policy us-politics)

  • No POD

    This group aims to consolidate opposition, give clear information and support letter writing and information awareness against the Dept. of Education's Primary Online Database.

    (tags: pod ireland privacy data-protection children kids schools)

  • Apple Pay suffering fraud problems

    Fraud in Apple Pay will in time, come to be managed – but the fact that easily available PII can waylay best in class protection should give us all pause.

    (tags: fraud apple apple-pay pii identity-theft)

  • Excellent example of failed "anonymisation" of a dataset

    Fred Logue notes how this failed Mayo TD Michelle Mulherin:

    From recent reports it mow appears that the Department of Education is discussing anonymisation of the Primary Online Database with the Data Protection Commissioner. Well someone should ask Mayo TD Michelle Mulherin how anonymisation is working for her. The Sunday Times reports that Ms Mulherin was the only TD in the Irish parliament on the dates when expensive phone calls were made to a mobile number in Kenya. The details of the calls were released under the Freedom of Information Act in an “anonymised” database. While it must be said the fact that Ms Mulherin was the only TD present on those occasions does not prove she made the calls – the reporting in the press is now raising the possibility that it was her. From a data protection point of view this is a perfect example of the difficulty with anonymisation. Data protection rules apply to personal data which is defined as data relating to a living individual who is or can be identified from the data or from the data in conjunction with other information. Anonymisation is often cited as a means for processing data outside the scope of data protection law but as Ms Mulherin has discovered individuals can be identified using supposedly anonymised data when analysed in conjunction with other data. In the case of the mysterious calls to Kenya even though the released information was “anonymised” to protect the privacy of public representatives, the phone log used in combination with the attendance record of public representatives and information on social media was sufficient to identify individuals and at least raise evidence of association between individuals and certain phone calls. While this may be well and good in terms of accounting for abuses of the phone service it also has worrying implications for the ability of public representatives to conduct their business in private. The bottom line is that anonymisation is very difficult if not impossible as Ms Mulherin has learned to her cost. It certainly is a lot more complex than simply removing names and other identifying features from a single dataset. The more data that there is and the more diverse the sources the greater the risk that individuals can be identified from supposedly anonymised datasets.

    (tags: data anonymisation fred-logue ireland michelle-mulherin tds kenya data-protection privacy)

Links for 2015-01-26

Published January 26, 2015

Links for 2015-01-25

Published January 25, 2015

Links for 2015-01-24

Published January 24, 2015

  • How to Catch a Terrorist - The New Yorker

    This is spot on --

    By flooding the system with false positives, big-data approaches to counterterrorism might actually make it harder to identify real terrorists before they act. Two years before the Boston Marathon bombing, Tamerlan Tsarnaev, the older of the two brothers alleged to have committed the attack, was assessed by the city’s Joint Terrorism Task Force. They determined that he was not a threat. This was one of about a thousand assessments that the Boston J.T.T.F. conducted that year, a number that had nearly doubled in the previous two years, according to the Boston F.B.I. As of 2013, the Justice Department has trained nearly three hundred thousand law-enforcement officers in how to file “suspicious-activity reports.” In 2010, a central database held about three thousand of these reports; by 2012 it had grown to almost twenty-eight thousand. “The bigger haystack makes it harder to find the needle,” Sensenbrenner told me. Thomas Drake, a former N.S.A. executive and whistle-blower who has become one of the agency’s most vocal critics, told me, “If you target everything, there’s no target.”

    (tags: terrorism false-positives filtering detection jttf nsa fbi surveillance gchq)

  • Politwoops

    'All deleted tweets from politicians'. Great idea

    (tags: delete twitter politics politicians ireland social-media news)

  • Zoë Keating on getting a shitty deal from Google's new Music Key licensing

    The Youtube music service was introduced to me as a win win and they don’t understand why I don’t see it that way. “We are trying to create a new revenue stream on top of the platform that exists today.” A lot of people in the music industry talk about Google as evil. I don’t think they are evil. I think they, like other tech companies, are just idealistic in a way that works best for them. I think this because I used to be one of them. The people who work at Google, Facebook, etc can’t imagine how everything they make is not, like, totally awesome. If it’s not awesome for you it’s because you just don’t understand it yet and you’ll come around. They can’t imagine scenarios outside their reality and that is how they inadvertently unleash things like the algorithmic cruelty of Facebook’s yearly review (which showed me a picture I had posted after a doctor told me my husband had 6-8 weeks to live).

    (tags: google business music youtube zoe-keating music-key licensing tech)

  • Smash the Engine

    Jacobin Magazine on the revolutionary political allegory in "Snowpiercer": 'If Snowpiercer had merely told the tale of an oppressed working class rising up to seize power from an evil overlord, it would already have been an improvement over most of the political messages in mainstream cinema. There are all sorts of nice touches in its portrayal of a declining capitalism that can maintain its ideological legitimacy even when it literally has no more bullets in its guns. But the story Bong tells goes beyond that. It’s about the limitations of a revolution which merely takes over the existing social machinery rather than attempting to transcend it. '

    (tags: dystopia revolution snowpiercer movies marxism sf politics)

  • Debunking The Dangerous “If You Have Nothing To Hide, You Have Nothing To Fear”

    A great resource bookmark from Falkvinge.

    There are at least four good reasons to reject this argument solidly and uncompromisingly: The rules may change, it’s not you who determine if you’re guilty, laws must be broken for society to progress, and privacy is a basic human need.

    (tags: nsa politics privacy security surveillance gchq rick-falkvinge society)

Links for 2015-01-20

Published January 20, 2015

Links for 2015-01-19

Published January 19, 2015

  • carbon-c-relay

    A much better carbon-relay, written in C rather than Python. Linking as we've been using it in production for quite a while with no problems.

    The main reason to build a replacement is performance and configurability. Carbon is single threaded, and sending metrics to multiple consistent-hash clusters requires chaining of relays. This project provides a multithreaded relay which can address multiple targets and clusters for each and every metric based on pattern matches.

    (tags: graphite carbon c python ops metrics)

  • Surveillance of social media not way to fight terrorism – Minister

    Blanket surveillance of social media is not the solution to combating terrorism and the rights of the individual to privacy must be protected, Data Protection Minister Dara Murphy said on Monday. [He] said Ireland and the European Union must protect the privacy rights of individuals on social media. “Freedom of expression, freedom of movement, and the protection of privacy are core tenets of the European Union, which must be upheld.”

    (tags: dara-murphy data-protection privacy surveillance europe eu ireland social-media)

Links for 2015-01-18

Published January 18, 2015

  • Amazing comment from a random sysadmin who's been targeted by the NSA

    'Here's a story for you. I'm not a party to any of this. I've done nothing wrong, I've never been suspected of doing anything wrong, and I don't know anyone who has done anything wrong. I don't even mean that in the sense of "I pissed off the wrong people but technically haven't been charged." I mean that I am a vanilla, average, 9-5 working man of no interest to anybody. My geographical location is an accident of my birth. Even still, I wasn't accidentally born in a high-conflict area, and my government is not at war. I'm a sysadmin at a legitimate ISP and my job is to keep the internet up and running smoothly. This agency has stalked me in my personal life, undermined my ability to trust my friends attempting to connect with me on LinkedIn, and infected my family's computer. They did this because they wanted to bypass legal channels and spy on a customer who pays for services from my employer. Wait, no, they wanted the ability to potentially spy on future customers. Actually, that is still not accurate - they wanted to spy on everybody in case there was a potentially bad person interacting with a customer. After seeing their complete disregard for anybody else, their immense resources, and their extremely sophisticated exploits and backdoors - knowing they will stop at nothing, and knowing that I was personally targeted - I'll be damned if I can ever trust any electronic device I own ever again. You all rationalize this by telling me that it "isn't surprising", and that I don't live in the [USA,UK] and therefore I have no rights. I just have one question. Are you people even human?'

    (tags: nsa via:ioerror privacy spying surveillance linkedin sysadmins gchq security)

  • DRI’s Unchanged Position on Eircode

    'Broadly, they are satisfied with what we are doing' versus: 'We have deep concerns about the Eircode initiative… We want to state clearly that we are not at all ‘satisfied’ with the postcode that has been designed or the implementation proposals.'

    (tags: dri ireland eircode postcodes privacy data-protection quotes misrepresentation)

Links for 2015-01-17

Published January 17, 2015

  • Misogyny in the Valley

    The young women interns [in one story in this post] worked in a very different way. As I explored their notes, I noticed that ideas were expanded upon, not abandoned. Challenges were identified, but the male language so often heard in Silicon Valley conference rooms - “Well, let me tell you what the problem with that idea is….” - was not in the room.  These young women, without men to define the “appropriate business behavior,” used different behaviors and came up with a startling and valuable solution. They showed many of the values that exist outside of dominance-based leadership: strategic thinking, intuition, nurturing and relationship building, values-based decision-making and acceptance of other’s input. Women need space to be themselves at work. Until people who have created their success by worshipping at the temple of male behavior, like Sheryl Sandberg, learn to value alternate behaviors, the working world will remain a foreign and hostile culture to women. And if we do not continuously work to build corporate cultures where there is room for other behaviors, women will be cast from or abandoned in a world not of our making, where we continuously “just do not fit in,” but where we still must go to earn our livings.

    (tags: sexism misogyny silicon-valley tech work sheryl-sandberg business collaboration)

  • Are you better off running your big-data batch system off your laptop?

    Heh, nice trolling.

    Here are two helpful guidelines (for largely disjoint populations): If you are going to use a big data system for yourself, see if it is faster than your laptop. If you are going to build a big data system for others, see that it is faster than my laptop. [...] We think everyone should have to do this, because it leads to better systems and better research.

    (tags: graph coding hadoop spark giraph graph-processing hardware scalability big-data batch algorithms pagerank)

  • BBC uses RIPA terrorism laws to catch TV licence fee dodgers in Northern Ireland

    Give them the power, they'll use that power. 'A document obtained under Freedom of Information legislation confirms the BBC's use of RIPA in Northern Ireland. It states: "The BBC may, in certain circumstances, authorise under the Regulation of Investigatory Powers Act 2000 and Regulation of Investigatory Powers (British Broadcasting Corporation) Order 2001 the lawful use of detection equipment to detect unlicensed use of television receivers... the BBC has used detection authorised under this legislation in Northern Ireland."'

    (tags: ripa privacy bbc tv license-fee uk northern-ireland law scope-creep)

  • Australia tries to ban crypto research – by ACCIDENT • The Register

    Researchers are warned off [discussing] 512-bits-plus key lengths, systems “designed or modified to perform cryptanalytic functions, or “designed or modified to use 'quantum cryptography'”. [....] “an email to a fellow academic could land you a 10 year prison sentence”.
    https://twitter.com/_miw/status/556023024009224192 notes 'the DSGL 5A002 defines it as >512bit RSA, >512bit DH, >112 bit ECC and >56 bit symmetric ciphers; weak as fuck i say.'

    (tags: law australia crime crypto ecc rsa stupidity fail)

Links for 2015-01-16

Published January 16, 2015

  • A Case Study of Toyota Unintended Acceleration and Software Safety

    I drive a Toyota, and this is scary stuff. Critical software systems need to be coded with care, and this isn't it -- they don't even have a bug tracking system!

    Investigations into potential causes of Unintended Acceleration (UA) for Toyota vehicles have made news several times in the past few years. Some blame has been placed on floor mats and sticky throttle pedals. But, a jury trial verdict was based on expert opinions that defects in Toyota's Electronic Throttle Control System (ETCS) software and safety architecture caused a fatal mishap.  This talk will outline key events in the still-ongoing Toyota UA litigation process, and pull together the technical issues that were discovered by NASA and other experts. The results paint a picture that should inform future designers of safety critical software in automobiles and other systems.

    (tags: toyota safety realtime coding etcs throttle-control nasa code-review embedded)

Links for 2015-01-15

Published January 15, 2015

  • Group warns of postcode project dangers | Irish Examiner

    “We have spoken to the National Consumer Agency, logistics companies and Digital Rights Ireland, with which we have had an indepth conversation to see if there is anything in the proposal that might be considered to have an impact on anyone’s privacy. Broadly, they are satisfied with what we are doing,” [Patricia Cronin, head of the Department of Communications’ postcodes division] told the committee. However in his letter, [DRI's] O’Lachtnain said the group “want to state clearly that we are not at all ‘satisfied’ with the postcode that has been designed or the implementation proposals”.
    Some nerve!

    (tags: dri nca privacy patricia-cronin goverment postcodes eircode dpc ireland)

Links for 2015-01-14

Published January 14, 2015

  • Of Course 23andMe's Plan Has Been to Sell Your Genetic Data All Along

    Today, 23andMe announced what Forbes reports is only the first of ten deals with big biotech companies: Genentech will pay up to $60 million for access to 23andMe's data to study Parkinson's. You think 23andMe was about selling fun DNA spit tests for $99 a pop? Nope, it's been about selling your data all along.

    (tags: testing ethics dna genentech 23andme parkinsons diseases health privacy)

  • Facette

    Really nice time series dashboarding app. Might consider replacing graphitus with this...

    (tags: time-series data visualisation graphs ops dashboards facette)

  • Getting good cancer care through 3D printing

    This is pretty incredible.

    Balzer downloaded a free software program called InVesalius, developed by a research center in Brazil to convert MRI and CT scan data to 3D images. He used it to create a 3D volume rendering from Scott’s DICOM images, which allowed him to look at the tumor from any angle. Then he uploaded the files to Sketchfab and shared them with neurosurgeons around the country in the hope of finding one who was willing to try a new type of procedure. Perhaps unsurprisingly, he found the doctor he was looking for at UPMC, where Scott had her thyroid removed. A neurosurgeon there agreed to consider a minimally invasive operation in which he would access the tumor through Scott’s left eyelid and remove it using a micro drill. Balzer had adapted the volume renderings for 3D printing and produced a few full-size models of the front section of Scott’s skull on his MakerBot. To help the surgeon vet his micro drilling idea and plan the procedure, Balzer packed up one of the models and shipped it off to Pittsburgh.

    (tags: diy surgery health cancer tumours medicine 3d-printing 3d scanning mri dicom)

Links for 2015-01-13

Published January 13, 2015

Links for 2015-01-12

Published January 12, 2015

Links for 2015-01-11

Published January 11, 2015

Links for 2015-01-10

Published January 10, 2015

Links for 2015-01-09

Published January 9, 2015

  • A World Transfixed by Screens - The Atlantic

    Excellent "In Focus" this week -- 'The continued massive growth of connected mobile devices is shaping not only how we communicate with each other, but how we look, behave, and experience the world around us. Smartphones and other handheld devices have become indispensable tools, appendages held at arm's length to record a scene or to snap a selfie. Recent news photos show refugees fleeing war-torn regions holding up their phones as prized possessions to be saved, and relatives of victims lost to a disaster holding up their smartphones to show images of their loved ones to the press. Celebrity selfies, people alone in a crowd with their phones, events obscured by the very devices used to record that event, the brightly lit faces of those bent over their small screens, these are some of the scenes depicted below.'

    (tags: mobile photography in-focus alan-taylor the-atlantic phones selfies pictures)

  • "Incremental Stream Processing using Computational Conflict-free Replicated Data Types" [paper]

    'Unlike existing alternatives, such as stream processing, that favor the execution of arbitrary application code, we want to capture much of the processing logic as a set of known operations over specialized Computational CRDTs, with particular semantics and invariants, such as min/max/average/median registers, accumulators, top-N sets, sorted sets/maps, and so on. Keeping state also allows the system to decrease the amount of propagated information. Preliminary results obtained in a single example show that Titan has an higher throughput when compared with state of the art stream processing systems.'

    (tags: crdt distributed stream-processing replication titan papers)

Links for 2015-01-07

Published January 7, 2015

Links for 2015-01-06

Published January 6, 2015

  • Mantis: Netflix's Event Stream Processing System

    Rx/reactive in style, autoscaling, support for queue/broker-based strong consistency as well as TCP-based lossy delivery

    (tags: netflix rx reactive autoscaling mantis stream-processing)

  • Bad Kids Jokes

    'I now a man with a wooden leg named sea what was the name of the other leg SAND'

    (tags: funny humor kids jokes humour)

  • The Hit Team

    Fergal Crehan's new gig -- good idea!

    The Hit Team helps you fight back against leaked photos and videos, internet targeting and revenge porn.

    (tags: revenge-porn revenge law privacy porn leaks photos videos images selfies)

  • F1: A Distributed SQL Database That Scales

    Beyond the interesting-enough stuff about scalability in a distributed SQL store, there's this really nifty point about avoiding the horrors of the SQL/ORM impedance mismatch:

    At Google, Protocol Buffers are ubiquitous for data storage and interchange between applications. When we still had a MySQL schema, users often had to write tedious and error-prone transformations between database rows and in-memory data structures. Putting protocol buffers in the schema removes this impedance mismatch and gives users a universal data structure they can use both in the database and in application code…. Protocol Buffer columns are more natural and reduce semantic complexity for users, who can now read and write their logical business objects as atomic units, without having to think about materializing them using joins across several tables.
    This is something that pretty much any store can already adopt. Go protobufs. (or Avro, etc.) Also, I find this really neat, and I hope this idea is implemented elsewhere soon: asynchronous schema updates:
    Schema changes are applied asynchronously on multiple F1 servers. Anomalies are prevented by the use of a schema leasing mechanism with support for only current and next schema versions; and by subdividing schema changes into multiple phases where consecutive pairs of changes are mutually compatible and cannot cause anomalies.

    (tags: schema sql f1 google papers orm protobuf)

Links for 2015-01-05

Published January 5, 2015

  • Avleen Vig on distributed engineering teams

    This is a really excellent post on the topic, rebutting Paul Graham's Bay-Area-centric thoughts on the topic very effectively. I've worked in both distributed and non-distributed, as well as effective and ineffective teams ;), and Avleen's thoughts are very much on target.

    I've been involved in the New York start up scene since I joined Etsy in 2010. Since that time, I've seen more and more companies there embrace having distributed teams. Two companies I know which have risen to the top while doing this have been Etsy and DigitalOcean. Both have exceptional engineering teams working on high profile products used by many, many people around the world. There are certainly others outside New York, including Automattic, GitHub, Chef Inc, Puppet... the list goes on. So how did this happen? And why do people continue to insist that distributed teams lower performance, and are a bad idea? Partly because we've done a poor job of showing our industry how to be successful at it, and partly because it's hard. Having successful distributed teams requires special skills from management, which arent't easily learned until you have to manage a distributed team. Catch 22.

    (tags: business culture management communication work distributed-teams avleen-vig engineering)

  • Hack workaround to get JVM thread priorities working on Linux

    As used in Cassandra ( http://grokbase.com/t/hbase/dev/13bf9kezes/about-xx-threadprioritypolicy-42 )!

    if you just set the "ThreadPriorityPolicy" to something else than the legal values 0 or 1, [...] a slight logic bug in Sun's JVM code kicks in, and thus sets the policy to be as if running with root - thus you get exactly what one desire. The operating system, Linux, won't allow priorities to be heightened above "Normal" (negative nice value), and thus just ignores those requests (setting it to normal instead, nice value 0) - but it lets through the requests to set it lower (setting the nice value to some positive value).

    (tags: cassandra thread-priorities threads java jvm linux nice hacks)

Links for 2015-01-04

Published January 4, 2015

  • Amiko Alien2 / Enigma Discussion Thread - boards.ie

    Enigma is a Linux based alternative to the default Spark operating system on these boxes. Enigma is a more customisable OS and provides the ability to add plugins which can accomplish many tasks enabling users to have a box which might look and perform like a Sky box, giving a 7 day EPG and an alternative to series link.
    Looks like a pretty solid hacker community...

    (tags: alien2 tv enigma dvr freeview saorview pvr)

  • Hague reassures MPs on Office 365 data storage as Microsoft ordered to hand over email data

    William Hague, the leader of the House of Commons, has responded to concerns raised by an MP about the security of parliamentary data stored on Microsoft’s Cloud-based servers in Europe. “The relevant servers are situated in the Republic of Ireland and the Netherlands, both being territories covered by the EC Data Protection Directive," William Hague wrote in a letter to John Hemming, MP for Birmingham Yardley. "Any access by US authorities to such data would have to be by way of mutual legal assistance arrangements with those countries.” [...] John Hemming MP told Computer Weekly Hague’s reassurances carried little weight in the face of aggressive legal action by the US government.  “The Microsoft case makes it clear that, in the end, the fact that Microsoft is a US company legally trumps the European Data Protection Directive [...] and where [the letter says] the US authorities could not exercise a right of search and seizure on an extraterritorial basis, well, they are doing that, in America, today.”
    Sounds like they didn't think that through...

    (tags: mail privacy parliament office-365 microsoft mlat surveillance)

Links for 2015-01-03

Published January 3, 2015

Links for 2015-01-03

Published January 3, 2015

Links for 2015-01-02

Published January 2, 2015

  • The open-office trend is destroying the workplace

    Wow, where has this person been for the past 20 years that they haven't had to encounter this? I can only imagine having a private office, tbh.

    my personal performance at work has hit an all-time low. Each day, my associates and I are seated at a table staring at each other, having an ongoing 12-person conversation from 9 a.m. to 5 p.m.  It’s like being in middle school with a bunch of adults. Those who have worked in private offices for decades have proven to be the most vociferous and rowdy. They haven’t had to consider how their loud habits affect others, so they shout ideas at each other across the table and rehash jokes of yore. As a result, I can only work effectively during times when no one else is around, or if I isolate myself in one of the small, constantly sought-after, glass-windowed meeting rooms around the perimeter.

    (tags: business office productivity work desks open-plan)

Links for 2015-01-02

Published January 2, 2015

  • The open-office trend is destroying the workplace

    Wow, where has this person been for the past 20 years that they haven't had to encounter this? I can only imagine having a private office, tbh.

    my personal performance at work has hit an all-time low. Each day, my associates and I are seated at a table staring at each other, having an ongoing 12-person conversation from 9 a.m. to 5 p.m.  It’s like being in middle school with a bunch of adults. Those who have worked in private offices for decades have proven to be the most vociferous and rowdy. They haven’t had to consider how their loud habits affect others, so they shout ideas at each other across the table and rehash jokes of yore. As a result, I can only work effectively during times when no one else is around, or if I isolate myself in one of the small, constantly sought-after, glass-windowed meeting rooms around the perimeter.

    (tags: business office productivity work desks open-plan)

Links for 2014-12-28

Published December 28, 2014

  • 'Uncertain: A First-Order Type for Uncertain Data' [paper, PDF]

    'Emerging applications increasingly use estimates such as sensor data (GPS), probabilistic models, machine learning, big data, and human data. Unfortunately, representing this uncertain data with discrete types (floats, integers, and booleans) encourages developers to pretend it is not probabilistic, which causes three types of uncertainty bugs. (1) Using estimates as facts ignores random error in estimates. (2) Computation compounds that error. (3) Boolean questions on probabilistic data induce false positives and negatives. This paper introduces Uncertain, a new programming language abstraction for uncertain data. We implement a Bayesian network semantics for computation and conditionals that improves program correctness. The runtime uses sampling and hypothesis tests to evaluate computation and conditionals lazily and efficiently. We illustrate with sensor and machine learning applications that Uncertain improves expressiveness and accuracy.' (via Tony Finch)

    (tags: uncertainty estimation types strong-typing coding probability statistics machine-learning sampling via:fanf)

Links for 2014-12-27

Published December 27, 2014

  • Why Airlines Want to Make You Suffer

    'The fee [airline pricing] model comes with systematic costs that are not immediately obvious. Here’s the thing: in order for fees to work, there needs be something worth paying to avoid. That necessitates, at some level, a strategy that can be described as “calculated misery.” Basic service, without fees, must be sufficiently degraded in order to make people want to pay to escape it. And that’s where the suffering begins.'

    (tags: travel airlines pricing fees economy consumer jetblue)

  • A Virtual Machine in Excel

    'Ádám was trying his hand at a problem in Excel, but the official rules prohibit the use of Excel macros. In a daze, he came up with one of the most clever uses of Excel: building an assembly interpreter with the most popular spreadsheet program. This is a virtual Harvard architecture machine without writable RAM; the stack is only lots and lots of IFs.'

    (tags: vms excel hacks spreadsheets coding)

Links for 2014-12-22

Published December 22, 2014

  • coz

    A causal profiler for C++.

    Causal profiling is a novel technique to measure optimization potential. This measurement matches developers' assumptions about profilers: that optimizing highly-ranked code will have the greatest impact on performance. Causal profiling measures optimization potential for serial, parallel, and asynchronous programs without instrumentation of special handling for library calls and concurrency primitives. Instead, a causal profiler uses performance experiments to predict the effect of optimizations. This allows the profiler to establish causality: "optimizing function X will have effect Y," exactly the measurement developers had assumed they were getting all along.
    I can see this being a good technique to stochastically discover race conditions and concurrency bugs, too.

    (tags: optimization c++ performance coding profiling speed causal-profilers)

  • Spark 1.2 released

    This is the version with the superfast petabyte-sort record:

    Spark 1.2 includes several cross-cutting optimizations focused on performance for large scale workloads. Two new features Databricks developed for our world record petabyte sort with Spark are turned on by default in Spark 1.2. The first is a re-architected network transfer subsystem that exploits Netty 4’s zero-copy IO and off heap buffer management. The second is Spark’s sort based shuffle implementation, which we’ve now made the default after significant testing in Spark 1.1. Together, we’ve seen these features give as much as 5X performance improvement for workloads with very large shuffles.

    (tags: spark sorting hadoop map-reduce batch databricks apache netty)

  • The VATMOSS debacle: does the "manual email" loophole work?

    As the 1 January deadline gallops towards the EU, microbusinesses desperate to stay open without breaking the law try to find out, "Can I email stuff out instead?" Well... Yes. - No - It depends - and simultaneously yes AND no, according to Schrödinger’s VAT. So that's clear, then.

    (tags: vat vatmoss eu tax fiasco email microbusiness sme)

  • One artist closing up their Bandcamp site due to new VATMOSS laws

    Nice work, EU

    (tags: eu law tax vat vatmoss matt-stevens bandcamp music downloads)

Links for 2014-12-21

Published December 21, 2014

Links for 2014-12-19

Published December 19, 2014

  • AN OFFER TO SONY FROM 2600

    To demonstrate that hackers have no interest in suppressing speech, quashing controversy, or being intimidated by vague threats, we ask that Sony allow the hacker community to distribute "The Interview" for them on the 25th of December. Now, we're aware that Sony may refer to this distribution method as piracy, but in this particular case, it may well prove to be the salvation of the motion picture industry. By freely offering the film online, millions of people will get to see it and decide for themselves if it has any redeeming qualities whatsoever - as opposed to nobody seeing it and the studios writing it off as a total loss. Theaters would be free from panic as our servers would become the target of any future vague threats (and we believe Hollywood will be most impressed with how resilient peer-to-peer distribution can be in the face of attacks). Most importantly, we would be defying intimidation, something the motion picture industry doesn't quite have a handle on, which is surprising considering how much they've relied upon it in the past.

    (tags: 2600 funny hackers security sony north-korea the-interview movies piracy)

Links for 2014-12-18

Published December 18, 2014

Links for 2014-12-17

Published December 17, 2014

  • 'Machine Learning: The High-Interest Credit Card of Technical Debt' [PDF]

    Oh god yes. This is absolutely spot on, as you would expect from a Google paper -- at this stage they probably have accumulated more real-world ML-at-scale experience than anywhere else. 'Machine learning offers a fantastically powerful toolkit for building complex systems quickly. This paper argues that it is dangerous to think of these quick wins as coming for free. Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning. The goal of this paper is highlight several machine learning specific risk factors and design patterns to be avoided or refactored where possible. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, changes in the external world, and a variety of system-level anti-patterns. [....] 'In this paper, we focus on the system-level interaction between machine learning code and larger systems as an area where hidden technical debt may rapidly accumulate. At a system-level, a machine learning model may subtly erode abstraction boundaries. It may be tempting to re-use input signals in ways that create unintended tight coupling of otherwise disjoint systems. Machine learning packages may often be treated as black boxes, resulting in large masses of “glue code” or calibration layers that can lock in assumptions. Changes in the external world may make models or input signals change behavior in unintended ways, ratcheting up maintenance cost and the burden of any debt. Even monitoring that the system as a whole is operating as intended may be difficult without careful design. Indeed, a remarkable portion of real-world “machine learning” work is devoted to tackling issues of this form. Paying down technical debt may initially appear less glamorous than research results usually reported in academic ML conferences. But it is critical for long-term system health and enables algorithmic advances and other cutting-edge improvements.'

    (tags: machine-learning ml systems ops tech-debt maintainance google papers hidden-costs development)

  • The FBI Used the Web's Favorite Hacking Tool to Unmask Tor Users | WIRED

    Since Operation Torpedo [use of a Metasploit side project], there’s evidence the FBI’s anti-Tor capabilities have been rapidly advancing. Torpedo was in November 2012. In late July 2013, computer security experts detected a similar attack through Dark Net websites hosted by a shady ISP called Freedom Hosting—court records have since confirmed it was another FBI operation. For this one, the bureau used custom attack code that exploited a relatively fresh Firefox vulnerability—the hacking equivalent of moving from a bow-and-arrow to a 9-mm pistol. In addition to the IP address, which identifies a household, this code collected the MAC address of the particular computer that infected by the malware. “In the course of nine months they went from off the shelf Flash techniques that simply took advantage of the lack of proxy protection, to custom-built browser exploits,” says Soghoian. “That’s a pretty amazing growth … The arms race is going to get really nasty, really fast.”

    (tags: fbi tor police flash security privacy anonymity darknet wired via:bruces)

Links for 2014-12-16

Published December 16, 2014

  • Digital Rights Ireland files Amicus Brief in Microsoft v USA with Liberty and ORG

    Microsoft -v- USA is an important ongoing case, currently listed for hearing in 2015 before the US Federal Court of Appeal of the 2nd Circuit. However, as the case centres around the means by which NY law enforcement are seeking to access data of an email account which resides in Dublin, it is also crucially significant to Ireland and the rest of the EU. For that reason, Digital Rights Ireland instructed us to file an Amicus Brief in the US case, in conjunction with the global law firm of White & Case, who have acted pro bono in their representation. Given the significance of the case for the wider EU, both Liberty and the Open Rights Group in the UK have joined Digital Rights Ireland as amici on this brief. We hope it will be of aid to the US court in assessing the significance of the order being appealed by Microsoft for EU citizens and European states, in the light of the existing US and EU Mutual Legal Assistance Treaty.

    (tags: amicus-briefs law us dri microsoft mlats org liberty eu privacy)

Links for 2014-12-15

Published December 15, 2014

Links for 2014-12-13

Published December 13, 2014

  • littleBits Synth Kit

    Wow, this looks cool. $159

    littleBits and Korg have demystified a traditional analog synthesizer, making it super easy for novices and experts alike to create music. connects to speakers, computers and headphones. can be used to make your own instruments. fits into the littleBits modular system for infinite combos of audio, visual and sensory experiences

    (tags: diy hardware music littlebits gadgets make analog synths)

Links for 2014-12-12

Published December 12, 2014

Links for 2014-12-11

Published December 11, 2014

Links for 2014-12-10

Published December 10, 2014

  • AWS Key Management Service Cryptographic Details

    "AWS Key Management Service (AWS KMS) provides cryptographic keys and operations scaled for the cloud. AWS KMS keys and functionality are used by other AWS cloud services, and you can use them to protect user data in your applications that use AWS. This white paper provides details on the cryptographic operations that are executed within AWS when you use AWS KMS."

    (tags: white-papers aws amazon kms key-management crypto pdf)

Links for 2014-12-09

Published December 9, 2014

  • Aurora for MySQL is coming

    some good details of Aurora innards

    (tags: mysql databases aurora aws ec2 sql storage transactions replication)

  • If Eventual Consistency Seems Hard, Wait Till You Try MVCC

    ex-Percona MySQL wizard Baron Schwartz, noting that MVCC as implemented in common SQL databases is not all that simple or reliable compared to big bad NoSQL Eventual Consistency:

    Since I am not ready to assert that there’s a distributed system I know to be better and simpler than eventually consistent datastores, and since I certainly know that InnoDB’s MVCC implementation is full of complexities, for right now I am probably in the same position most of my readers are: the two viable choices seem to be single-node MVCC and multi-node eventual consistency. And I don’t think MVCC is the simpler paradigm of the two.

    (tags: nosql concurrency databases mysql riak voldemort eventual-consistency reliability storage baron-schwartz mvcc innodb postgresql)

  • Scaling email transparency

    This is quite interesting/weird -- Stripe's protocol for mass-CCing email as they scale up the company, based around http://en.wikipedia.org/wiki/Civil_inattention

    (tags: communication culture email management stripe cc transparency civil-inattention)

  • Shanley Kane of Model View Culture Challenges a “Corrupt” Silicon Valley | MIT Technology Review

    If their interests were better serving the world, using technology as a force for social justice, and equitably distributing technology wealth to enrich society … sure, they’d be acting against their interests. But the reality is that tech companies centralize power and wealth in a small group of privileged white men. When that’s the goal, then exploiting the labor of marginalized people and denying them access to power and wealth is 100 percent in line with the endgame. A more diverse tech industry would be better for its workers and everyone else, but it would be worse for the privileged white men at the top of it, because it would mean they would have to give up their monopoly on money and power. And they will fight that with everything they’ve got, which is why we see barriers to equality at every level of the industry.

    (tags: culture feminism tech mit-tech-review shanley-kane privilege vcs silicon-valley)

  • Announcing Snappy Ubuntu

    Awesome! I was completely unaware this was coming down the pipeline.

    A new, transactionally updated Ubuntu for the cloud. Ubuntu Core is a new rendition of Ubuntu for the cloud with transactional updates. Ubuntu Core is a minimal server image with the same libraries as today’s Ubuntu, but applications are provided through a simpler mechanism. The snappy approach is faster, more reliable, and lets us provide stronger security guarantees for apps and users — that’s why we call them “snappy” applications. Snappy apps and Ubuntu Core itself can be upgraded atomically and rolled back if needed — a bulletproof approach to systems management that is perfect for container deployments. It’s called “transactional” or “image-based” systems management, and we’re delighted to make it available on every Ubuntu certified cloud.

    (tags: ubuntu linux packaging snappy ubuntu-core transactional-updates apt docker ops)

  • Dan McKinley :: Thoughts on the Technical Track

    Ouch. I think Amazon did a better job of the Technical Track concept than this, at least

    (tags: engineering management technical-track principal-engineer career work)

  • OSTree

    "git for operating system binaries". OSTree is a tool for managing bootable, immutable, versioned filesystem trees. It is not a package system; nor is it a tool for managing full disk images. Instead, it sits between those levels, offering a blend of the advantages (and disadvantages) of both. You can use any build system you like to place content into it on a build server, then export an OSTree repository via static HTTP. On each client system, "ostree admin upgrade" can incrementally replicate that content, creating a new root for the next reboot. This provides fully atomic upgrades. Any changes made to /etc are propagated forwards, and all local state in /var is shared. A key goal of the project is to complement existing package systems like RPM and Debian packages, and help further their evolution. In particular for example, RPM-OSTree (linked below) has as a goal a hybrid tree/package model, where you replicate a base tree via OSTree, and then add packages on top.

    (tags: os gnome git linux immutable deployment packaging via:fanf)

Links for 2014-12-06

Published December 6, 2014

  • State sanctions foreign phone and email tapping

    Well, this stinks.

    Foreign law enforcement agencies will be allowed to tap Irish phone calls and intercept emails under a statutory instrument signed into law by Minister for Justice Frances Fitzgerald. Companies that object or refuse to comply with an intercept order could be brought before a private “in camera” court. The legislation, which took effect on Monday, was signed into law without fanfare on November 26th, the day after documents emerged in a German newspaper indicating the British spy agency General Communications Headquarters (GCHQ) had directly tapped undersea communications cables between Ireland and Britain for years.

    (tags: ireland law gchq surveillance mlats phone-tapping)

  • "Looks like Chicago PD had a stingray out at the Eric Garner protest last night"

    Your tax dollars at work: Spying on people just because they demand that the government's agents stop killing black people. [...] Anonymous has released a video featuring what appear to be Chicago police radio transmissions revealing police wiretapping of organizers' phones at the protests last night the day after Thanksgiving, perhaps using a stingray. The transmissions pointing to real-time wiretapping involve the local DHS-funded spy 'fusion' center.

    (tags: imsi-catcher stingray surveillance eric-garner protests privacy us-politics anonymous chicago police wiretapping dhs)

  • When data gets creepy: the secrets we don’t realise we’re giving away | Technology | The Guardian

    Very good article around the privacy implications of derived and inferred aggregate metadata from Ben Goldacre.

    We are entering an age – which we should welcome with open arms – when patients will finally have access to their own full medical records online. So suddenly we have a new problem. One day, you log in to your medical records, and there’s a new entry on your file: “Likely to die in the next year.” We spend a lot of time teaching medical students to be skilful around breaking bad news. A box ticked on your medical records is not empathic communication. Would we hide the box? Is that ethical? Or are “derived variables” such as these, on a medical record, something doctors should share like anything else?

    (tags: advertising ethics privacy security law data aggregation metadata ben-goldacre)

  • Stellar/Ripple suffer a failure of their consensus system, resulting in a split-brain failure

    Prof. Mazières’s research indicated some risk that consensus could fail, though we were nor certain if the required circumstances for such a failure were realistic. This week, we discovered the first instance of a consensus failure. On Tuesday night, the nodes on the network began to disagree and caused a fork of the ledger. The majority of the network was on ledger chain A. At some point, the network decided to switch to ledger chain B. This caused the roll back of a few hours of transactions that had only been recorded on chain A. We were able to replay most of these rolled back transactions on chain B to minimize the impact. However, in cases where an account had already sent a transaction on chain B the replay wasn’t possible.

    (tags: consensus distcomp stellar ripple split-brain postmortems outages ledger-fork payment)

  • the "Unknown Pleasures" cover, emulated in Mathematica

    In July 1967, astronomers at the Cavendish Laboratory in Cambridge, observed an unidentified radio signal from interstellar space, which flashed periodically every 1.33730 seconds. This object flashed with such regularity that it was accurate enough to be used as a clock and only be off by one part in a hundred million. It was eventually determined that this was the first discovery of a pulsar, CP-1919.  This is an object that has about the same mass as the Sun, but is the size of the San Francisco Bay at its widest (~20 kilometers) that is rotating so fast that its emitting a beam of light towards Earth like a strobing light house! Pulsars are neutron stars that are formed from the remnants of a massive star when it experiences stellar death. A hand drawn graph plotted in the style of a waterfall plot, in the Cambridge Encyclopedia of Astronomy, later became renown for its use on the cover of the album "Unknown Pleasures"  by 1970s English band Joy Division.
    The entire blog at http://intothecontinuum.tumblr.com/ is pretty great. Lots of nice mathematical animated GIFs, accompanied by Mathematica source and related ponderings.

    (tags: maths gifs animation art unknown-pleasures mathematica cp-1919 pulsars astronomy joy-division waterfall-plots cambridge blogs)

Links for 2014-12-05

Published December 5, 2014

  • Pubs With A Fire

    'Pubs & Bars With Raging Fires in Dublin'. This is important!

    (tags: pubs dublin fires fireplaces beer christmas)

  • Aurora for MySQL is coming

    'Anurag@AWS posts a quite interesting comment on Aurora failover: We asynchronously write to 6 copies and ack the write when we see four completions. So, traditional 4/6 quorums with synchrony as you surmised. Now, each log record can end up with a independent quorum from any other log record, which helps with jitter, but introduces some sophistication in recovery protocols. We peer to peer to fill in holes. We also will repair bad segments in the background, and downgrade to a 3/4 quorum if unable to place in an AZ for any extended period. You need a pretty bad failure to get a write outage.' (via High Scalability)

    (tags: via:highscalability mysql aurora failover fault-tolerance aws replication quorum)

Links for 2014-12-02

Published December 2, 2014

Links for 2014-12-01

Published December 1, 2014

  • Speeding up Rails 4.2

    Reading between the lines, it looks like Rails 4 is waaay slower than 3....

    (tags: rails ruby performance profiling discourse)

  • Day 1 - Docker in Production: Reality, Not Hype

    Good Docker info from Bridget Kromhout, on their production and dev usage of Docker at DramaFever. lots of good real-world tips

    (tags: docker ops boot2docker tips sysadvent)

  • Lost avant-garde painting found in Stuart Little’s living room

    Two years later, he heard from Lisa S., an assistant set designer on [the movie] Stuart Little. She had bought the painting for $500 from an antiques store in Pasadena specifically for the movie because she thought its cool elegance was perfectly suited for the Little’s New York City apartment. Lisa S. had tracked it down in another warehouse and purchased it from Sony just because she liked it so much. When she contacted Barki, she had no idea of the history of the painting hanging on her bedroom wall. After Barki visited the painting in person and confirmed its identity, Lisa sold it to a private collector. That collector has now been persuaded to sell it in Hungary. It will go up for auction at the Virag Judit Art Gallery in Budapest on December 13th with a starting price of 110,000 euros ($160,000). Gergely Barki won’t make a dime off of his discovery, but he will have a great story to tell in his biography of the artist.

    (tags: stuart-little art history hungary pasadena movies set-design antiques robert-bereny post-impressionism)

Links for 2014-11-30

Published November 30, 2014

  • rjbs's rubric: In Soviet Minecraft, server op you!

    wow, that is too much effort for a 7-year-old's Minecraft server ;) Very impressive

    (tags: minecraft game-servers kids teleport gaming rjbs perl)

  • Rust borrow and lifetimes

    How Rust avoids GC overhead using it's "borrow" system:

    Rust achieves memory safety without GC by using a sophiscated borrow system. For any resource (stack memory, heap memory, file handle and so on), there is exactly one owner which takes care of its resource deallocation, if needed. You may create new bindings to refer to the resource using & or &mut, which is called a borrow or mutable borrow. The compiler ensures all owners and borrowers behave correctly.

    (tags: languages rust gc borrow lifecycle stack heap allocation)

  • Java for Everything

    Actually, I'm really agreeing with a lot of this. Particularly this part:

    Programmers will cringe at writing some kind of command dispatch list: if command = "up": up() elif command = "status": status() elif command = "revert": revert() ... so they’ll go off and write some introspecting auto-dispatch cleverness, but that takes longer to write and will surely confuse future readers who’ll wonder how the heck revert() ever gets called. Yet the programmer will incorrectly feel as though he saved himself time. This is the trap of the dynamic language. It feels like you’re being more productive, but aside from the first 10 minutes of a new program, you’re not. Just write the stupid dispatch manually and get on with the real work.
    I've also gone right off dynamic languages for any kind of non-toy work. Mind you he needs to get around to ditching Vim for a proper IDE. That's the key thing that makes coding in a statically-typed language really pleasant -- when graphical refactoring becomes easy and usable, and errors are visible as you type them...

    (tags: java coding static-typing python unit-tests)

  • Facebook Fabric Networking Deconstructed

    whoa, this is incredibly in-depth

    (tags: facebook datacenter networking clos-networks infrastructure networks fat-tree)

Links for 2014-11-28

Published November 28, 2014

  • OS X doesn't support 'ndots' DNS resolution

    "ping foo.bar" will not append the "search" domains configured in /etc/resolv.conf. Apparently this has been broken since OS X Lion, no sign of a fix. Nice work Apple

    (tags: apple fail bugs resolv dns domains osx)

  • TCP incast

    a catastrophic TCP throughput collapse that occurs as the number of storage servers sending data to a client increases past the ability of an Ethernet switch to buffer packets. In a clustered file system, for example, a client application requests a data block striped across several storage servers, issuing the next data block request only when all servers have responded with their portion (Figure 1). This synchronized request workload can result in packets overfilling the buffers on the client's port on the switch, resulting in many losses. Under severe packet loss, TCP can experience a timeout that lasts a minimum of 200ms, determined by the TCP minimum retransmission timeout (RTOmin).

    (tags: incast networking performance tcp bandwidth buffering switch ethernet capacity)

  • Solving the Mystery of Link Imbalance: A Metastable Failure State at Scale | Engineering Blog | Facebook Code

    Excellent real-world war story from Facebook -- a long-running mystery bug was eventually revealed to be a combination of edge-case behaviours across all the layers of the networking stack, from L2 link aggregation at the agg-router level, up to the L7 behaviour of the MySQL client connection pool.

    Facebook collocates many of a user’s nodes and edges in the social graph. That means that when somebody logs in after a while and their data isn’t in the cache, we might suddenly perform 50 or 100 database queries to a single database to load their data. This starts a race among those queries. The queries that go over a congested link will lose the race reliably, even if only by a few milliseconds. That loss makes them the most recently used when they are put back in the pool. The effect is that during a query burst we stack the deck against ourselves, putting all of the congested connections at the top of the deck.

    (tags: architecture debugging devops facebook layer-7 mysql connection-pooling aggregation networking tcp-stack)

  • "Macaroons" for fine-grained secure database access

    Macaroons are an excellent fit for NoSQL data storage for several reasons. First, they enable an application developer to enforce security policies at very fine granularity, per object. Gone are the clunky security policies based on the IP address of the client, or the per-table access controls of RDBMSs that force you to split up your data across many tables. Second, macaroons ensure that a client compromise does not lead to loss of the entire database. Third, macaroons are very flexible and expressive, able to incorporate information from external systems and third-party databases into authorization decisions. Finally, macaroons scale well and are incredibly efficient, because they avoid public-key cryptography and instead rely solely on fast hash functions.

    (tags: security macaroons cookies databases nosql case-studies storage authorization hyperdex)

  • Richard Tynan on Twitter: "GCHQ Tapping Eircom owned cable"

    Cable listed as owned by Eircom and Cable and Wireless (now Vodafone?)

    (tags: vodafone cables tapping surveillance eircom internet uk)

  • Hermitage: Testing the "I" in ACID

    [Hermitage is] a test suite for databases which probes for a variety of concurrency issues, and thus allows a fair and accurate comparison of isolation levels. Each test case simulates a particular kind of race condition that can happen when two or more transactions concurrently access the same data. Each test can pass (if the database’s implementation of isolation prevents the race condition from occurring) or fail (if the race condition does occur).

    (tags: acid architecture concurrency databases nosql)