3 Lessons From The Amazon Takedown – Fortune
They are: The leaders we admire aren’t always that admirable; Economic performance and costs trump employee well-being; and people participate in and rationalize their own subjugation. ‘In the end, “Amazonians” are not that different from other people in their psychological dynamics. Their company is just a more extreme case of what many other organizations regularly do. And most importantly, let’s locate the problem, if there is one, and its solution where it most appropriately belongs—not with a CEO who is greatly admired (and wealthy beyond measure) running a highly admired company, but with a society where money trumps human well-being and where any price, maybe even lives, is paid for status and success.’ (via Lean)
(tags: amazon work work-life-balance life us fortune via:ldoody ceos employment happiness)
What does it take to make Google work at scale? [slides]
50-slide summary of Google’s stack, compared vs Facebook, Yahoo!, and open-source-land, with the odd interesting architectural insight
(tags: google architecture slides scalability bigtable spanner facebook gfs storage)
Scaling Analytics at Amplitude
Good blog post on Amplitude’s lambda architecture setup, based on S3 and a custom “real-time set database” they wrote themselves. antirez’ comment from a Redis angle on the set database: http://antirez.com/news/92 HN thread: https://news.ycombinator.com/item?id=10118413
(tags: lambda-architecture analytics via:hn redis set-storage storage databases architecture s3 realtime)
Justin's Linklog Posts
-
toxy is a fully programmatic and hackable HTTP proxy to simulate server failure scenarios and unexpected network conditions. It was mainly designed for fuzzing/evil testing purposes, when toxy becomes particularly useful to cover fault tolerance and resiliency capabilities of a system, especially in service-oriented architectures, where toxy may act as intermediate proxy among services. toxy allows you to plug in poisons, optionally filtered by rules, which essentially can intercept and alter the HTTP flow as you need, performing multiple evil actions in the middle of that process, such as limiting the bandwidth, delaying TCP packets, injecting network jitter latency or replying with a custom error or status code.
(tags: toxy proxies proxy http mitm node.js soa network failures latency slowdown jitter bandwidth tcp)
Drone Oversight Is Coming to Construction Sites
Grim Meathook Future
(tags: grim-meathook-future drones work panopticon future sacramento building-sites)
-
Open source security team has had enough of embedded-systems vendors taking the piss with licensing:
This announcement is our public statement that we’ve had enough. Companies in the embedded industry not playing by the same rules as every other company using our software violates users’ rights, misleads users and developers, and harms our ability to continue our work. Though I’ve only gone into depth in this announcement on the latest trademark violation against us, our experience with two GPL violations over the previous year have caused an incredible amount of frustration. These concerns are echoed by the complaints of many others about the treatment of the GPL by the embedded Linux industry in particular over many years. With that in mind, today’s announcement is concerned with the future availability of our stable series of patches. We decided that it is unfair to our sponsors that the above mentioned unlawful players can get away with their activity. Therefore, two weeks from now, we will cease the public dissemination of the stable series and will make it available to sponsors only. The test series, unfit in our view for production use, will however continue to be available to the public to avoid impact to the Gentoo Hardened and Arch Linux communities. If this does not resolve the issue, despite strong indications that it will have a large impact, we may need to resort to a policy similar to Red Hat’s, described here or eventually stop the stable series entirely as it will be an unsustainable development model.
(tags: culture gpl linux opensource security grsecurity via:nelson gentoo arch-linux gnu)
London Calling: Two-Factor Authentication Phishing From Iran
some rather rudimentary anti-2FA attempts, presumably from Iranian security services
(tags: authentication phishing security iran activism 2fa mfa)
Vegemite May Power The Electronics Of The Future
Professor Marc in het Panhuis at the ARC Centre of Excellence for Electromaterials Science figured out that you can 3D print the paste and use it to carry current, effectively creating Vegemite bio-wires. What does this mean? Soon you can run electricity through your food. “The iconic Australian Vegemite is ideal for 3D printing edible electronics,” said the professor. “It contains water so it’s not a solid and can easily be extruded using a 3D printer. Also, it’s salty, so it conducts electricity.”
I’m sure the same applies for Marmite…(tags: vegemite marmite 3d-printing electronics bread food silly)
Beoir.org Community – Recent Attack on McGargles
bizarre conspiracy theory going around about McGargles microbrewery being owned by Molson in an “astroturf craft beer” operation — they apparently were set up by a bunch of ex-Molson employees. Their beer is getting stickered in off-licenses. Mental!
(tags: beer craft-beer ireland mcgargles conspiracy-theories bizarre beoir)
Mining High-Speed Data Streams: The Hoeffding Tree Algorithm
This paper proposes a decision tree learner for data streams, the Hoeffding Tree algorithm, which comes with the guarantee that the learned decision tree is asymptotically nearly identical to that of a non-incremental learner using infinitely many examples. This work constitutes a significant step in developing methodology suitable for modern ‘big data’ challenges and has initiated a lot of follow-up research. The Hoeffding Tree algorithm has been covered in various textbooks and is available in several public domain tools, including the WEKA Data Mining platform.
(tags: hoeffding-tree algorithms data-structures streaming streams cep decision-trees ml learning papers)
Chinese scammers are now using Stingray tech to SMS-phish
A Stingray-style false GSM base station, hidden in a backpack; presumably they detect numbers in the vicinity, and SMS-spam those numbers with phishing messages. Reportedly the scammers used this trick in “Guangzhou, Zhuhai, Shenzhen, Changsha, Wuhan, Zhengzhou and other densely populated cities”. Dodgy machine translation:
March 26, Zhengzhou police telecommunications fraud cases together, for the first time seized a small backpack can hide pseudo station equipment, and arrested two suspects. Yesterday, the police informed of this case, to remind the general public to pay attention to prevention. “I am the landlord, I changed number, please rent my wife hit the bank card, card number ×××, username ××.” Recently, Jiefang Road, Zhengzhou City Public Security Bureau police station received a number of cases for investigation brigade area of ??the masses police said, frequently received similar phone scam messages. Alarm, the police investigators to determine: the suspect may be in the vicinity of twenty-seven square, large-scale use of mobile pseudo-base release fraudulent information. […] Yesterday afternoon, the Jiefang Road police station, the reporter saw the portable pseudo-base is made up of two batteries, a set-top box the size of the antenna box and a chassis, as well as a pocket computer composed together at most 5 kg.
(via t byfield and Danny O’Brien)(tags: via:mala via:tbyfield privacy scams phishing sms gsm stingray base-stations mobile china)
In search of performance – how we shaved 200ms off every POST request — GoCardless Blog
tl;dr: don’t use Ruby’s Net::HTTP and/or HAProxy prior to 1.4.19
(tags: http ruby tcp nagle performance rtt networking haproxy ack curl)
Non-Celiac Gluten Sensitivity May Not Exist
The data clearly indicated that a nocebo effect, the same reaction that prompts some people to get sick from wind turbines and wireless internet, was at work here. Patients reported gastrointestinal distress without any apparent physical cause. Gluten wasn’t the culprit; the cause was likely psychological. Participants expected the diets to make them sick, and so they did.
(tags: gluten placebo nocebo food science health diet gluten-free fodmaps)
-
Some nice real-world experimentation around large-scale data processing in differential dataflow:
If you wanted to do an iterative graph computation like PageRank, it would literally be faster to sort the edges from scratch each and every iteration, than to use unsorted edges. If you want to do graph computation, please sort your edges. Actually, you know what: if you want to do any big data computation, please sort your records. Stop talking sass about how Hadoop sorts things it doesn’t need to, read some papers, run some tests, and then sort your damned data. Or at least run faster than me when I sort your data for you.
(tags: algorithms graphs coding data-processing big-data differential-dataflow radix-sort sorting x-stream counting-sort pagerank)
Docker image creation, tagging and traceability in Shippable
this is starting to look quite impressive as a well-integrated Docker-meets-CI model; Shippable is basing its builds off Docker baselines and is automatically cutting Docker images of the post-CI stage. Must take another look
Analysis of PS4’s security and the state of hacking
FreeBSD jails and Return-Oriented Programming:
Think of [Return-Oriented Programming] as writing a new chapter to a book, using only words that have appeared at the end of sentences in the previous chapters.
(tags: ps4 freebsd jails security exploits hacking sony rop return-oriented-programming)
10 Lesser-Known Cocktails You Should Be Drinking
like the sound of some of these
My wife found my email in the Ashley Madison database
On misdirected emails and the potential side-effects:
The reasons why these people give out my email instead of one that they can access have always been a bit mysterious to me. It’s one thing to save yourself some spam by using a throwaway address. But why use someone else’s for correspondence you actually want to receive? The closest I’ve come to a working theory is that a lot of them, having been slow off the mark to obtain their own gmail, have addresses like eratliff75@gmail.com. Either they believe they can leave off the numbers and receive the messages anyway, or they often simply forget. That or the E. Ratliffs of the world just view eratliff@gmail.com as some kind of shared resource.
(tags: email mail ashley-madison gmail mistakes misdirected-email)
How to Make Raspberry-Thyme Shrub
looks tasty/non-tricky
How gaming terminology is part of modern mainstream Chinese slang
A few years ago, my mom called to ask for my advice on webcams. She explained (in the English-peppered Chinese that’s the official language of our Chinese-American household) that some of her friends had started sharing videos of themselves singing karaoke. She thought she could do better. “?????PK??,” she remarked: “I want to PK them a little.”
-
Powerline networking is vulnerable to sniffing and brute-force attacks. See also http://www.nosuchcon.org/talks/2014/D1_03_Sebastien_Dudek_HomePlugAV_PLC.pdf
(tags: powerline-networking power networking han home exploits security qualcomm homeplug plcs)
buildfarm_deployment/cleanup_docker_images.py
Cleanup old/obsolete Docker images in a repo.
(tags: disk-space ops docker cleanup cron)
-
Chronos (the Mesos distributed scheduler) comes out looking pretty crappy here
(tags: aphyr mesos chronos cron scheduling outages ops jepsen testing partitions cap)
Kubernetes and AWS VPC Peering – Ben Straub
the perils of overloading 10/8
(tags: 10/8 ec2 aws vpc kubernetes ops internet ip-addresses)
How your entire financial life will be stored in a new ‘digital vault’ – Telegraph
In a move to make it easier to open bank accounts and Isas, people will be asked to share all of their accounts, tax records and personal details with a central service. To check someone’s identity, a company would then ask potential customers a series of questions and check the answers against the information in the vault. The checks would replace the current system in which new customers must send by post copies of their passports, cross-signed by a friend, along with bank statements and utility bills.
hahahaha NO FUCKING WAY.(tags: bills banking uk tax privacy digital-vault accounts authentication identity-theft bad-ideas)
-
_FBGraphQLConnectionStorePersistentPageLoaderOperationDelegate-Protocol.h _FBReactionAcornSportsContentSettingsSetShouldNotPushNotificationsMutationCall.h FBBoostedComponentCreateInputDataCreativeObjectStorySpecLinkDataCallToActionValue.h FBEventUpdateNotificationSubscriptionLevelMutationOptimisticPayloadFactoryProtocol-Protocol.h
I just threw up a little. See also https://www.facebook.com/notes/facebook-engineering/under-the-hood-dalvik-patch-for-facebook-for-android/10151345597798920 , in which the FB Android devs happily reveal that they hot-patch the Dalvik VM at runtime to work around a limit — rather than refactoring their app.(tags: facebook horrors coding ios android dalvik hot-patching apps)
-
I can’t believe this is the state of food blogging in the UK and Ireland. full-on payola for reviews. See also @damienmulley’s excellent rant on the subject in this country: https://twitter.com/damienmulley/status/633353368757497858 — there’s even rate cards for positive review tweets/posts/facebook updates etc.
(tags: food blogging restaurants uk bakeries reviews payola blogger-blackmail pr)
The reusable holdout: Preserving validity in adaptive data analysis
Useful stats hack from Google: “We show how to safely reuse a holdout data set many times to validate the results of adaptively chosen analyses.”
(tags: statistics google reusable-holdout training ml machine-learning data-analysis holdout corpus sampling)
Recommender Systems (Machine Learning Summer School 2014 @ CMU)
Extremely authoritative slide deck on building a recommendation system, from Xavier Amatriain, Research/Engineering Manager at Netflix
(tags: netflix recommendations recommenders ml machine-learning cmu clustering algorithms)
-
our full-featured, high performance, scalable web server designed to compete with the likes of nginx. It has been built from the ground-up with no external library dependencies entirely in x86_64 assembly language, and is the result of many years’ experience with high volume web environments. In addition to all of the common things you’d expect a modern web server to do, we also include assembly language function hooks ready-made to facilitate Rapid Web Application Server (in Assembler) development.
(tags: assembly http performance https ssl x86_64 web ops rwasa tls)
The world beyond batch: Streaming 101 – O’Reilly Media
To summarize, in this post I’ve: Clarified terminology, specifically narrowing the definition of “streaming” to apply to execution engines only, while using more descriptive terms like unbounded data and approximate/speculative results for distinct concepts often categorized under the “streaming” umbrella. Assessed the relative capabilities of well-designed batch and streaming systems, positing that streaming is in fact a strict superset of batch, and that notions like the Lambda Architecture, which are predicated on streaming being inferior to batch, are destined for retirement as streaming systems mature. Proposed two high-level concepts necessary for streaming systems to both catch up to and ultimately surpass batch, those being correctness and tools for reasoning about time, respectively. Established the important differences between event time and processing time, characterized the difficulties those differences impose when analyzing data in the context of when they occurred, and proposed a shift in approach away from notions of completeness and toward simply adapting to changes in data over time. Looked at the major data processing approaches in common use today for bounded and unbounded data, via both batch and streaming engines, roughly categorizing the unbounded approaches into: time-agnostic, approximation, windowing by processing time, and windowing by event time.
(tags: streaming batch big-data lambda-architecture dataflow event-processing cep millwheel data data-processing)
What the hell is going on with SoundCloud?
tl;dr: major labels.
Despite having revenue coming in from ads and subscriptions, SoundCloud still relies on outside investment. While the company received $150 million in a funding round at the end of last year, it pales next to the reported $526 million Spotify gained in June, and if one report is to be believed, SoundCloud is running very low on cash. Furthermore, sources suggest that potential investors are waiting to see what happens with Sony and Universal before ploughing in more money. With the high sums reported to be involved, it’s a stalemate that could potentially break the company whether it decides to pay or not.
(tags: soundcloud music mp3 copyright sony universal spotify funding startups)
GSMem: Data Exfiltration from Air-Gapped Computers over GSM Frequencies
Holy shit.
Air-gapped networks are isolated, separated both logically and physically from public networks. Although the feasibility of invading such systems has been demonstrated in recent years, exfiltration of data from air-gapped networks is still a challenging task. In this paper we present GSMem, a malware that can exfiltrate data through an air-gap over cellular frequencies. Rogue software on an infected target computer modulates and transmits electromagnetic signals at cellular frequencies by invoking specific memory-related instructions and utilizing the multichannel memory architecture to amplify the transmission. Furthermore, we show that the transmitted signals can be received and demodulated by a rootkit placed in the baseband firmware of a nearby cellular phone.
(tags: gsmem gsm exfiltration air-gaps memory radio mobile-phones security papers)
An Amazonian’s response to “Inside Amazon: Wrestling Big Ideas in a Bruising Workplace” — Medium
excellent response to the NYT hatchet job
-
This is great. Featuring Mount Buggery:
There were no tracks of any sort until they reached Mt Howitt and Stewart, perhaps not quite as fit as he could have been, was finding the going tough after the descent from Mt Speculation. Faced with the prospect of yet another laborious climb he exploded with the words ‘What another bugger! I’ll call this mountain Mt Buggery.’
and Mount Arsehole:“We always called it Mt Arsehole… Then they came along with all their fancy bloody maps and ideas. Changed it to Mt Arthur. Christ knows why. Bastard of a place anyway!”
(tags: swearing australia mount-buggery mount-arsehole nsw victoria places history names mountains)
minimaxir/big-list-of-naughty-strings
Late to this one — a nice list of bad input (Unicode zero-width spaces, etc) for testing
(tags: testing strings text data unicode utf-8 tests input corrupt)
Preventing Dependency Chain Attacks in Maven
using a whitelist of allowed dependency JARs and their SHAs
(tags: security whitelisting dependencies coding jar maven java jvm)
-
This is excellent — I wish more companies took this attitude. Applause for Travis CI.
after a couple of weeks of research, we made a decision to offer our expectant mothers AND fathers: 2 weeks before the due date paid at 100% (optional, but recommended); 20 weeks for normal births paid at 100%; 24 weeks for births with complications paid at 100%; Flexible working hours after the 20/24 weeks are complete (part-time arrangements can be made); Your job will be here for you when you return. When we relayed this information to the two US employees, one became a little teary because her last employer (a much bigger and older company), didn’t offer anything. This being her second child, it was a huge relief to know she was going to have paid time off with flexibility upon return. While it was a great reaction, it shouldn’t happen this way. If you value your employees, you should value their need for time away. At the same time, if you want to hire someone, whether or not they are already pregnant should be irrelevant.
Well exceeding even the Irish maternity leave entitlements, since it covers fathers too. And this is a startup!(tags: travisci startups work life family kids paternity-leave maternity-leave)
Improving The Weather On Twitter
lovely open-source dataviz improvement for near-term historical rainfall-radar images
(tags: dataviz weather rain rainfall radar nws twitter bots graphics ui)
Somewhere Over the Rainbow: How to Make Effective Use of Colors in Meteorological Visualizations
Linked from the “Improving the Weather On Twitter” post — choosing the “best” colour scheme for meteorological visualization. Great dataviz resource post
(tags: dataviz colour color meteorological weather nws papers rgb hcl)
Reddit comments from a nuclear-power expert
Reddit user “Hiddencamper” is a senior nuclear reactor operator in the US, and regularly posts very knowledgeable comments about reactor operations, safety procedures, and other details. It’s fascinating (via Maciej)
(tags: via:maciej nuclear-power nuclear atomic power energy safety procedures operations history chernobyl scram)
Amazon EC2 2015 Benchmark: Testing Speeds Between AWS EC2 and S3 Regions
Here we are again, a year later, and still no bloody percentiles! Just amateurish averaging. This is not how you measure anything, ffs. Still, better than nothing I suppose
(tags: fail latency measurement aws ec2 percentiles s3)
background doc on the Jeep hack
“Remote Exploitation of an Unaltered Passenger Vehicle”, by Dr. Charlie Miller (cmiller@openrce.org) and Chris Valasek (cvalasek@gmail.com). QNX, unauthenticated D-Bus, etc.
‘Since a vehicle can scan for other vulnerable vehicles and the exploit doesn’t require any user interaction, it would be possible to write a worm. This worm would scan for vulnerable vehicles, exploit them with their payload which would scan for other vulnerable vehicles, etc. This is really interesting and scary. Please don’t do this. Please.’
Care.data and access to UK health records: patient privacy and public trust
‘In 2013, the United Kingdom launched care.data, an NHS England initiative to combine patient records, stored in the machines of general practitioners (GPs), with information from social services and hospitals to make one centralized data archive. One aim of the initiative is to gain a picture of the care being delivered between different parts of the healthcare system and thus identify what is working in health care delivery, and what areas need greater attention and resources. This case study analyzes the complications around the launch of care.data. It explains the historical context of the program and the controversies that emerged in the course of the rollout. It explores problems in management and communications around the centralization effort, competing views on the safety of “anonymous” and “pseudonymous” health data, and the conflicting legal duties imposed on GPs with the introduction of the 2012 Health and Social Care Act. This paper also explores the power struggles in the battle over care.data and outlines the tensions among various stakeholders, including patients, GPs, the Health and Social Care Information Centre (HSCIC), the government, privacy experts and data purchasers. The predominant public policy question that emerges from this review centers on how best to utilize technological advances and simultaneously strike a balance between the many competing interests around health and personal privacy.’
(tags: care.data privacy healthcare uk nhs trust anonymity anonymization gps medicine)
-
coining a term for the awful buyer’s experience on sites like car-hire or air-travel websites
(tags: hate-selling conversion marking upselling travel web consumer)
How Irish Navy’s expertise saved 367 from 30-second sinking in Mediterranean
War-game exercises saved the day:
As the Ribs made their assessment of the situation and began reassuring those on board that help was at hand, the hopelessly overloaded vessel suddenly listed and sank. The sinking took just over 30 seconds. In those 30 seconds, the Captain of the LE Niamh took a number of instant command decisions that saved hundreds of lives. Most of the refugees cannot swim. Their life expectancy in the water would be measured in seconds. The crew of the Ribs immediately began throwing orange lifejackets into the water – encouraging the now frenzied and milling survivors to cling to them. Individuals, then groups clung to the lifejackets – and one another – as the Ribs rallied around trying to keep the floating human mass from dispersal into wider waters and almost certain death. In the meantime, the commander of the LE Niamh managed to manoeuvre close in to the survivors where spare life-rafts were launched into the water. These 25-man inflatable life-rafts were specifically ordered and kept on board the LE Niamh following a “war-gaming” exercise, where the officers and crew envisaged such a nightmare scenario. Had this forward planning not taken place – there would have been no such extra inflatable lifeboats on board.
(tags: war-gaming planning navy ireland mediterranean sea boats refugees migration drowning liferafts)
-
A well-maintained list with a potted description of each one (via HN)
(tags: postmortems ops uptime reliability)
Advantages of Monolithic Version Control
another Dan Luu post — good summary of the monorepo’s upside
(tags: monorepo git mercurial versioning source-control coding dependencies)
“A Review Of Criticality Accidents, 2000 Revision”
Authoritative report from LANL on accidents involving runaway nuclear reactions over the years from 1945 to 1999, around the world. Illuminating example of how incident post-mortems are handled in other industries, and (of course) fascinating in its own right
(tags: criticality nuclear safety atomic lanl post-mortems postmortems fission)
-
Netflix’ official test video — contains various scenarios which exercise frequent tricky edge cases in video compression and playback; A/V sync, shades of black, running water, etc.
(tags: networking netflix streaming video compression tests)
How to get your water tested for lead in Dublin
Ossian has written up this very informative post:
Irish Water is writing to thousands of people living in Dublin this week to warn them that their water is supplied through lead pipes. Irish Water says that most people receiving these letters have a level of lead in their water which is above safe limits. So, if you get one of these letters how do you get your water tested? Irish Water is refusing to supply test kits or to test everyone’s water who asks. However the HSE’s Public Analyst Lab has told me that they will test water for lead for a fee of €10.
(tags: ossian-smyth dun-laoghaire dublin drinking-water water lead green hse irish-water health)
-
Good advice on production-quality, decent-scale usage of Kinesis in Java with the official library: batching, retries, partial failures, backoff, and monitoring. (Also, jaysus, the AWS Cloudwatch API is awful, looking at this!)
(tags: kpl aws kinesis tips java batching streaming production cloudwatch monitoring coding)
IrishCycle.com on the Irish Times’ terrible victim-blaming anti-cycling op-ed
Even if The Irish Times wants to deny that it has engaged in victim blaming at a high level, it has also clearly errored in fact in a very significant way. It would be more forgiving if this was an isolated editorial. But it’s after two days of wrong or misleading coverage, which now seems to be a trend with the newspaper with unbalanced articles or headlines negatively focusing on cycle routes.
(tags: irish-times newspapers op-eds cycling dublin ireland safety)
-
plenty of stuff out of bounds in Dublin tomoz
India lifts porn ban after widespread outrage – BBC News
After a brief couple of days.
News of the ban caused a furore on Indian social media, with several senior politicians and members of civil society expressing their opposition to the move. The Indian government said that it was merely complying with the Supreme Court order and was committed to the freedom of communication on the Internet. “I reject with contempt the charge that it is a Talibani government, as being said by some of the critics. Our government supports free media, respects communication on social media and has respected freedom of communication always,” Mr Prasad told PTI.
(tags: india porn filtering isps internet web child-porn censorship)
17 of the most important things to ever happen to Irish Twitter
definitive. The David O’Doherty / Not The RTE Guide “your ma” battle is legendary (http://thedailyedge.thejournal.ie/your-ma-david-odoherty-1290482-Jan2014/)
(tags: ireland twitter funny social-media)
Amazon S3 Introduces New Usability Enhancements
bucket limit increase, and read-after-write consistency in US Standard. About time too! ;)
(tags: aws s3 storage consistency)
New study shows Spain’s “Google tax” has been a disaster for publishers
A study commissioned by Spanish publishers has found that a new intellectual property law passed in Spain last year, which charges news aggregators like Google for showing snippets and linking to news stories, has done substantial damage to the Spanish news industry. In the short-term, the study found, the law will cost publishers €10 million, or about $10.9 million, which would fall disproportionately on smaller publishers. Consumers would experience a smaller variety of content, and the law “impedes the ability of innovation to enter the market.” The study concludes that there’s no “theoretical or empirical justification” for the fee.
(tags: google news publishing google-tax spain law aggregation snippets economics)
Inside the sad, expensive failure of Google+
“It was clear if you looked at the per user metrics, people weren’t posting, weren’t returning and weren’t really engaging with the product,” says one former employee. “Six months in, there started to be a feeling that this isn’t really working.” Some lay the blame on the top-down structure of the Google+ department and a leadership team that viewed success as the only option for the social network. Failures and disappointing data were not widely discussed. “The belief was that we were always just one weird feature away from the thing taking off,” says the same employee.
(tags: google google+ failures post-mortems business facebook social-media fail bureaucracy vic-gundotra)
8,000 sq ft start-up meeting space revealed for Dublin
Neat. this is a good location for post-work user-group meetups and the like (via Oisin)
(tags: via:oisin meetups meetings ulster-bank dublin startups chq)
Introducing Nurse: Auto-Remediation at LinkedIn
Interesting to hear about auto-remediation in prod — we built a (very targeted) auto-remediation system in Amazon on the Network Monitoring team, but this is much bigger in focus
(tags: nurse auto-remediation outages linkedin ops monitoring)
-
Choco is [FOSS] dedicated to Constraint Programming[2]. It is a Java library written under BSD license. It aims at describing hard combinatorial problems in the form of Constraint Satisfaction Problems and solving them with Constraint Programming techniques. The user models its problem in a declarative way by stating the set of constraints that need to be satisfied in every solution. Then, Choco solves the problem by alternating constraint filtering algorithms with a search mechanism. […] Choco is among the fastest CP solvers on the market. In 2013 and 2014, Choco has been awarded many medals at the MiniZinc challenge that is the world-wide competition of constraint-programming solvers.
(tags: choco constraint-programming solving search combinatorial algorithms)
Three Flavours Cornetto trilogy
Shaun Of The Dead, Hot Fuzz, and The World’s End are a trilogy. I had no idea! (via David Malone)
(tags: movies edgar-wright via:dwmalone funny film cornetto)
Postmortem for July 27 outage of the Manta service – Blog – Joyent
Summary: PostgreSQL’s dreaded unpredictable “vacuum” GC
-
YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.
-
RunJOP (Run Just Once Please) is a distributed execution framework to run a command (i.e. a job) only once in a group of servers [built using AWS DynamoDB and S3].
nifty! Distributed cron is pretty easy when you’ve got Dynamo doing the heavy lifting.(tags: dynamodb cron distributed-cron scheduling runjop danilop hacks aws ops)
Why Docker is Not Yet Succeeding Widely in Production
Spot-on points which Docker needs to address. It’s still production-ready, and _should_ be used there, it just has significant rough edges…
(tags: docker containers devops deployment releases linux ops)
How to Create RSS Feeds for Twitter
The latest hacky workaround to Twitter’s API shortcoming
(tags: rss-feeds feeds twitter favorites api social-media workaround google-script)
Testing without mocking in Scala
mocks are the sound of your code crying out, “please structure me differently!”
+1(tags: scala via:jessitron mocks mock-objects testing testability coding)
Newegg vs. Patent Trolls: When We Win, You Win
go NewEgg: ‘Newegg went against a company that claimed its patent covered SSL and RC4 encryption, a common encryption system used by many retailers and websites. This particular patent troll has gone against over 100 other companies, and brought in $45 million in settlements before going after Newegg. We won.’
(tags: via:nelson ip law patent-trolls patents newegg crypto)
-
A lovely eulogy for Nóirín Plunkett, from Rich Bowen. RIP Nóirín :(
A Visual Introduction to Machine Learning
beautiful visualisation of a decision tree
(tags: decision-trees dataviz via:nelson d3 ml machine-learning)
Taming Complexity with Reversibility
This is a great post from Kent Beck, putting a lot of recent deployment/rollout patterns in a clear context — that of supporting “reversibility”:
Development servers. Each engineer has their own copy of the entire site. Engineers can make a change, see the consequences, and reverse the change in seconds without affecting anyone else. Code review. Engineers can propose a change, get feedback, and improve or abandon it in minutes or hours, all before affecting any people using Facebook. Internal usage. Engineers can make a change, get feedback from thousands of employees using the change, and roll it back in an hour. Staged rollout. We can begin deploying a change to a billion people and, if the metrics tank, take it back before problems affect most people using Facebook. Dynamic configuration. If an engineer has planned for it in the code, we can turn off an offending feature in production in seconds. Alternatively, we can dial features up and down in tiny increments (i.e. only 0.1% of people see the feature) to discover and avoid non-linear effects. Correlation. Our correlation tools let us easily see the unexpected consequences of features so we know to turn them off even when those consequences aren’t obvious. IRC. We can roll out features potentially affecting our ability to communicate internally via Facebook because we have uncorrelated communication channels like IRC and phones. Right hand side units. We can add a little bit of functionality to the website and turn it on and off in seconds, all without interfering with people’s primary interaction with NewsFeed. Shadow production. We can experiment with new services under real load, from a tiny trickle to the whole flood, without affecting production. Frequent pushes. Reversing some changes require a code change. On the website we never more than eight hours from the next schedule code push (minutes if a fix is urgent and you are willing to compensate Release Engineering). The time frame for code reversibility on the mobile applications is longer, but the downward trend is clear from six weeks to four to (currently) two. Data-informed decisions. (Thanks to Dave Cleal) Data-informed decisions are inherently reversible (with the exceptions noted below). “We expect this feature to affect this metric. If it doesn’t, it’s gone.” Advance countries. We can roll a feature out to a whole country, generate accurate feedback, and roll it back without affecting most of the people using Facebook. Soft launches. When we roll out a feature or application with a minimum of fanfare it can be pulled back with a minimum of public attention. Double write/bulk migrate/double read. Even as fundamental a decision as storage format is reversible if we follow this format: start writing all new data to the new data store, migrate all the old data, then start reading from the new data store in parallel with the old.
We do a bunch of these in work, and the rest are on the to-do list. +1 to these!(tags: software deployment complexity systems facebook reversibility dark-releases releases ops cd migration)
Benchmarking GitHub Enterprise – GitHub Engineering
Walkthrough of debugging connection timeouts in a load test. Nice graphs (using matplotlib)
(tags: github listen-backlog tcp debugging timeouts load-testing benchmarking testing ops linux)
How .uk came to be (and why it’s not .gb)
WB: By the late 80s the IANA [the Internet Assigned Numbers Authority, set up in 1988 to manage global IP address allocations] was trying to get all those countries that were trying to join the internet to use the ISO 3166 standard for country codes. It was used for all sorts of things?—?you see it on cars, “GB” for the UK. […] At that point, we’re faced with a problem that Jon Postel would like to have changed it to .gb to be consistent with the rest of the world. Whereas .uk had already been established, with a few tens of thousands of domain names with .uk on them. I remember chairing one of the JANET net workshops that were held every year, and the Northern Irish were adamant that they were part of the UK?—?so the consensus was, we’d try and keep .uk, we’d park .gb and not use it. PK: I didn’t particularly want to change to .gb because I was responsible for Northern Ireland as well. And what’s more, there was a certain question as to whether a research group in the US should be allowed to tell the British what to do. So this argy-bargy continued for a little while and, in the meantime, one of my clients was the Ministry of Defence, and they decided they couldn’t wait this long, and they decided I was going to lose the battle, and so bits of MOD went over to .gb?—?I didn’t care, as I was running .gb and .uk in any case.
(tags: dot-uk history internet dot-gb britain uk northern-ireland ireland janet)
That time the Internet sent a SWAT team to my mom’s house – Boing Boing
The solution is for social media sites and the police to take threats or jokes about swatting, doxxing, and organized crime seriously. Tweeting about buying a gun and shooting up a school would be taken seriously, and so should the threat of raping, doxxing, swatting or killing someone. Privacy issues and online harassment are directly linked, and online harassment isn’t going anywhere. My fear is that, in reaction to online harassment, laws will be passed that will break down our civil freedoms and rights online, and that more surveillance will be sold to users under the guise of safety. More surveillance, however, would not have helped me or my mother. A platform that takes harassment and threats seriously instead of treating them like jokes would have.
(tags: twitter gamergate 4chan 8chan privacy doxxing swatting harrassment threats social-media facebook law feminism)
Why Google’s Deep Dream Is Future Kitsch
Deep Dream estranges us from our fears, perhaps, but it doesn’t make them go away. It’s easy to discuss Deep Dream as an independent creature, a foreign intelligence that we interact with for fun. Yet like all kitsch, it comes straight back to its creators.
(tags: kitsch deep-dream art graphics google inceptionism)
It’s Not Climate Change?—?It’s Everything Change
now this is a Long Read. the inimitable Margaret Atwood on climate change, beautifully illustrated
(tags: climate climate-change margaret-atwood long-reads change life earth green future)
In Praise of the AK-47 — Dear Design Student — Medium
While someone can certainly make the case that an AK-47, or any other kind of gun or rifle is designed, nothing whose primary purpose is to take away life can be said to be designed well. And that attempting to separate an object from its function in order to appreciate it for purely aesthetic reasons, or to be impressed by its minimal elegance, is a coward’s way of justifying the death they’ve designed into the word, and the money with which they’re lining their pockets.
-
turns out Ruby has a good set of random-text-generation gems on offer
(tags: random ruby coding text-generation markov-chain gems)
The Titanium Gambit | History | Air & Space Magazine
Amazing story of 1960s detente via Maciej: ‘During the Cold War, Boeing execs got a strange call from the State Department: Would you guys mind trading secrets with the Russians?’
(tags: via:maciej titanium history cold-war detente ussr usa boeing russia aerospace)
I’ve seen more than my fair share of abuse online, but Lorraine Higgins’ bill isn’t the answer
Tom Murphy:
This bill prioritises other peoples’ “alarm or distress” over your communications not just TO them but also ABOUT them. Don’t like what Joan Burton is doing with the water charges? Want to write something on independent media about what you think of that? Better not alarm or distress or harm her! This is the core of my issue with the bill. It’s not just that almost all the agreeable parts of it are already covered by other laws. It’s not just that it’s utterly unenforceable with our current justice system. It’s not just that it’s so vague and fluffy. It’s that it’s so ill-defined and over-reaching that its interpretation will inevitably have to be left to judges. Leaving anything to judges is a bad idea in general. This overly broad and poorly worded bill is a god-send to people who like to bully others into silence. Ironic that eh?!
(tags: lorraine-higgins law seanad abuse harrassment trolls)
-
Lambdas in Java 8 introduce some unpredictable performance implications, due to reliance on escape analysis to eliminate object allocation on every lambda invocation. Peter Lawrey has some details
(tags: lambdas java-8 java performance low-latency optimization peter-lawrey coding escape-analysis)
Mikhail Panchenko’s thoughts on the July 2015 CircleCI outage
an excellent followup operational post on CircleCI’s “database is not a queue” outage
(tags: database-is-not-a-queue mysql sql databases ops outages postmortems)
Men who harass women online are quite literally losers, new study finds
(1) players are anonymous, and the possibility of “policing individual behavior is almost impossible”; (2) they only encounter each other a few times in passing — it’s very possible to hurl an expletive at another player, and never “see” him or her again; and (3) finally, and perhaps predictably, the sex-ratio of players is biased pretty heavily toward men. (A 2014 survey of gender ratios on Reddit found that r/halo was over 95 percent male.) [….] In each of these environments, Kasumovic suggests, a recent influx of female participants has disrupted a pre-existing social hierarchy. That’s okay for the guys at the top — but for the guys at the bottom, who stand to lose more status, that’s very threatening. (It’s also in keeping with the evolutionary framework on anti-lady hostility, which suggests sexism is a kind of Neanderthal defense mechanism for low-status, non-dominant men trying to maintain a shaky grip on their particular cave’s supply of women.) “As men often rely on aggression to maintain their dominant social status,” Kasumovic writes, “the increase in hostility towards a woman by lower-status males may be an attempt to disregard a female’s performance and suppress her disturbance on the hierarchy to retain their social rank.”
(tags: losers sexism mysogyny women halo gaming gamergate 4chan abuse harrassment papers bullying social-status)
The old suburban office park is the new American ghost town – The Washington Post
Most analyses of the market indicate that office parks simply aren’t as appealing or profitable as they were in the 20th century and that Americans just aren’t as keen to cloister themselves in workspaces that are reachable only by car.
(tags: cbd cities work life office-parks commuting america history workplaces)
HACKERS REMOTELY KILL A JEEP ON THE HIGHWAY—WITH ME IN IT
Jaysus, this is terrifying.
Miller and Valasek’s full arsenal includes functions that at lower speeds fully kill the engine, abruptly engage the brakes, or disable them altogether. The most disturbing maneuver came when they cut the Jeep’s brakes, leaving me frantically pumping the pedal as the 2-ton SUV slid uncontrollably into a ditch.
Avoid any car which supports this staggeringly-badly-conceived Uconnect feature:All of this is possible only because Chrysler, like practically all carmakers, is doing its best to turn the modern automobile into a smartphone. Uconnect, an Internet-connected computer feature in hundreds of thousands of Fiat Chrysler cars, SUVs, and trucks, controls the vehicle’s entertainment and navigation, enables phone calls, and even offers a Wi-Fi hot spot.
:facepalm: Also, Chrysler’s response sucks: “Chrysler’s patch must be manually implemented via a USB stick or by a dealership mechanic.”(tags: hacking security cars driving safety brakes jeeps chrysler fiat uconnect can-bus can)
RFC 3339: Date and Time on the Internet: Timestamps
the RFC take on ISO-8601. I need to update my mental bookmarks to start referring to this instead
“Customer data is a liability, not an asset.”
Great turn of phrase from Matthew Green (@matthew_d_green). Emin Gün Sirer adds some detail: “well, an asset with bounded value, and an unbounded liability”
(tags: data privacy data-protection ashleymadison hacks security liability)
Deep Dive Into Docker Storage Drivers
good detail in this presentation
(tags: docker overlayfs aufs btrfs filesystems ops linux containers)
-
oh look, Google has a flight search engine! I had no idea
-
TorrentFreak has the story of a UK-producer and songwriter named Lee Adams who took part in an official remix competition of boy band One Direction’s music, put on by the band and its label, Sony Music. The stems for remixing were released on Soundcloud. The rules of the contest required entrants to upload their remixes on Soundcloud… and that’s exactly what Adams did. And yet those works still got taken down via copyright claims from Sony Music as infringing.
(tags: sony soundcloud anti-piracy automation piracy stems remixing one-direction lee-adams)
WereBank | Were Bank Energy for the People
The Freeman-On-The-Land movement is starting a bank. lols guaranteed
(tags: freemen funny werebank banking money on-my-oath maritime-law)
Angela Merkel told a sobbing girl she couldn’t save her from deportation. It was a lie. – Vox
Argentina has, as a matter of constitutional law, effectively open borders. There are no caps or quotas or lottery systems. You can move there legally if you have an employer or family member to sponsor you. That’s all you need. If you don’t have a sponsor, and make your way in illegally, you’re recognized as an “irregular migrant.” Discrimination against irregular migrants in health care or education is illegal, and deportation in noncriminal cases is exceptionally rare. Large-scale amnesties are the norm. Obviously Argentina is not nearly as rich as Germany or the US or the UK. But it’s considerably richer than three of its neighbors (Bolivia, Paraguay, and Brazil). And yet it doesn’t try hard to keep their residents out. It welcomes them — as it should. “One could have expected catastrophe—an uncontrollable flow of poorer immigrants streaming into the country coupled with angry public backlash,” Elizabeth Slater writes in the World Policy Journal. “That hasn’t happened.” Angela Merkel clearly expects catastrophe if she lets people like this weeping young Palestinian girl stay in Germany. That catastrophe is simply a myth; it wouldn’t happen. What would happen is that Germany’s economy would grow, its culture would grow richer, and that girl and more like her could see their lives improve immeasurably.
(tags: argentina immigration angela-merkel germany eu migrants deportation economics)
-
‘A programming language based on the one liners of Arnold Schwarzenegger’. Presenting hello.arnoldc: IT’S SHOWTIME TALK TO THE HAND “hello world” YOU HAVE BEEN TERMINATED (via Robert Walsh)
(tags: via:rjwalsh c arnold-schwarzenegger one-liners funny coding silly languages)
A simple guide to 9-patch for Android UI
This is a nifty hack. TIL! ‘9-patch uses png transparency to do an advanced form of 9-slice or scale9. The guides are straight, 1-pixel black lines drawn on the edge of your image that define the scaling and fill of your image. By naming your image file name.9.png, Android will recognize the 9.png format and use the black guides to scale and fill your bitmaps.’
(tags: android design 9-patch scaling images bitmaps scale9 9-slice ui graphics)
Government forum to discuss increasing use of personal data
Mr Murphy said it was the Government’s objective for Ireland to be a leader on data protection and data-related issues. The members of the forum include Data Protection Commissioner Helen Dixon, John Barron, chief technology officer with the Revenue Commissioners, Seamus Carroll, head of civil law reform division at the Department of Justice and Tim Duggan, assistant secretary with the Department of Social Protection. Gary Davis, director of privacy and law enforcement requests with Apple, is also on the forum. Mr Davis is a former deputy data protection commissioner in Ireland. There are also representatives from Google, Twitter, LinkedIn and Facebook, from the IDA, the Law Society and the National Statistics Board. Chair of Digital Rights Ireland Dr TJ McIntyre and Dr Eoin O’Dell, associate professor, School of Law, Trinity College Dublin are also on the voluntary forum.
(tags: ireland government dri law privacy data data-protection dpc)
-
From DataDog. See also “How to collect NGINX metrics”: https://www.datadoghq.com/blog/how-to-collect-nginx-metrics/
From Zero to Docker: Migrating to the Whale
nicely detailed writeup of how New Relic are dockerizing
(tags: docker ops deployment packaging new-relic)
Docker with OverlayFS first impressions
a brief howto
(tags: overlayfs docker filesystems ops linux)
-
a new sketch algorithm from Baron Schwartz and Preetam Jinka of VividCortex; similar to Count-Min but with last-seen timestamp instead of frequency.
(tags: sketch algorithms estimation approximation sampling streams big-data)
-
The UK Ordnance Survey’s “open data’ free product, free for all uses:
Code-Point Open is FREE to view, download and use for commercial, educational and personal purposes.
(via Antoin)(tags: via:antoin postcodes mapping open-data ordnance-survey uk gb royal-mail maps)
Apple now biases towards IPv6 with a 25ms delay on connections
Interestingly, they claim that IPv6 tends to be more reliable and has lower latency now:
Based on our testing, this makes our Happy Eyeballs implementation go from roughly 50/50 IPv4/IPv6 in iOS 8 and Yosemite to ~99% IPv6 in iOS 9 and El Capitan betas. While our previous implementation from four years ago was designed to select the connection with lowest latency no matter what, we agree that the Internet has changed since then and reports indicate that biasing towards IPv6 is now beneficial for our customers: IPv6 is now mainstream instead of being an exception, there are less broken IPv6 tunnels, IPv4 carrier-grade NATs are increasing in numbers, and throughput may even be better on average over IPv6.
(tags: apple ipv6 ip tcp networking internet happy-eyeballs ios osx)
-
lest we forget — this is a 2014-era writeup of OpenPostcode (open), Loc8 and GoCode (proprietary) as alternative options to the Eircode system
(tags: eircode openpostcode loc8 gocode ireland geocoding mapping location history open-data)
-
handy step-by-step clickthrough guide
(tags: leaf tree nature identification plant)
Outlier Detection at Netflix | Hacker News
Excellent HN thread re automated anomaly detection in production, Q&A with the dev team
(tags: machine-learning ml remediation anomaly-detection netflix ops time-series clustering)
-
A new HTTP client library for Android and Java, with a lot of nice features:
HTTP/2 and SPDY support allows all requests to the same host to share a socket. Connection pooling reduces request latency (if SPDY isn’t available). Transparent GZIP shrinks download sizes. Response caching avoids the network completely for repeat requests. OkHttp perseveres when the network is troublesome: it will silently recover from common connection problems. If your service has multiple IP addresses OkHttp will attempt alternate addresses if the first connect fails. This is necessary for IPv4+IPv6 and for services hosted in redundant data centers. OkHttp initiates new connections with modern TLS features (SNI, ALPN), and falls back to TLS 1.0 if the handshake fails. Using OkHttp is easy. Its 2.0 API is designed with fluent builders and immutability. It supports both synchronous blocking calls and async calls with callbacks.
(tags: android http java libraries okhttp http2 spdy microservices jdk)
-
via Ossian.
(tags: via:smytho tech-specs specs eircode addresses geocoding ireland mapping)
AWS Best Practices for DDoS Resiliency [pdf]
Reasonably solid white paper
Self-driving cars drive like your grandma
‘Honestly, I don’t think it will take long for other drivers to realize that self-driving cars are “easy targets” in traffic.’ — also, an insurance expert suggests that self-driving cars won’t increase premiums
-
NZ’s HDC Act gets the EFF thumbs-down
jgc on Cloudflare’s log pipeline
Cloudflare are running a 40-machine, 50TB Kafka cluster, ingesting at 15 Gbps, for log processing. Also: Go producers/consumers, capnproto as wire format, and CitusDB/Postgres to store rolled-up analytics output. Also using Space Saver (top-k) and HLL (counting) estimation algorithms.
(tags: logs cloudflare kafka go capnproto architecture citusdb postgres analytics streaming)
-
a command line tool for JVM diagnostic troubleshooting and profiling.
(tags: java jvm monitoring commandline jmx sjk tools ops)
-
‘Simplistic interactive filtering tool’ — live incremental-search filtering in a terminal window
(tags: cli shell terminal tools go peco interactive incremental-search search ui unix)