Slicer: Auto-sharding for datacenter applications
Paper from Google describing one of their internal building block services:
A general purpose sharding service. I normally think of sharding as something that happens within a (typically data) service, not as a general purpose infrastructure service. What exactly is Slicer then? It has two key components: a data plane that acts as an affinity-aware load balancer, with affinity managed based on application-specified keys; and a control plane that monitors load and instructs applications processes as to which keys they should be serving at any one point in time. In this way, the decisions regarding how to balance keys across application instances can be outsourced to the Slicer service rather than building this logic over and over again for each individual back-end service. Slicer is focused exclusively on the problem of balancing load across a given set of backend tasks, other systems are responsible for adding and removing tasks.
interesting.(tags: google sharding slicer architecture papers)
Cherami: Uber Engineering’s Durable and Scalable Task Queue in Go - Uber Engineering Blog
a competing-consumer messaging queue that is durable, fault-tolerant, highly available and scalable. We achieve durability and fault-tolerance by replicating messages across storage hosts, and high availability by leveraging the append-only property of messaging queues and choosing eventual consistency as our basic model. Cherami is also scalable, as the design does not have single bottleneck. [...] Cherami is completely written in Go, a language that makes building highly performant and concurrent system software a lot of fun. Additionally, Cherami uses several libraries that Uber has already open sourced: TChannel for RPC and Ringpop for health checking and group membership. Cherami depends on several third-party open source technologies: Cassandra for metadata storage, RocksDB for message storage, and many other third-party Go packages that are available on GitHub. We plan to open source Cherami in the near future.
(tags: cherami uber queueing tasks queues architecture scalability go cassandra rocksdb)
The Perfect Weapon: How Russian Cyberpower Invaded the U.S. - The New York Times
This is scary shit. It's amazing how Russia has weaponised transparency, but I guess it's not new to observers of "kompromat": https://en.wikipedia.org/wiki/Kompromat
(tags: kompromat russia cyberpower cyberwar security trump us-politics dnc)
Category: Uncategorized
Did the Russians “hack” the election? A look at the established facts | Ars Technica
solid roundup. There's a whole lot of evidence pointing Russia's way, basically
(tags: usa russia hacking politics security us-politics trump)
-
good preso from Percona Live 2015 on the messiness of MySQL vs UTF-8 and utf8mb4
(tags: utf-8 utf8mb4 mysql storage databases slides booking.com character-sets)
-
A new data structure for accurate on-line accumulation of rank-based statistics such as quantiles and trimmed means. The t-digest algorithm is also very parallel friendly making it useful in map-reduce and parallel streaming applications. The t-digest construction algorithm uses a variant of 1-dimensional k-means clustering to product a data structure that is related to the Q-digest. This t-digest data structure can be used to estimate quantiles or compute other rank statistics. The advantage of the t-digest over the Q-digest is that the t-digest can handle floating point values while the Q-digest is limited to integers. With small changes, the t-digest can handle any values from any ordered set that has something akin to a mean. The accuracy of quantile estimates produced by t-digests can be orders of magnitude more accurate than those produced by Q-digests in spite of the fact that t-digests are more compact when stored on disk.
Super-nice feature is that it's mergeable, so amenable to parallel usage across multiple hosts if required. Java implementation, ASL licensing.(tags: data-structures algorithms java t-digest statistics quantiles percentiles aggregation digests estimation ranking)
How Putin's cronies seized control of VKontakte
via Pinboard. grim
(tags: putin russia vk vkontacte social-media politics censorship totalitarianism)
Ask HN: Best current model routers for OpenWRT, DD-WRT, Tomato, etc.?
good hardware recommendations
A Yale history professor's 20-point guide to defending democracy under a Trump presidency — Quartz
Good advice -- let's hope it doesn't come to this. Example: '17. Watch out for the paramilitaries: When the men with guns who have always claimed to be against the system start wearing uniforms and marching around with torches and pictures of a Leader, the end is nigh. When the pro-Leader paramilitary and the official police and military intermingle, the game is over.'
(tags: trump activism government politics us-politics right-wing history hitler nazis fascism)
Commentary: The ‘Irish’ Startup Attribution Problem
Why don't Irish tech startup activity show up on a EU-wide comparisons? Turns out we tend to transition to a US-based model, with US-based management and EU-based operations and engineering, like $work does:
Successful Irish tech companies have a skewed geographic profile. This presents a data gathering problem for the data companies but its also a strong indicator of the market reality for Irish startups. The size of the local market and a focus on software business in particular means many Irish startups are transitioning to the US (some earlier and with more commitment than others), and getting backed by a spectrum of local and international VCs.
Correcting for this put Ireland's tech venture investment in the second half of 2014 at $125m, midway between Sweden and Finland, 8th in Europe overall.
Hi-tech caves bring prehistoric Sistine chapel back to life
ooh, Lascaux 4 is finally opening:
St-Cyr added: “It’s impossible for anyone to see the original now, but this is the next best thing. What is lost in not having the real thing is balanced by the fact people can see so much more of the detail of the wonderful paintings and engravings.”
Contactless credit cards vulnerable to a range of scams
Johanson said it's possible to use an RFID "gate antenna" — two electronic readers spanning a doorway, similar to the anti-theft gates in retail stores — to scan the credit cards of people passing through. With enough high-powered gates installed at key doorways in a city or across the country, someone could collect comprehensive information on people's movements, buying habits and social patterns. "These days you can buy a $500 antenna to mount in doorways that can read every card that goes through it," Johanson said.
Amazingly, these seem to be rife with holes -- they still use the legacy EMV protocol, do not require online verification with backend systems, and allow replay attacks. A Journal.ie article today claims that attackers are sniffing EMV data, then replaying it against card readers in shops in Dublin, which while it may not be true, the attack certainly seems viable...(tags: rfid security scams emv wireless contactless credit-cards replay-attacks)
Counterfeit Macbook charger teardown: convincing outside but dangerous inside
rather dramatic differences
(tags: apple macbook chargers components hardware clones counterfeit)
Developer Preview – EC2 Instances (F1) with Programmable Hardware
this is frankly amazing. Elastic FPGAs!
-
USB DAC strongly recommended by Soren Ragsdale -- EUR66
(tags: usb dac music audio hardware recommendations tips toget)
Trump’s lies have a purpose. They are an assault on democracy
Donald Trump's media strategy as a form of Surkovian control via post-truth 'destabilised perception', through deliberate flooding with fake news:
By attacking the very notion of shared reality, the president-elect is making normal democratic politics impossible. When the truth is little more than an arbitrary personal decision, there is no common ground to be reached and no incentive to look for it. To men like Surkov, that is exactly as it should be. Government policy should not be set through democratic oversight; instead, the government should “manage” democracy, ensuring that people can express themselves without having any influence over the machinations of the state. According to a 2011 openDemocracy article by Richard Sakwa, a professor of Russian and European politics at the University of Kent, Surkov is “considered the main architect of what is colloquially known as ‘managed democracy,’ the administrative management of party and electoral politics.” “Surkov’s philosophy is that there is no real freedom in the world, and that all democracies are managed democracies, so the key to success is to influence people, to give them the illusion that they are free, whereas in fact they are managed,” writes Sakwa. “In his view, the only freedom is ‘artistic freedom.’”
(tags: post-truth lies donald-trump surkov breitbart pr media news propaganda fake-news)
PayWave & PayPass deletion via RFID antenna kill
remove RFID from a payment card with a single drilled hole
(tags: rfid banking cards debit-cards credit-cards)
Fast Forward Labs: Probabilistic Data Structure Showdown: Cuckoo Filters vs. Bloom Filters
Nice comparison of a counting Bloom filter and a Cuckoo Filter, implemented in Python:
This post provides an update by exploring Cuckoo filters, a new probabilistic data structure that improves upon the standard Bloom filter. The Cuckoo filter provides a few advantages: 1) it enables dynamic deletion and addition of items 2) it can be easily implemented compared to Bloom filter variants with similar capabilities, and 3) for similar space constraints, the Cuckoo filter provides lower false positives, particularly at lower capacities. We provide a python implementation of the Cuckoo filter here, and compare it to a counting Bloom filter (a Bloom filter variant).
(tags: algorithms probabilistic approximation bloom-filters cuckoo-filters sets estimation python)
-
Football Manager includes what is effectively a parallel universe, so they modelled the effects of Brexit on the UK Premier League: 'In my own current “save”, Brexit kicked in at the end of season three. Unfortunately I got one of the hard options, where all non-homegrown players are now going through a work permit system, albeit one that’s slightly relaxed. It means I can no longer bring in that 19-year-old Italian keeper I’d been eyeing up as one for the future. Instead I have to wait for him to break into the Italian squad, and play 30% of their fixtures over the next two years. Then he’ll be mine. Meanwhile, my TV revenue has just dropped by a few million. Let’s hope that doesn’t continue, or I won’t even be able to afford him.'
(tags: brexit uk games gaming football-manager forecasts simulation)
Accidentally Quadratic — Rust hash iteration+reinsertion
It was recently discovered that some surprising operations on Rust’s standard hash table types could go quadratic.
Quite a nice unexpected accidental detour into O(n^2)(tags: big-o hashing robin-hood-hashing siphash algorithms hashtables rust)
Reproducible research: Stripe’s approach to data science
This is intriguing -- using Jupyter notebooks to embody data analysis work, and ensure it's reproducible, which brings better rigour similarly to how unit tests improve coding. I must try this.
Reproducibility makes data science at Stripe feel like working on GitHub, where anyone can obtain and extend others’ work. Instead of islands of analysis, we share our research in a central repository of knowledge. This makes it dramatically easier for anyone on our team to work with our data science research, encouraging independent exploration. We approach our analyses with the same rigor we apply to production code: our reports feel more like finished products, research is fleshed out and easy to understand, and there are clear programmatic steps from start to finish for every analysis.
(tags: stripe coding data-science reproducability science jupyter notebooks analysis data experiments)
Introducing Veneur: high performance and global aggregation for Datadog
neat -- aggregation of histograms for Datadog statsd
(tags: datadog statsd metrics percentiles ops)
-
auditd -> go-audit -> elasticsearch at Slack
Irish eyes are crying: Tens of thousands of broadband modems wide open to hijacking
Eir ship vulnerable firmware images AGAIN. ffs
(tags: eircom eir fail firmware security zyxel d1000 tr-064)
"Solving Imaginary Scaling Issues" ... "At Scale"
Amazing virtuoso performance -- be sure to scroll up all the way to Chapter 1
(tags: scalability funny lol twitter oreilly)
Webhooks do’s and dont’s: what we learned after integrating +100 APIs
interesting tips
(tags: webhooks api http https architecture)
Dynamically Scale Applications on Amazon EMR with Auto Scaling
good call -- new EMR feature
Testing@LMAX – Time Travel and the TARDIS
LMAX' approach to acceptance/system-testing time-dependent code. We are doing something similar in Swrve too, so finding that LMAX have taken a similar approach is a great indicator
(tags: lmax testing system-tests acceptance-tests tests time)
Heineken refreshed by ‘craft beer’ ruling
scumbags. Attempting to pass off their pissy beer under alternative names to con consumers into buying it! 'There will be no sanctions against Heineken for passing off non-craft beer as “locally produced”, the Food Safety Authority of Ireland (FSAI) has said. The FSAI and HSE launched a joint investigation last month after it emerged that Heineken Ireland had sold some of its products, including Foster’s lager, under craft-type names such as Blasket Blonde and Beanntrai Bru. Two well-known stouts, Beamish and Murphy’s, were also sold under craft-type names by the international brewing giant. C&C, a Tipperary-based drinks company, was also investigated after it admitted selling its Clonmel 1650 lager under a different name, Pana Cork, in Cork.'
Simple Queue Service FIFO Queues with Exactly-Once Processing & Deduplication
great, I've looked for this so many times. Only tricky limit I can spot is the 300 tps limit, and it's US-East/US-West only for now
"Get In Control Of Your Workflows With Airflow"
good intro to Airflow usage preso
(tags: airflow presentations ops workflow scheduling scheduler)
Etsy Debriefing Facilitation Guide
by John Allspaw, Morgan Evans and Daniel Schauenberg; the Etsy blameless postmortem style crystallized into a detailed 27-page PDF ebook
(tags: etsy postmortems blameless ops production debriefing ebooks)
-
'bike-shedding', or needless arguing about trivial issues, actually dates back to 1957 as C. Northcote Parkinson's 'law of triviality'
(tags: triviality bikeshed bikeshedding management arguments decisions history)
Julia Evans reverse engineers Skyliner.io
simple usage of Docker, blue/green deploys, and AWS ALBs
(tags: docker alb aws ec2 blue-green-deploys deployment ops tools skyliner via:jgilbert)
IPBill ICRs are the perfect material for 21st-century blackmail
ICRs are the perfect material for blackmail, which makes them valuable in a way that traditional telephone records are not. And where potentially large sums of money are involved, corruption is sure to follow. Even if ICR databases are secured with the best available technology, they are still vulnerable to subversion by individuals whose jobs give them ready access. This is no theoretical risk. Just one day ago, it emerged that corrupt insiders at offshore call centres used by Australian telecoms were offering to sell phone records, home addresses, and other private details of customers. Significantly, the price requested was more if the target was an Australian "VIP, politician, police [or] celebrity."
(tags: blackmail privacy uk-politics uk snooping surveillance icrs australia phone-records)
Docklands Print Commission 2016: Colin Martin
I love Colin's work. just may spring for this one
(tags: colin-martin art prints etchings dublin history vinyl)
-
a low-cost online vendor in Ireland, recommended by @irldexter on ITS (along with webdoctor.ie): 'For basic consultations I halved the cost €55 to engage a GP with https://www.webdoctor.ie/ down to €25 (for limited domains) and after paying €8.48 and €9.48 respectively for a Ventolin inhaler, I now get them for €3.50 at http://www.purepharmacy.ie/ (closer to mainland EU costs). I also benchmarked my parents medicine costs which worked out 40% cheaper too.'
(tags: recommendations pharmacy ireland doctors health medicine)
Rents dwarf Celtic Tiger era with ‘disastrous effect’ on society
“The scale of the challenge here remains depressing,” says the report. “It has never been viable to build apartment blocks in the vast majority of this country.” [...] The report notes that the rise in living costs of almost three quarters in less than five years is “a symptom of strong demand for housing” as economic recovery continues and the population grows. “But there is nothing inevitable about housing costs rising with demand,” it says. “That only happens when supply fails to respond, and the complete absence of any meaningful level of construction over the past five years is a systemic failure in desperate need of policy solutions. “There is no more urgent task facing the Minister for Housing, his department and advisers, and the Housing Agency, than understanding why the costs of building, and building apartments in particular, is so dramatically out of line with our own incomes and indeed with the cost in other countries.”
-
I'm not remotely interested in shockingly good graphics, in murder simulators, in guns and knives and swords. I'm not that interested in adrenaline. My own life is thrilling enough. There is enough fear and hatred in the world to get my heart pounding. My Facebook feed and Twitter feed are enough for that. Walking outside in summer clothing is enough for that. I'm interested in care, in characters, in creation, in finding a path forward inside games that helps me find my path forward in life. I am interested in compassion and understanding. I'm interested in connecting. As Miranda July said, "all I ever wanted to know is how other people are making it through life." I want to make games that help other people understand life. We are all overwhelmed with shock, with information, with change. The degree of interactivity in our lives is amazing and wonderful and I wouldn't exchange it for anything, but it is also shocking and overwhelming and it's causing us to dig in and try to find some peace by shutting each other out. On all sides of the political spectrum we've stopped listening to each other and I fear we are all leaning toward fascist thinking. We should be using this medium to help us adapt to our new, interactive lives. This is how we become relevant.
(tags: essay feminism society culture games gaming life art)
Tesco Bank: 20,000 customers lose money - BBC News
"Any financial loss that results from this fraudulent activity will be borne by the bank," Mr Higgins said. "Customers are not at financial risk."
Well, that would be surprising....-
Hooray for nuclear power. (via Ossian Smyth)
(tags: nukes nuclear-power power apple datacenters ireland ida)
-
Ivan's Childhood, Andrei Rublev, Solaris, The Mirror, and Stalker -- all viewable for free on YouTube thanks to Mosfilm. quality not great though....
Jeff Erickson's Algorithms, Etc.
This page contains lecture notes and other course materials for various algorithms classes I have taught at the University of Illinois, Urbana-Champaign. The notes are numbered in the order I cover the material in a typical undergraduate class, wtih notes on more advanced material (indicated by the symbol ?) interspersed appropriately. [...] In addition to the algorithms notes I have been maintaining since 1999, this page also contains new notes on "Models of Computation", which cover a small subset of the material normally taught in undergraduate courses in formal languages and automata. I wrote these notes for a new junior-level course on "Algorithms and Models of Computation" that Lenny Pitt and I developed, which is now required for all undergraduate computer science and computer engineering majors at UIUC.
Via Tony Finch(tags: via:fanf book cs algorithms jeff-erickson uiuc)
How Macedonia Became A Global Hub For Pro-Trump Misinformation - BuzzFeed News
“I started the site for a easy way to make money,” said a 17-year-old who runs a site [from Veles] with four other people. “In Macedonia the economy is very weak and teenagers are not allowed to work, so we need to find creative ways to make some money. I’m a musician but I can’t afford music gear. Here in Macedonia the revenue from a small site is enough to afford many things.”
(tags: macedonia veles scams facebook misinformation donald-trump us-politics)
-
'an antagonistic GSM base station [disguised] in the form of an innocuous office printer. It brings the covert design practice of disguising cellular infrastructure as other things - like trees and lamp-posts - indoors, while mimicking technology used by police and intelligence agencies to surveil mobile phone users.'
(tags: gsm hardware art privacy surveillance hacks printers mobile-phones)
Testing Docker multi-host network performance - Percona Database Performance Blog
wow, Docker Swarm looks like a turkey right now if performance is important. Only "host" gives reasonably perf numbers
(tags: docker networking performance ops benchmarks testing swarm overlay calico weave bridge)
-
Subreddit devoted to becoming a software developer in Ireland, with a decent wiki
Great comment on the "realism" of space photos
In short, the answer to the question “is this what it would look like if I was there?” is almost always no, but that is true of every photograph. The photos taken from space cameras are no more fake or false than the photos taken from any camera. Like all photos they are a visual interpretation using color to display data. Most space photos have information online about how they were created, what filters were used, and all kinds of interesting details about processing. The discussion about whether a space photo is real or fake is meaningless. There's no distinction between photoshopped and not. It's a nuanced view but the nature of the situation demands it.
(tags: photography photos space cassini probes cameras light wavelengths science vision realism real)
-
LOL as DST bug uncovers spurious automated noise complaints:
In January last year the airport unearthed a scheme whereby campaigners were using automated software to generate complaints against the airport. Officials caught out the set-up when the two anti-Heathrow enthusiasts forgot to take into account the hour going back in October, and began complaining about flights that had not yet taken off or arrived.
(tags: bugs dst daylight-savings-time funny heathrow complaints automation noise)
Facebook scuppers Admiral Insurance plan to base premiums on your posts
Well, this is amazingly awful:
The Guardian claims to have further details of the kind of tell-tale signs that Admiral's algorithmic analysis would have looked out for in Facebook posts. Good traits include "writing in short concrete sentences, using lists, and arranging to meet friends at a set time and place, rather than just 'tonight'." On the other hand, "evidence that the Facebook user might be overconfident—such as the use of exclamation marks and the frequent use of 'always' or 'never' rather than 'maybe'—will count against them."
The future is shitty.(tags: insurance facebook scoring computer-says-no algorithms text-analysis awful future)
MemC3: Compact and concurrent Memcache with dumber caching and smarter hashing
An improved hashing algorithm called optimistic cuckoo hashing, and a CLOCK-based eviction algorithm that works in tandem with it. They are evaluated in the context of Memcached, where combined they give up to a 30% memory usage reduction and up to a 3x improvement in queries per second as compared to the default Memcached implementation on read-heavy workloads with small objects (as is typified by Facebook workloads).
(tags: memcached performance key-value-stores storage databases cuckoo-hashing algorithms concurrency caching cache-eviction memory throughput)
Total Nightmare: USB-C and Thunderbolt 3
the coming incompatibility nightmare of USB-C cabling
(tags: usb usb-c thunderbolt apple cables hardware confusion)
Amazon ElastiCache for Redis Update – Sharded Clusters, Engine Improvements, and More | AWS Blog
Elasticache now supports sharding
(tags: elasticache sharding storage aws databases redis ops)
-
The square root staffing law is a rule of thumb derived from queueing theory, useful for getting an estimate of the capacity you might need to serve an increased amount of traffic.
(tags: ops capacity planning rules-of-thumb qed-regime efficiency architecture)
Measuring Docker IO overhead - Percona Database Performance Blog
See also https://www.percona.com/blog/2016/02/05/measuring-docker-cpu-network-overhead/ for the CPU/Network equivalent. The good news is that nowadays it's virtually 0 when the correct settings are used
(tags: docker percona overhead mysql deployment performance ops containers)
React’s license: necessary and open?
Luis Villa: 'Is the React license elegant? No. Should you be worried about using it? Probably not. If anything, Facebook’s attempt to give users an explicit patent license should probably be seen as a good faith gesture that builds some confidence in their ecosystem. But yeah, don’t use it if your company intends to invest heavily in React and also sue Facebook over unrelated patents. That… would be dumb. :)'
(tags: luis-villa open-source react facebook patents swpats licensing licenses bsd)
-
Writeup of their Consul-based service discovery system, a bit similar to smartstack. Good description of the production problems that they saw with Consul too, and also they figured out that strong consistency isn't actually what you want in a service discovery system ;) HN comments are good too: https://news.ycombinator.com/item?id=12840803
(tags: consul api microservices service-discovery dns load-balancing l7 tcp distcomp smartstack stripe cap-theorem scalability)
Here's Why Facebook's Trending Algorithm Keeps Promoting Fake News - BuzzFeed News
Kalina Bontcheva leads the EU-funded PHEME project working to compute the veracity of social media content. She said reducing the amount of human oversight for Trending heightens the likelihood of failures, and of the algorithm being fooled by people trying to game it. “I think people are always going to try and outsmart these algorithms — we’ve seen this with search engine optimization,” she said. “I’m sure that once in a while there is going to be a very high-profile failure.” Less human oversight means more reliance on the algorithm, which creates a new set of concerns, according to Kate Starbird, an assistant professor at the University of Washington who has been using machine learning and other technology to evaluate the accuracy of rumors and information during events such as the Boston bombings. “[Facebook is] making an assumption that we’re more comfortable with a machine being biased than with a human being biased, because people don’t understand machines as well,” she said.
(tags: facebook news gaming adversarial-classification pheme truth social-media algorithms ml machine-learning media)
seriot.ch - Parsing JSON is a Minefield ????
Crockford chose not to version [the] JSON definition: 'Probably the boldest design decision I made was to not put a version number on JSON so there is no mechanism for revising it. We are stuck with JSON: whatever it is in its current form, that’s it.' Yet JSON is defined in at least six different documents.
"Boldest". ffs. :facepalm:(tags: bold courage json parsing coding data formats interchange fail standards confusion)
mjg59 | Fixing the IoT isn't going to be easy
We can't easily fix the already broken devices, we can't easily stop more broken devices from being shipped and we can't easily guarantee that we can fix future devices that end up broken. The only solution I see working at all is to require ISPs to cut people off, and that's going to involve a great deal of pain. The harsh reality is that this is almost certainly just the tip of the iceberg, and things are going to get much worse before they get any better.
-
This is shocking:
At the end of the panel on artificial intelligence, a young black woman asked [Sebastian Thrun, CEO of the education startup Udacity, who is best known for founding Google X] whether bias in machine learning “could perpetuate structural inequality at a velocity much greater than perhaps humans can.” She offered the example of criminal justice, where “you have a machine learning tool that can identify criminals, and criminals may disproportionately be black because of other issues that have nothing to do with the intrinsic nature of these people, so the machine learns that black people are criminals, and that’s not necessarily the outcome that I think we want.” In his reply, Thrun made it sound like her concern was one about political correctness, not unconscious bias. “Statistically what the machines do pick up are patterns and sometimes we don’t like these patterns. Sometimes they’re not politically correct,” Thrun said. “When we apply machine learning methods sometimes the truth we learn really surprises us, to be honest, and I think it’s good to have a dialogue about this.”
"the truth"! Jesus. We are fucked(tags: google googlex bias racism implicit-bias machine-learning ml sebastian-thrun udacity inequality policing crime)
-
Holy shit.
Using a proxy, remove “securityQuestion0” and “securityQuestion1” from the post data.
Massive facepalm. -
amazing architectural-oddities Tumblr (via Present and Correct)
(tags: tumblr art photography architecture weird odd)
-
I’ve had the privilege of experiencing a few different management levels (responsibilities? jobs?) at Etsy since I’ve joined. At each stage, I felt like the job of being a manager totally changed. What I did day-to-day changed, what was hard about it changed, how I measured my own success changed, and though I feel like the experiences built on one another, it continues to be an enormous shift in brainpower each time the gig changes a bit. Given how intangible (and often hidden) management work can be, I’ve outlined some highlights of what my work has been like as a manager over the last four years. (Obvious, major caveat: this is just my experience, and there’s lots in here that is unique to this particular work environment, hierarchy, requirements, and challenges!)
(tags: business engineering management career lara-hogan managing)
JG Ballard, on the "pram in the hall"
Cyril Connolly, the 50s critic and writer, said that the greatest enemy of creativity is the pram in the hall, but I think that was completely wrong. It was the enemy of a certain kind of dilettante life that he aspired to, the man of letters, but for the real novelist the pram in the hall is the greatest ally - it brings you up sharp and you realise what reality is all about. My children were a huge inspiration for me. Watching three young minds creating their separate worlds was a very enriching experience.
(tags: writing creativity jg-ballard quotes pram-in-the-hall children kids parenting biography)
50% of American Adults Are in Police Facial-Recognition Databases
holy crap this is going to be a serious problem
(tags: facial-recognition ml algorithms policing us-politics future dystopia)
Danger is Everwhere Docter Noel Zone Halloween costume
may need this, depending on kiddie preferences this year ;)
(tags: noel-zone danger-is-everywhere books costumes halloween)
A Guide to Communication, Shotcalling, and Etiquette in Competitive Overwatch
Excellent post on team voice comms tactics. Many tips here
(tags: voice voice-comms gaming overwatch communication strats)
-
Love the 'decade of' dig at FB and Amazon -- 'we were doing it first' ;) Great details on how Google have built out and improved their DC networking. Includes a hint that they now use DCTCP (datacenter-optimized TCP congestion control) on their internal hosts....
(tags: datacenter google presentation networks networking via:irldexter ops sre clos-networks fabrics switching history datacenters)
Cloudy Gamer: Playing Overwatch on Azure's new monster GPU instances
pretty amazing. full 60FPS, 2560x1600, everything on Epic quality, streaming from Azure, for $2 per hour
Anti-Brexit traitors outed on twitter
oh god this is funny. Louise Mensch and various UKIPpers fall for transparent pisstake involving "taking Article 50 out of the ring binder and shredding it. It now goes straight from 49 to 51" etc.
Image Synthesis from Yahoo's open_nsfw
What makes an image NSFW, according to Yahoo? I explore this question with a clever new visualization technique
Deep Dream applied to an NSFW classifier. This is a bit NSFW, as it happens
Sean O'Dowd on Twitter: "damn someone at apple needs a hug"
stack trace leads with a symbol called "_thereIsNoSadnessLikeTheDeathOfOptimism"
-
interesting presentation describing how to architect Airflow ETL setups; see also https://gtoonstra.github.io/etl-with-airflow/principles.html
_Airport Noise NIMBYism: An Empirical Investigation_
'Generally, a very small number of people account for a disproportionately high share of the total number of noise complaints. In 2015, for example, 6,852 of the 8,760 complaints submitted to Ronald Reagan Washington National Airport originated from one residence in the affluent Foxhall neighborhood of northwest Washington, DC. The residents of that particular house called Reagan National to express irritation about aircraft noise an average of almost 19 times per day during 2015.' Somebody needs help.
ETL best practices with Airflow
good advice on how to ETL
(tags: etl airflow documentation best-practices batch architecture)
New Scientist: Home abortions are safe – we should let women do it themselves
the Women on Web approach is backed by a column in New Scientist: 'It’s also safer than many other medicines that we are allowed to buy from pharmacies without a prescription, such as Viagra in the UK. So why can’t women get abortion pills from pharmacies and manage the process themselves at home if they choose? It might sound radical but it’s already widespread in countries where abortion is illegal, with women buying the pills from online pharmacies. While some countries, such as Poland, are trying to tighten their already strict abortion laws, the advent of mail-order abortion pills means the law is becoming almost irrelevant.'
(tags: new-scientist safety abortion pro-choice medicine mifepristone pills poland ireland repealthe8th)
What $50 buys you at Huaqiangbei, the world’s most fascinating electronics market
This is amazing -- what a wonderland! For instance:
Six dollars for: a GSM chipset, a CPU, an LCD screen, a battery, a PCB, a metal housing, a molded silicone watch band, a MicroUSB cable, and a box. And the labor to assemble and test all of that.
(tags: gadgets crap shenzen huaqiangbei shopping hardware china)
-
The UK's version of the POD database project was used by the Home Office to track immigrants for various reasons -- in other words, exactly the reasons why parents will choose not to provide that data
(tags: parents databases data pod uk home-office education schools)
How One 19-Year-Old Illinois Man Is Distorting National Polling Averages - The New York Times
One "outlier" voter—a 19-year old black Trump supporter—was weighted so heavily that it shifted the whole poll significantly. Stats fail
(tags: statistics nytimes politics via:reddit donald-trump hilary-clinton polling panels polls)
Kafka Streams - Scaling up or down
this is a nice zero-config scaling story -- good work Kafka Streams
(tags: scaling scalability architecture kafka streams ops)
Medium’s DynamoDB Data Source for Apache Spark
nice approach
(tags: dynamodb spark architecture tools aws backups export)
-
'a Mexican cerveza preparada made with beer, lime juice, and assorted sauces, spices, and peppers. It is served in a chilled, salt-rimmed glass. There are numerous variations of this beverage throughout Mexico and Latin America.'
-
a variation on the mimosa, with an IPA and grapefruit juice -- sounds like Brewdog's Elvis Juice
-
'This crisp, refreshing beer and ginger beer cocktail is a Cuban favorite, and it's super-simple to make.'
(tags: recipes beer ginger-beer cuba cocktails)
The "Alpha Wolf" notion is outmoded and incorrect
via Saladin Ahmed -- the scientist who coined the term abandoned it as useless years ago:
The concept of the alpha wolf is well ingrained in the popular wolf literature at least partly because of my book "The Wolf: Ecology and Behavior of an Endangered Species," written in 1968, published in 1970, republished in paperback in 1981, and currently still in print, despite my numerous pleas to the publisher to stop publishing it. Although most of the book's info is still accurate, much is outdated. We have learned more about wolves in the last 40 years then in all of previous history. One of the outdated pieces of information is the concept of the alpha wolf. "Alpha" implies competing with others and becoming top dog by winning a contest or battle. However, most wolves who lead packs achieved their position simply by mating and producing pups, which then became their pack. In other words they are merely breeders, or parents, and that's all we call them today, the "breeding male," "breeding female," or "male parent," "female parent," or the "adult male" or "adult female." In the rare packs that include more than one breeding animal, the "dominant breeder" can be called that, and any breeding daughter can be called a "subordinate breeder."
(tags: biology animals wolves alpha alpha-males mra science wolf-packs society competition parenting)
The private monorail tunnel under North London
20km of private monorail owned and operated by the electricity grid, used to monitor and inspect one of the longest 400kV circuits in Europe
(tags: power monorail transport london underground tunnelling electricity cool)
AWS latency comparison: API Gateway vs Lambda vs Bare EC2
ugh, 213ms mean response overhead
(tags: aws latency lambda api-gateway architecture http)
Airfixers - Hosting without the hassle
Full AirBnB property management service in Dublin
(tags: dublin airbnb management rental services)
Mt. Gox had a chair worth $28,000
According to the bankruptcy documents, one of the assets listed is 'a chair worth 2,902,119JPY, or roughly $28,000USD.'
(tags: chairs funny mtgox scams bitcoin furniture assets bankruptcy)
Ludicrous Patent of the Week: Rectangles on a computer screen
"Chinese internet giant Tencent" have been granted a USPTO patent for drawing a box on a screen.
(tags: boxes screen tencent patents uspto funny absurd swpats via:markdennehy)
-
This is the new fear -- that FF/FG will accidentally and stupidly disengage Ireland from the EU as a side effect of trying to keep the UK happy and cross-border trade intact
How Fucked Up is Your Management?
Oh dear.
Score 1 “My management culture is fucked up” point for each of the following: We have an unlimited vacation policy; We don’t do regular 1:1s, but we have open office hours/are super available if anyone wants to chat; We don’t have a process for interviewing, we just hire awesome people when we meet them; We super care about diversity, but we don’t want to lower the bar so we just hire the best person for the job even if it means diversity suffers; We don’t have defined levels and career paths for our employees, we’re a really flat org; We don’t have formal managers for every staff member, everyone just gets their work done; We don’t have, like, HR HR, but our recruiter/office manager/only female employee is super good if you want someone to talk to; We don’t do performance improvement plans for employees that are struggling. We just have a super honest conversation about how they aren’t a good fit and fire them; We would have some hard explaining to do if our salary list accidentally became public.
(tags: startups management culture work vacation hiring office-hours managers diversity careers hr)
Google and Monotype launch Noto, an open-source typeface family for all the world’s languages
Great font factoid: 'The name “Noto” comes from the little squares that show when a font is not supported by a computer. This are often referred to as “tofu”, because of their shape, therefore the font is short for No Tofu.'
Simple testing can prevent most critical failures
Specifically, the following 3 classes of errors were implicated in 92% of the major production outages in this study and could have been caught with simple code review:
Error handlers that ignore errors (or just contain a log statement); error handlers with “TODO” or “FIXME” in the comment; and error handlers that catch an abstract exception type (e.g. Exception or Throwable in Java) and then take drastic action such as aborting the system.
(Interestingly, the latter was a particular favourite approach of some misplaced "fail fast"/"crash-only software design" dogma in Amazon. I wasn't a fan)(tags: fail-fast crash-only-software coding design bugs code-review review outages papers logging errors exceptions)
We are witnessing nothing less than a Tory reformation | Rafael Behr | Opinion | The Guardian
An excellent explanation of what is going on in the UK right now. What a nightmare:
Finally there are the self-styled buccaneers of the free-trade seas. Boris Johnson would probably cast himself as Sir Walter Raleigh – polymath, wordsmith, adventurer. That leaves Liam Fox to play Sir Francis Drake, looking for domestic glory in global circumnavigation but seen from abroad as a pirate. This is all myth and fantasy, of course. But parties have always been sustained by internal mythologies, and the task of exiting the EU is so complicated and fraught with danger that fantasy becomes a necessary comfort. As one former minister says of the puritan choristers: “They have spent their lives working towards this dream. Of course they don’t want to accept that it’s a nightmare.” Tory pro-Europeans are in the impossible position of using rational argument against faith. If they counsel compromise on migration or the single market, they are accused of talking Britain down or trying to refight the referendum. They have few reinforcements across the political water. Labour is a shambles. The Lib Dems are puny in parliament. Scotland has its own distinct politics, and in Nicola Sturgeon its own remainian queen with her own independence agenda. The Tories do not speak for all of England, but in the absence of credible opposition they feel as if they do, and will act accordingly. To those millions who did not vote to leave the EU, the message is clear: you are free to pray for whatever you like. Your antique rites will be tolerated. But do not expect your concerns to be represented in the court of Queen Theresa. Be humble instead. Swallow your doubts and take a pew in the reformed national church of Brexit.
(tags: reformation uk politics brexit eu puritanism fanaticism)
-
Martin Fowler's take on the 4 kinds of tech debt
(tags: programming design tech-debt technical-debt deadlines product ship)
Charity Majors responds to the CleverTap Mongo outage war story
This is a great blog post, spot on:
You can’t just go “dudes it’s faster” and jump off a cliff. This shit is basic. Test real production workloads. Have a rollback plan. (Not for *10 days* … try a month or two.)
The only thing I'd nitpick on is that it's all very well to say "buy my book" or "come see me talk at Blahcon", but a good blog post or webpage would be thousands of times more useful.(tags: databases stateful-services services ops mongodb charity-majors rollback state storage testing dba)
Remarks at the SASE Panel On The Moral Economy of Tech
Excellent talk. I love this analogy for ML applied to real-world data which affects people:
Treating the world as software promotes fantasies of control. And the best kind of control is control without responsibility. Our unique position as authors of software used by millions gives us power, but we don't accept that this should make us accountable. We're programmers—who else is going to write the software that runs the world? To put it plainly, we are surprised that people seem to get mad at us for trying to help. Fortunately we are smart people and have found a way out of this predicament. Instead of relying on algorithms, which we can be accused of manipulating for our benefit, we have turned to machine learning, an ingenious way of disclaiming responsibility for anything. Machine learning is like money laundering for bias. It's a clean, mathematical apparatus that gives the status quo the aura of logical inevitability. The numbers don't lie.
Particularly apposite today given Y Combinator's revelation that they use an AI bot to help 'sift admission applications', and don't know what criteria it's using: https://twitter.com/aprjoy/status/783032128653107200(tags: culture ethics privacy technology surveillance ml machine-learning bias algorithms software control)
-
So assuming the mission continues well, in 2014 the Rosetta Probe will land on Comet 67P/Churyumov-Gerasimenko, where it will measure the comet's molecular composition. Then it will remain at rest as the comet orbits the sun for hundreds of millions of years. So somewhere in the solar system, where it is safe but hard to reach, a backup sample of human languages is stored, in case we need one.
As jwz says: 'The Rosetta Disc is now safely installed on 67P/Churyumov-Gerasimenko.'(tags: rosetta long-now history language comets solar-system space)
Airflow/AMI/ASG nightly-packaging workflow
Some tantalising discussion on twitter of an Airflow + AMI + ASG workflow for ML packaging: 'We build models using Airflow. We deploy new models as AMIs where each AMI is model + scoring code. The AMI is hence a version of code + model at a point in time : #immutable_infrastructure. It's natural for Airflow to build & deploy the model+code with each Airflow DAG Run corresponding to a versioned AMI. if there's a problem, we can simply roll back to the previous AMI & identify the problematic model building Dag run. Since we use ASGs, Airflow can execute a rolling deploy of new AMIs. We could also have it do a validation & ASG rollback of the AMI if validation fails. Airflow is being used for reliable Model build+validation+deployment.'
(tags: ml packaging airflow asg ami deployment ops infrastructure rollback)
Commodore 64C going strong after over 25 years in production
C64C used in Polish auto repair shop to balance driveshafts, working non-stop for over a quarter of a century. I'd like to see a Spectrum do _this_ ;)
(tags: c64 c64c commodore history poland auto-repair production)
-
'Build Negroni over ice, top with sparkling wine, stir briefly. They'll get you messed up quick, but it's a great way to go.' aka. the Double Sbagliato. yum
(tags: negroni cocktails recipes campari prosecco sparkling-wine red-vermouth gin)
Snooping powers saw 13 people wrongly held on child sex charges in the UK
Sorry, Daily Mail article --
Blunders in the use of controversial snooping powers meant 13 people were wrongly arrested last year on suspicion of being paedophiles. Another four individuals had their homes searched by detectives following errors in attempts to access communications data, a watchdog revealed yesterday. Other mistakes also included people unconnected to an investigation being visited by police and delayed welfare checks on vulnerable people including children whose lives were at risk, said the Interception of Communications Commissioner. [....] A large proportion of the errors involved an internet address which was wrongly linked to an individual. Of the 23 serious mistakes, 14 were human errors and the other nine ‘technical system errors’.
(tags: surveillance ip-addresses privacy uk daily-mail snooping interception errors)
The best thing to mark National Stalking Awareness Week would be to scrap the law on stalking
"The Secret Barrister" explains a classic case of empty-gesture lawmaking in the UK:
in 2012, the coalition government, in a fit of virtue signalling, announced a bold plan to offer extra protection to victims of stalking, following a rash of reported cases where obsessive nutjobs had slipped through the net. Hence, via the 2012 Act, section 2A was shoved into the Protection from Harassment Act, creating a shiny new offence of stalking. What is stalking, you ask? Well here’s the clever bit. Stalking is…”a course of conduct which amounts to harassment…and [where] the acts or omissions involved are ones associated with stalking“. To inject some colour into the dull circularity of the definition, section 2A(3) provides “examples of acts or omissions associated with stalking”. In other words, you need to prove that the defendant is guilty of both harassment and stalking, in order to convict them of stalking. Therefore, proving stalking is by definition harder for the prosecution than simply proving harassment. And what do you get if you opt for the harder road? What prize awaits the victorious prosecutor who has slogged her way through the additional evidential burden thrust upon her by section 2A? The answer is….nothing. Or at least, nothing more than if you successfully prosecuted for harassment. The maximum sentence in each case is 6 months’ imprisonment. It is the very definition of empty gesture legislating. Section 2A is so very pointlessly pointless that I want urgently to go back in time to the day when then-crime prevention minister Jeremy Browne was hubristically prattling on about what a difference this law is going to make and shove a whoopee pie right up his schnoz. Section 2A does nothing other than create a new offence that is harder to prove than an existing offence that prohibits the same conduct, solely, it seems, to allow for the drawing of an entirely semantic distinction between “harassment” and “stalking”.
(tags: harrassment stalking law legislation uk police crime prosecution)
"Better truck design could save hundreds of pedestrian and cyclist lives"
European transport group, Transport and Environment, said that the Loughborough study shows that better design “could save hundreds of pedestrian and cyclists’ lives”. It added that the study “finds huge differences in the direct vision – what drivers can see with their own eyes – of best and worst-in-class trucks in all categories, and that ‘low-entry cabs’ like the Mercedes Econic out perform all of today’s best performing vehicles.” A P-Series truck, from truck maker Scania, was rated at the best of its class with zero blind spots — this could go a long way to explaining why the makers of a Road Safety Authority video using another P-Series truck reportedly had to fake blind spots last year. Mandatory extra mirrors has been EU policy to try to reduce collisions with people cycling and walking but researchers point out that blind spots remain on many trucks and improving direct vision may be a better policy than improving indirect vision using mirrors. [...] The EU currently has a deadline of 2028 for improved vision in trucks but Transport and Environment said: “Given that better vision cabs are already available on the market and in all market segments (best in class, smarter configurations, low entry vehicles) a 2028 deadline is not justifiable.”
-
good page on the Universal Scalability Law and how to apply it
(tags: usl performance scalability concurrency capacity measurement excel equations metrics)
Artist Tricks Tourists With Elaborate Monument To Staten Island Ferry Octopus Attack
'You probably don't know much about the Staten Island Ferry Disaster Memorial Museum, which honors the 400 victims who died when a giant octopus attacked the Cornelius G. Kolff, a Staten Island Ferry boat, on Nov. 22, 1963. That isn't because the event was overshadowed by the assassination of JFK that same day—it's because, as you may have guessed based on the word "tricks" in the headline, there was no such octopus-induced tragedy.'
-
great idea -- donate old, obsolete iPhone 4/4s phones to a charity which repurposes them for autistic/non-verbal kids
(tags: autism communication health phones recycling charity iphones)
Brian Krebs - The Democratization of Censorship
Events of the past week have convinced me that one of the fastest-growing censorship threats on the Internet today comes not from nation-states, but from super-empowered individuals who have been quietly building extremely potent cyber weapons with transnational reach. More than 20 years after Gilmore first coined [his] turn of phrase, his most notable quotable has effectively been inverted — “Censorship can in fact route around the Internet.” The Internet can’t route around censorship when the censorship is all-pervasive and armed with, for all practical purposes, near-infinite reach and capacity.
(tags: brian-krebs censorship ddos internet web politics crime security iot)
SMC Education Blog — Girls and Their Frenemies
on "relational aggression" among schoolkids
"The couple, who had no experience of wine-making but much faith in professorial expertise…"
I love this story -- a wealthy couple buy a vineyard in the Languedoc for its theoretically-optimal microclimate for wine-making. Defying what one's preconceptions would expect (mine included!), the results were fantastic.
In the Languedoc there is a vineyard that teaches us an important lesson about textbook learning and its application to the world. In the early Seventies it was bought by a wealthy couple, who consulted professors Emile Peynaud and Henri Enjalbert, the world’s leading academic oenologist and oenological geologist respectively. Between them these men convinced the couple that their new vineyard had a theoretically ideal microclimate for wine-making. When planted with theoretically ideal vines whose fruits would be processed in the optimal way according to the up-to-date science of oenology, this vineyard had the potential to produce wine to match the great first growths of Bordeaux. The received wisdom that great wine was the product of an inscrutable (and untransferable) tradition was quite mistaken, the professors said: it could be done with hard work and a fanatical attention to detail. The couple, who had no experience of wine-making but much faith in professorial expertise, took a deep breath and went ahead. If life were reliably like novels, their experiment would have been a disaster. In fact Aimé and Véronique Guibert have met with a success so unsullied that it would make a stupefying novel (it has already been the subject of a comatogenic work of non-fiction). The first vintage they declared (in 1978) was described by Gault Millau as ‘Château Lafite du Languedoc’; others have been praised to the heights by the likes of Hugh Johnson and Robert Parker. The wine is now on the list at the Tour d’Argent and the 1986 vintage retails at the vineyard for £65 a bottle. The sole shadow on the lives of these millionaires is cast by the odd hailstorm. No one to whom I have begun recounting the story believes it will end well. Most people are extremely unwilling to grant that faith in textbook knowledge should ever be crowned with success. We have a very strong narrative bias against such stories. It is a bias we forget once our children fall sick or we have to travel in an aeroplane, but so long as we are in storytelling mode we simply deny that systematic textbook reasoning can make headway against whimsy and serendipity. Apart from anything else, it is deeply unfair that it should.
(tags: books science languedoc wine academia microclimates preconceptions)
The Problem With Cul-de-Sac Design - CityLab
“A lot of people feel that they want to live in a cul-de-sac, they feel like it’s a safer place to be,” Marshall says. “The reality is yes, you’re safer – if you never leave your cul-de-sac. But if you actually move around town like a normal person, your town as a whole is much more dangerous.” This is the opposite of what traffic engineers (and home buyers) have thought for decades. And it’s just the beginning of what we’re now starting to understand about the relative advantages of going back to the way we designed communities a century ago. Marshall and Garrick took the same group of California cities and also examined all their minutely classified street networks for the amount of driving associated with them. On average, they found, people who live in more sparse, tree-like communities drive about 18 percent more than people who live in dense grids. And that’s a conservative calculation.
(via Tony Finch)(tags: cul-de-sacs cities city design layout simcity grids safety)
-
"A modern standard for event-oriented data". Avro schema, events have time and type, schema is external and not part of the Avro stream. 'a modern standard for representing event-oriented data in high-throughput operational systems. It uses existing open standards for schema definition and serialization, but adds semantic meaning and definition to make integration between systems easy, while still being size- and processing-efficient. An Osso event is largely use case agnostic, and can represent a log message, stack trace, metric sample, user action taken, ad display or click, generic HTTP event, or otherwise. Every event has a set of common fields as well as optional key/value attributes that are typically event type-specific.'
(tags: osso events schema data interchange formats cep event-processing architecture)
Highly Available Counters Using Cassandra
solid discussion of building HA counters using CRDTs and similar eventually-consistent data structures
(tags: crdts algorithms data-structures cassandra ha counters)
J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required"
Some good slides with tips on running java apps in production in Docker
(tags: java docker ops containers)
Algorithmic management as the new Taylorism
'its legacy can be seen in factories, call centres and warehouses today, although new technology has taken the place of Taylor’s instruction cards and stopwatches. Many warehouse workers for companies such as Amazon use handheld devices that give them step-by-step instructions on where to walk and what to pick from the shelves when they get there, all the while measuring their “pick rate” in real time. For Jeremias Prassl, a law professor at Oxford university, the algorithmic management techniques of Uber and Deliveroo are Taylorism 2.0. “Algorithms are providing a degree of control and oversight that even the most hardened Taylorists could never have dreamt of,” he says.'
(tags: algorithms labour work labor taylorism management silicon-valley tech deliveroo uber piece-work)
[Cryptography] Bridge hand record generator cracked
'How to cheat at Bridge by breaking the tournament card-dealing random number generator', via Tony Finch
(tags: crypto security rngs prngs random bridge cards via:fanf)
A Loud Sound Just Shut Down a Bank's Data Center for 10 Hours | Motherboard
The purpose of the drill was to see how the data center's fire suppression system worked. Data centers typically rely on inert gas to protect the equipment in the event of a fire, as the substance does not chemically damage electronics, and the gas only slightly decreases the temperature within the data center. The gas is stored in cylinders, and is released at high velocity out of nozzles uniformly spread across the data center. According to people familiar with the system, the pressure at ING Bank's data center was higher than expected, and produced a loud sound when rapidly expelled through tiny holes (think about the noise a steam engine releases). The bank monitored the sound and it was very loud, a source familiar with the system told us. “It was as high as their equipment could monitor, over 130dB”. Sound means vibration, and this is what damaged the hard drives. The HDD cases started to vibrate, and the vibration was transmitted to the read/write heads, causing them to go off the data tracks. “The inert gas deployment procedure has severely and surprisingly affected several servers and our storage equipment,” ING said in a press release.
(tags: ing hardware outages hard-drives fire fire-suppression vibration data-centers storage)
Fuck Your Noguchi Coffee Table
"fuck your bookshelf with all of the spines facing in"
(tags: funny fuck-this noguchi coffee-tables furniture design)
-
'All the information you need while traveling including visa requirements, currency, electricity, communication info and more.'
The Internet Thinks I’m Still Pregnant - The New York Times
This is pretty awful -- an accidental, careless and brutal side effect of marketers passing on sensitive info to one another, without respect for their users' privacy: 'I hadn’t realized, however, that when I had entered my information into the pregnancy app, the company would then share it with marketing groups targeting new mothers. Although I logged my miscarriage into the app and stopped using it, that change in status apparently wasn’t passed along. Seven months after my miscarriage, mere weeks before my due date, I came home from work to find a package on my welcome mat. It was a box of baby formula bearing the note: “We may all do it differently, but the joy of parenthood is something we all share.”'
(tags: privacy pregnancy miscarriage data-protection apps babies parenthood)
Peco Beermakers Professional Boiler 32lt (Includes Hop Strainer) - The Homebrew Company
on the list
Tekkonkinkreet Art Book Shinji Kimura - White Side: Shinji Kimura: 9784870317659: Amazon.com: Books
Beautiful background art from a 2006 anime by Shinji Kimura, as a 10" x 7" full-colour hardback art book.
did Google blacklist their own site?
The leading theory that I've seen going around is that Google is actually blocking all links in any FeedBurner feed, because it's a violation of its own terms of service. Seriously. The link-shortener "goo.gl", run by Google, is blocking all URLs generated by Feedburner, run by Google. pic.twitter.com/IR7wrlv6xj — Great Again Also (@agentdero) September 6, 2016 That's because Google's URL shortener's terms of service bans "URL re-directors" and it appears that the genius engineers at Google have decided that Google-run FeedBurner is nothing more than a URL re-director and killed off everyone's links without notice or explanation. This despite the fact that they're the same damn company and that FeedBurner unilaterally moved everyone's RSS feed to use Goo.gl links in the first place.
(tags: urls url-shorteners history goo.gl google feedburner redirectors rss links)
-
Coda Hale's new gig on how they're using Docker, AWS, etc. I like this: "Use containers. Not too much. Mostly for packaging."
Lessons Learned from Using Regexes At Scale
great post from Loggly on production usage of regular expressions on shared, multitenant architecture, where a /.*/ can really screw things up. "NFA isn't a golden ticket" paragraph included
Auto Scaling for EC2 Spot Fleets
'we are enhancing the Spot Fleet model with the addition of Auto Scaling. You can now arrange to scale your fleet up and down based on a Amazon CloudWatch metric. The metric can originate from an AWS service such as EC2, Amazon EC2 Container Service, or Amazon Simple Queue Service (SQS). Alternatively, your application can publish a custom metric and you can use it to drive the automated scaling.'
(tags: asg auto-scaling ec2 spot-fleets ops scaling)
How a Japanese cucumber farmer is using deep learning and TensorFlow
Unfortunately the usual ML problem arises at the end:
One of the current challenges with deep learning is that you need to have a large number of training datasets. To train the model, Makoto spent about three months taking 7,000 pictures of cucumbers sorted by his mother, but it’s probably not enough. "When I did a validation with the test images, the recognition accuracy exceeded 95%. But if you apply the system with real use cases, the accuracy drops down to about 70%. I suspect the neural network model has the issue of "overfitting" (the phenomenon in neural network where the model is trained to fit only to the small training dataset) because of the insufficient number of training images."
In other words, as with ML since we were using it in SpamAssassin, maintaining the training corpus becomes a really big problem. :((tags: google machine-learning tensorflow cucumbers deep-learning ml)
Northland man denies burning down house but insurer refuses to pay out
This is a mad story. The insurance company is accusing a guy in NZ of using remote-login software from 400km away to trigger a "print" command to a complicated Heath Robinson setup in order to light a fire to burn down his house
(tags: fraud insurance weird nz crime printers remote-login)
Google Intrusion Detection Problems
'We have lost access to multiple critical data stores because Google has an automated threat detection system that is incapable of handling false positives.'
(tags: google security cloud false-positives intrusion-detection automation fail)
Engineering Intelligence Through Data Visualization at Uber
bloody hell, Uber have a 15-person dataviz team. More money than sense! The resulting output is pretty though
How the NSA snooped on encrypted Internet traffic for a decade | Ars Technica
In a revelation that shows how the National Security Agency was able to systematically spy on many Cisco Systems customers for the better part of a decade, researchers have uncovered an attack that remotely extracts decryption keys from the company's now-decommissioned line of PIX firewalls. The discovery is significant because the attack code, dubbed BenignCertain, worked on PIX versions Cisco released in 2002 and supported through 2009. Even after Cisco stopped providing PIX bug fixes in July 2009, the company continued offering limited service and support for the product for an additional four years. Unless PIX customers took special precautions, virtually all of them were vulnerable to attacks that surreptitiously eavesdropped on their VPN traffic.
NPR Website To Get Rid Of Comments
Sadly, this makes sense and I'd have to agree.
Mike Durio, of Phoenix, seemed to sum it up in an email to my office back in April. "Have you considered doing away with the comments sections, or tighter moderation?" he wrote. "The comments have devolved into the Punch-and-Judy-Fest of moronic, un-illuminating observations and petty insults I've seen on other pretty much every other Internet site that allows comments." He added, "This is not in keeping with NPR's take-a-step-back, take-a-deep-breath reporting," and noted, "Now, thread hijacking and personal insults are becoming the stock in trade. Frequent posters use the forums to duke it out with one another." A user named Mary, from Raleigh, N.C., wrote to implore: "Remove the comments section from your articles. The rude, hateful, racist, judgmental comments far outweigh those who may want to engage in some intelligent sideline conversation about the actual subject of the article. I am appalled at the amount of 'free hate' that is found on a website that represents honest and unbiased reporting such as NPR. What are you really gaining from all of these rabid comments other than proof that a sad slice of humanity that preys on the weak while spreading their hate?"
Meeting the Free Speech Crusaders Who Want to End Political Correctness | VICE | United Kingdom
The 'Young British Heritage Society', aka gam*rgate as a college society
(tags: gamergate funny sad trolls ybhs reactionaries uk politics)
The Mattress Industry is One Big Scam
yes, yes it is
(tags: mattresses scams buying shopping consumer)
Unchecked exceptions for IO considered harmful - Google Groups
Insightful thread from the mechanical sympathy group, regarding the checked-vs-unchecked style question:
Peter Lawrey: Our view is that Checked Exception makes more sense for library writers as they can explicitly pass off errors to the caller. As a caller, especially if you are new to a product, you don't understand the exceptions or what you can do about them. They add confusion. For this reason we use checked exceptions internally in the lower layers and try to avoid passing them in our higher level interfaces. Note: A high percentage of our fall backs are handling iOExceptons and recovering from them. [....] My experience is that the more complex and layered your libraries the more essential checked exceptions become. I see them as essential for scalability of your software.
(tags: exceptions java style coding checked-exceptions ioexceptions io error-handling)
TV detector vans may have been a con all along
This is shaking my world view -- although I find it more plausible that (as responses to https://www.theguardian.com/notesandqueries/query/0,5753,-22440,00.html claim) they _did_ work until about 10-20 years ago, by detecting RF emissions from the local oscillator inside the TV. Ross Anderson, at https://www.cl.cam.ac.uk/~rja14/Papers/SE-15.pdf , notes:
During [..] World War II, radio engineering saw advances in radar, passive direction finding, and low-probability-of-intercept techniques, which I’ll discuss in the next chapter. By the 1960s, the stray RF leaking from the local oscillator signals in domestic television sets was being targeted by direction-finding equipment in “TV detector vans,” in Britain, where TV owners must pay an annual license fee that is supposed to support public broadcast services. Its use has since expanded to satellite and cable TV operators, who use detector vans to find pirate decoders. Some people in the computer security community were also aware that information could leak from cross-coupling and stray RF (see, for example, [259, 791]).
(tags: rf radio tv bbc tv-licenses tv-license-detector-vans security emissions tempest)
“I Want to Know What Code Is Running Inside My Body” — Backchannel
Sandler wants to be able to explore the code running her device for programming flaws and vulnerability to hacking, but she can’t. “Because I don’t have access to the source code, I have no power to do anything about it,” she says. In her eyes, it’s a particularly obvious example of a problem that now cuts across much of modern life: proprietary software has become crucial to daily survival, and yet is often locked away from public exploration and discussion by copyright.
(tags: copyright safety health pacemakers law proprietary-software life medicine implants)
-
'Researchers learn about wire-fraud scam after Nigerian scammers infect themselves with their own malware.'
The researchers observed Wire-Wire scores of $5,000 to $250,000 with the average between $30,000-$50,000 from small- and medium-sized businesses. The scammers themselves were "well-respected and admired" in their communities.
I've heard about this scam -- it's nasty, and worst of all, banks won't reimburse the losses. Showing bottle: one man's vision crafted a revolution
A eulogy for Oliver Hughes, founder of the Porterhouse and Dingle Distillery, and arguably the progenitor of Ireland's craft beer scene. I had the pleasure of sharing a table with him at a beer tasting in Sweeney's off license a while back, and it was both educational and a good fun night. RIP
(tags: oliver-hughes porterhouse beer ireland dublin dingle-distillery rip deaths)
-
'FakeTime is simulated time."
When testing RealTime software a simulator is often employed, which injects events into the program which do not occur in RealTime. If you are writing software that controls or monitors some process that exists in the real world, it takes a long time to test it. But if you simulate it, there is no reason in the simulated software (if it is disconnected from the real world completely) not to make the apparent system time inside your software appear to move at a much faster rate. For example, I have written simulators that can verify the operational steps taken by industrial controllers over a 12 hour FakeTime period, which executes in 60 seconds. This allows me to run '12 hours' of fake time through my test cases and test scenarios, without waiting 12 hours for the testing to complete. Of course, after a successful fakeTime test, an industrial RealTime system still needs to be tested in non-simulated fashion.
-
'Event driven Diagnostic and Remediation Platform' -- aka 'runbooks as code'
(tags: runbooks winston netflix remediation outages mttr ops devops)
International Olympic Committee bans GIFs
hahaha. gtfo, IOC
Ratas - A hierarchical timer wheel
excellent explanation and benchmarks of a timer wheel implementation
(tags: timer-wheels timing-wheels algorithms c linux timers data-structures)
-
ECS, Docker, ELB, SQS, SNS, RDS, VPC, and spot instances. Pretty canonical setup these days...
The mytaxi app is also now able to predict daily and weekly spikes. In addition, it has gained the elasticity required to meet demand during special events. Herzberg describes a typical situation on New Year's Eve: “Shortly before midnight everyone needs a taxi to get to parties, and after midnight people want to go home. In past years we couldn't keep up with the demand this generated, which was around three and a half times as high as normal. In November 2015 we moved our Docker container architecture to Amazon ECS, and for the first time ever in December we were able to celebrate a new year in which our system could handle the huge number of requests without any crashes or interruptions—an accomplishment that we were extremely proud of. We had faced the biggest night on the calendar without any downtime.”
(tags: mytaxi aws ecs docker elb sqs sns rds vpc spot-instances ops architecture)
Exit Scam Survival Guide : Buttcoin
Bitcoin lols:
Honesty is most important. Be sure to carefully explain that (excluding the mountain of evidence to the contrary) there was no way to foresee the [Bitcoin] exchange hacking. Practice phrases like, "this operation was the most trustworthy exchange running out of a vacant building in Singapore" and "no we can't just call the exchange, they don't have a phone number". If your significant other criticizes your decision to buy cryptocurrencies, be sure to fall back on technical merits of cryptocurrencies. Mention, "it's backed by math" and "[insert cryptocurrency here] didn't fail, people failed".
(tags: bitcoin buttcoin lol funny cryptocurrency security exchanges)
-
awesome resource.
This WIKI collects information about prepaid (or PAYG) mobile phone plans from all over the world. Not just any plans though, they must include good data rates, perfect for smartphone travellers, as well as tablet or mobile modem users.
-
'aw yiss comic generator'. AW YISS
(tags: aw-yiss memes meme-generators funny kate-beaton)
Photographer Files $1 Billion Suit Against Getty for Licensing Her Public Domain Images
Massive, massive copyright fail by Alamy and Getty Images.
Since each violation of copyright in this case allows the plaintiff to seek damages up to $25,000, the statutory damages for Getty’s 18,755 violations amount to $468,875,000. But because the company was found to have violated the same copyright law within the past three years — in 2013, Daniel Morel was awarded $1.2 million in a suit against Getty, after the agency pulled his photos from Twitter and distributed them without permission to several major publications — Highsmith can elect to seek three times that amount: hence the $1 billion suit. “The economic damage that Ms. Highsmith has suffered includes, without limitation, any and all revenue received by the Defendants based on purported licenses sold for the Highsmith Photos. These funds represent money that Ms. Highsmith could have received had she attempted to monetize her photos through the Defendants,” the complaint states. “The injury to Ms. Highsmith’s reputation has been … severe,” it continues. “There is at least one example of a recipient of a threatening letter for use of a Highsmith Photo researching the issue and determining that Ms. Highsmith had made her photos freely available and free to use through the Library website. … Therefore, anyone who sees the Highsmith Photos and knows or learns of her gift to the Library could easily believe her to be a hypocrite.”
(tags: getty alamy images copyright licensing relicensing public-domain carol-highsmith)
Why Uber Engineering Switched from Postgres to MySQL
Uber bringing the smackdown for the HN postgres fanclub, with some juicy technical details of issues that caused them pain. FWIW, I was bitten by crappy postgres behaviour in the past (specifically around vacuuming and pgbouncer), so I've long been a MySQL fan ;)
(tags: database mysql postgres postgresql uber architecture storage sql)
Noirmoutier Indigo Campsite, France
As recommended by J & F: 'Most of the campsites we've stayed in have had great facilities for kids - pools, activities, entertainment etc - but the problem with that is you spend your day being dragged from one to the other. There's none of that at Camping Indigo in Noirmoutier apart from a playground, some kayaks and some music in the bar at night but it is on the beach so the kids either run wild around the campsite or play on the beach - it was the best and most relaxing holiday we ever had and we definitely met the coolest people there. There's a really nice town in the centre of the island and great beaches all around it so hire bikes and roam free.' Bookmarking for next year's holiday planning!
(tags: holidays fun france camping noirmoutier chaize-wood loire nantes recommendations)
-
his Monitorama 2016 talk, talking about the "deep health checks" concept (which I implemented at Swrve earlier this year ;)
(tags: monitorama health deep-health-checks healthz testing availability reliability)
-
I never knew we had a native take on the sauna, the “teach alluis”:
Sweathouses were used for the treatment for a wide range of ailments up to the late 19th and early 20th centuries, primarily rheumatism but also including sciatica, lameness, sore eyes, gout, skin disorders, psychiatric disorders, impotence and infertility. Surviving records indicate that treatment was often a group activity for 4-8 persons. The sweathouse was heated by filling the interior with fuel (turf, heather, wood etc. as available), and firing the structure for a period of up to two days to heat the stone structure, the hot ashes were then raked out and the interior floor lined with bracken, grass or straw. The bathers entered and blocked the entrance with turves, clothes or some other means. The sweating period could last a number of hours while the structure retained heat. Some authors note that water was thrown on hot stones to create steam. Afterwards, the “patients” would either take a cold plunge in the nearby water source, or go home and rest for a few hours, or simply return to their normal daily activities.
(via Aileen)(tags: via:aileen sweating sweat-houses irish history saunas heat)
-
excellent recipe for this classic cocktail
(tags: cocktails recipes old-fashioned booze)
-
meditations on this classic cocktail (with solid recipes)
-
Course notes from Gerald Jay Sussman's "Adventures in Advanced Symbolic Programming" class at MIT. Hard to argue with this:
The syntax of the regular-expression language is awful. There are various incompatable forms of the language and the quotation conventions are baroquen [sic]. Nevertheless, there is a great deal of useful software, for example grep, that uses regular expressions to specify the desired behavior. Although regular-expression systems are derived from a perfectly good mathematical formalism, the particular choices made by implementers to expand the formalism into useful software systems are often disastrous: the quotation conventions adopted are highly irregular; the egregious misuse of parentheses, both for grouping and for backward reference, is a miracle to behold. In addition, attempts to increase the expressive power and address shortcomings of earlier designs have led to a proliferation of incompatible derivative languages.
(via Rob Pike's twitter: https://twitter.com/rob_pike/status/755856685923639296)(tags: regex regexps regular-expressions functional combinators gjs rob-pike coding languages)
A Cute Internet Star Flirts. All He Wants Is Your Password. - The New York Times
whoa.
Mr. Johnson’s fans are not naïve. Handing over their passwords to some strange, cute boy actually constitutes a minor act of youthful rebellion. The whole encounter delivers a heady mix of intimacy and transgression — the closest digital simulation yet to a teenage crush.
(via Adam Shostack)(tags: via:adam-shostack passwords authentication security teens rebellion)
Just As We Warned: A Chinese Tech Giant Goes On The Patent Attack -- In East Texas | Techdirt
Techdirt has been warning for years that the West's repeated demands for China to "respect" patents could backfire badly. [...] And guess what? That is exactly what has just happened, as The Wall Street Journal reports: 'Huawei Technologies Co. said it has filed a lawsuit against T-Mobile US Inc., alleging the U.S. telecommunications carrier violated the Chinese company’s patents related to wireless networks. In its complaint filed this week in the U.S. District Court for the Eastern District of Texas, Huawei said T-Mobile is using its patented technology without signing a licensing agreement.'
At least this is the most likely scenario to result in patent reform, finally.A New Wrinkle in the Gig Economy: Workers Get Most of the Money - The New York Times
So using money from the sale of iStock to Getty, she and Mr. Livingstone set out to create Stocksy, paying photographers 50 to 75 percent of sales. That is well above the going rate of 15 to 45 percent that is typical in the stock photography field. The company also distributes 90 percent of its profit at the end of each year among its photographers. Stocksy is part of a new wave of start-ups that are borrowing the tools of Silicon Valley to create a more genuine “sharing” economy that rewards the individuals generating the value.
(tags: stocksy stock-photos photos fair sharing photography work)
-
(a) 83.5% uptime over 24 hours. GOOD JOB (b) excellent marketing by Datadog!
(tags: datadog games monitoring pokemon-go pokemon uptime)
-
eye-poppingly bizarre half-assed safety features of the 1950s -- a megaton nuclear weapon rendered safe from accidental criticality accidents only by a plastic bag full of ball bearings
(tags: nuclear-weapons nukes safety 1950s uk funny bizarre violet-club ball-bearings via:cstross)
Frankly Useless Crank “Knowledge,” Only For Fools
A wonderfully-sweary post on the etymology of swear words, and how they're not derived from acronyms, really.
shit? Also from an old Germanic root, descended equally to modern German Scheiss (which sounds closer to Scots shite). It shows up in Old English, fully inflected: “Wiþ þon þe men mete untela melte & gecirre on yfele wætan & scittan” (that scittan is an infinitive form of ‘shit’ and was said like “shit-tan”). I can assure you that an acronym Ship High In Transit – supposedly meaning that manure was to be loaded in the upper parts of ships – was not possible in the language in the Old English period, not just because transit was not borrowed from Latin until half a millennium later, or because they didn’t use acronyms like that then, but because what the fuck are you even thinking. They didn’t need to ship manure. Animals produce it on the spot everywhere. Holy shit, fucking seriously.
QA Instability Implies Production Instability
Invariably, when I see a lot of developer effort in production support I also find an unreliable QA environment. It is both unreliable in that it is frequently not available for testing, and unreliable in the sense that the system’s behavior in QA is not a good predictor of its behavior in production.
(tags: qa testing architecture patterns systems production)
-
Doorman is a solution for Global Distributed Client Side Rate Limiting. Clients that talk to a shared resource (such as a database, a gRPC service, a RESTful API, or whatever) can use Doorman to voluntarily limit their use (usually in requests per second) of the resource. Doorman is written in Go and uses gRPC as its communication protocol. For some high-availability features it needs a distributed lock manager. We currently support etcd, but it should be relatively simple to make it use Zookeeper instead.
From google -- very interesting to see they're releasing this as open source, and it doesn't rely on G-internal services(tags: distributed distcomp locking youtube golang doorman rate-limiting rate-limits limits grpc etcd)
-
'based my observations while I was a Site Reliability Engineer at Google', courtesy of Rob Ewaschuk
. Seem pretty reasonable (tags: monitoring sysadmin alerting alerts nagios pager ops sre rob-ewaschuk)
-
'Best Plex Media Server' -- this looks pretty superb for EUR240 or thereabouts
(tags: media-servers plex video home tv toget nvidia shield android)
-
'a small library to manage encrypted secrets using asymmetric encryption.'
The main benefits provided by ejson are: Secrets can be safely stored in a git repo. Changes to secrets are auditable on a line-by-line basis with git blame. Anyone with git commit access has access to write new secrets. Decryption access can easily be locked down to production servers only. Secrets change synchronously with application source (as opposed to secrets provisioned by Configuration Management). Simple, well-tested, easily-auditable source.
(tags: crypto security credentials encryption ejson json configuration config)
The mysterious syndrome impairing astronauts’ sight - The Washington Post
Visual impairment intracranial pressure syndrome (VIIP) is named for the leading theory to explain it. On Earth, gravity pulls bodily fluids down toward the feet. That doesn’t happen in space, and it is thought that extra fluid in the skull increases pressure on the brain and the back of the eye.
Designing the Perfect Anti-Object
This pale, amorphous lump of sculpted concrete is designed to resist almost everything in a city that it might come into contact with. Named for the London authority that commissioned it, the Camden Bench has a special coating which makes it impervious to graffiti and vandalism. The squat, featureless surface gives drug dealers nowhere to hide their secret caches. The angled sides repel skateboarders and flyposters, litter and rain. The cambered top throws off rough sleepers. In fact, it is specially crafted to make sure that it is not used as anything except a bench. This makes it a strange artifact, defined far more by what it is not than what it is. The Camden Bench is a concerted effort to create a non-object.
(tags: non-objects objects city camden benches vandalism skating london)
The Apollo 11 AGC source code was uploaded to Github, and someone opened an issue
For the famous Apollo 13 near-fatal failure scenario:
'A customer has had a fairly serious problem with stirring the cryogenic tanks with a circuit fault present. To reproduce: Build CSM; Perform mission up to translunar coast; During translunar coast, attempt to stir cryo tanks If a wiring fault exists, the issue may be replicated. Be aware that this may be hazardous to the tester attempting it.' Sample response: 'Does it happens only with translunar coast (sol-3-a), or any moon coasting? It may be a problem with the moon. Just trying to narrow down the issue.'
(tags: lol funny apollo apollo-11 apollo-13 agc history space github)
Law to allow snooping on social media defies European court ruling
Karlin on fire:
But there’s lots in this legislation that should scare the public far more. For example, the proposal that the legislation should allow the retention of “superfluous data” gathered in the course of an investigation, which is a direct contravention of the ECJ’s demand that surveillance must be targeted and data held must be specifically relevant, not a trawl to be stored for later perusal “just in case”. Or the claim that interception and retention of data, and access to it, will only be in cases of the most serious crime or terrorism threats. Oh, please. This was, and remains, the supposed basis for our existing, ECJ-invalidated legislation. Yet, as last year’s Gsoc investigation into Garda leaks revealed, it turns out a number of interconnected pieces of national legislation allow at least 10 different agencies access to retained data, including Gsoc, the Competition Authority, local authorities and the Irish Medicines Board.
(tags: surveillance ireland whatsapp viber snowden snooping karlin-lillington facebook internet data-retention)
Raintank investing in Graphite
paying Jason Dixon to work on it, improving the backend, possibly replacing the creaky Whisper format. great news!
(tags: graphite metrics monitoring ops open-source grafana raintank)
conventional-changelog-atom 502 Bad Gateway · Issue #13284 · npm/npm
npm down for most of the (EU) day. What a shitshow
(tags: npm fail javascript dependencies coding)
Camille Fournier's excellent rant on microservices
I haven’t even gotten into the fact that your microservices are an inter-dependent environment, as much as you may wish otherwise, and one service acting up can cause operational problems for the whole team. Maybe if you have Netflix-scale operational hardening that’s not a problem. Do you? Really? Is that the best place to spend your focus and money right now, all so teams can throw shit against the wall to see if it sticks? Don’t sell people fantasies. This is not the reality for a mid-sized tech team working in microservices. There are enough valuable components to building out such a system without the fantastical claims of self-organizing teams who build cool hack projects in 2 week sprints that change the business. Microservices don’t make organizational problems disappear due to self-organization. They allow for some additional degrees of team and process independence and force very explicit decoupling, in exchange, there is overall system complexity and overall system coordination overhead. I personally think that’s enough value, especially when you are coming from a monolith that is failing to scale, but this model is not a panacea.
(tags: microservices rants camille-fournier architecture decoupling dependencies)
-
quotable: "I spend a lot of time on this task. I should write a program automating it!"
-
Quotable: "how long can work on making a routine task more efficient before you're spending more time than you save?"
(tags: quotes time automation hacks life imdb productivity efficiency)
Why Did Yankee Doodle Call a Feather ‘Macaroni’? | Mental Floss
history!
(tags: usa history macaroni yankee-doodle language dandies 18th-century)
-
John Rauser on this oft-cited dictum of percentile usage in monitoring, and when it's wrong and it's actually possible to reason with averaged percentiles, and when it breaks down.
(tags: statistics percentiles quantiles john-rauser histograms averaging mean p99)
MRI software bugs could upend years of research - The Register
In their paper at PNAS, they write: “the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results.” For example, a bug that's been sitting in a package called 3dClustSim for 15 years, fixed in May 2015, produced bad results (3dClustSim is part of the AFNI suite; the others are SPM and FSL). That's not a gentle nudge that some results might be overstated: it's more like making a bonfire of thousands of scientific papers. Further: “Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian shape”. The researchers used published fMRI results, and along the way they swipe the fMRI community for their “lamentable archiving and data-sharing practices” that prevent most of the discipline's body of work being re-analysed. ®
(tags: fmri science mri statistics cluster-inference autocorrelation data papers medicine false-positives fps neuroimaging)
-
'a Ruby regular expression editor and tester'. Great for prototyping regexps with a little set of test data, providing a neat permalink for the results
Stick Insect Eggs - Live Bug Kits
going to do this with the kids next!
(tags: stick-insects pets animals insects)
-
by avoiding division
(tags: coding algorithms performance optimization shuffle shuffling)
Self-driving cars: overlooking data privacy is a car crash waiting to happen
Interesting point -- self-driving cars are likely to be awash in telemetry data, "phoned home"
(tags: self-driving cars vehicles law data privacy data-privacy surveillance)
Push notifications delayed, Hearbeat Interval not reliable - Google Product Forums
Good thread on GCM notifications and their interactions with NAT -- they are delivered over a single TCP connection to port 5228 to the google servers, kept alive, and NAT timeouts can hang the conn resulting in delayed notifications. Particularly useful is the *#*#426#*#* dial code, which displays a log screen on Android devices with GCM debugging info.
USE Method: Linux Performance Checklist
Really late in bookmarking this, but has some up-to-date sample commandlines for sar, mpstat and iostat on linux
(tags: linux sar iostat mpstat cli ops sysadmin performance tuning use metrics)
My kids don't have a YouTube channel — but they pretend they do
“Dad is making a right turn now,” my 5-year-old son Jack will say as he newscasts the ride to school to a fictional audience. “Don’t forget to subscribe,” his sister Ella, 6, will often interject -- again, to no one in particular. When I was their age, I’d pretend to be a soldier or a baseball player. Today, kids apparently aspire to be vloggers. It’s not enough for them to watch their favorite shows. They want to broadcast their lives, banter with commenters and keep their make-believe view counts high.
(tags: youtube kids wtf video broadcasting)
Cops Use Stingray To Almost Track Down Suspected Fast Food Thief
Law enforcement spokespeople will often point to the handful of homicide or kidnapping investigations successfully closed with the assistance of cell site simulators, but they'll gloss over the hundreds of mundane deployments performed by officers who will use anything that makes their job easier -- even if it's a tool that's Constitutionally dubious. Don't forget, when a cell site simulator is deployed, it gathers cell phone info from everyone in the surrounding area, including those whose chicken wings have been lawfully purchased. And all of this data goes... somewhere and is held onto for as long as the agency feels like it, because most agencies don't seem to have Stingray data retention policies in place until after they've been FOIA'ed/questioned by curious legislators. Regular policework -- which seemed to function just fine without cell tracking devices -- now apparently can't be done without thousands of dollars of military equipment. And it's not just about the chicken wing thieves law enforcement can't locate. It's about the murder suspects who are caught but who walk away when the surveillance device wipes its feet on the Fourth Amendment as it serves up questionable, post-facto search warrants and pen register orders.
(tags: stingrays mobile surveillance imsi-catchers data-retention privacy chicken-wings fast-food)
A fast alternative to the modulo reduction
(x * N) div 2^32 is an equally fair map reduction, but faster on modern 64-bit CPUs
(tags: fairness modulo arithmetic algorithms fair-mapping reduce daniel-lemire)
There are liars and then there’s Boris Johnson and Michael Gove
Post-brexit post-mortem from Nicholas Cohen in the grauniad:
The Vote Leave campaign followed the tactics of the sleazy columnist to the letter. First, it came out with the big, bold solution: leave. Then it dismissed all who raised well-founded worries with “the country is sick of experts”. Then, like Johnson the journalist, it lied.
(tags: eu politics uk brexit boris-johnson michael-gove)
-
The prime minister evidently thought that the whole debate could be cleanly started and finished in a matter of months. His Eton contemporary Boris Johnson – and, really, can you believe that the political story of the last four months has effectively been a catastrophic contest between two people who went to the same exclusive school? – opportunistically embraced the cause of Brexit in much the same spirit. What they had not figured out was that a diffuse, scattershot popular anger had not yet decisively found a powerful enough outlet, but that the staging of a referendum and the cohering of the leave cause would deliver exactly that. Ukip were held back by both the first-past-the-post electoral system, and the polarising qualities of Farage, but the coalition for Brexit effectively neutralised both. And so it came to pass: the cause of leaving the EU, for so long the preserve of cranks and chancers, attracted a share of the popular vote for which any modern political party would give its eye teeth.
In Wisconsin, a Backlash Against Using Data to Foretell Defendants’ Futures - The New York Times
More trial-by-algorithm horrors:
Company officials say the algorithm’s results are backed by research, but they are tight-lipped about its details. They do acknowledge that men and women receive different assessments, as do juveniles, but the factors considered and the weight given to each are kept secret. “The key to our product is the algorithms, and they’re proprietary,” said Jeffrey Harmon, Northpointe’s general manager. “We’ve created them, and we don’t release them because it’s certainly a core piece of our business. It’s not about looking at the algorithms. It’s about looking at the outcomes.” That secrecy is at the heart of Mr. Loomis’s lawsuit. His lawyer, Michael D. Rosenberg, who declined to be interviewed because of the pending appeal, argued that Mr. Loomis should be able to review the algorithm and make arguments about its validity as part of his defense. He also challenges the use of different scales for each sex. The Compas system, Mr. Rosenberg wrote in his brief, “is full of holes and violates the requirement that a sentence be individualized.”
(tags: ethics compas sentencing wisconsin northpointe law trial-by-algorithm algorithms)
-
high bike tire pressures are not faster, counterintuitively. I never knew! (via Tony Finch)