Tuning Spark Back Pressure by Simulation
Interesting, Spark uses a PID controller algorithm to manage backpressure:
Spark back pressure, which can be enabled by setting spark.streaming.backpressure.enabled=true, will dynamically resize batches so as to avoid queue build up. It is implemented using a Proportional Integral Derivative (PID) algorithm. This algorithm has some interesting properties, including the lack of guarantee of a stable fixed point. This can manifest itself not just in transient overshoot, but in a batch size oscillating around a (potentially optimal) constant throughput. The overshoot incurs latency; the undershoot costs throughput. Catastrophic overshoot leading to OOM is possible in degenerate circumstances (you need to choose the parameters quite deviously to cause this to happen). Having witnessed undershoot and slow recovery in production streaming jobs, I decided to investigate further by testing the algorithm with a simulator.
(tags: backpressure streaming queueing pid-controllers algorithms congestion-control)
New – EC2 Auto Scaling Groups With Multiple Instance Types & Purchase Options | AWS News Blog
Basically getting EC2 Fleet’s featureset into ASGs, good news
(tags: ec2 fleet asg ops architecture cost-control)
SpamAssassin is back [LWN.net]
The SpamAssassin 3.4.2 release was the first from that project in well over three years. At the 2018 Open Source Summit Europe, Giovanni Bechis talked about that release and those that will be coming in the near future. It would seem that, after an extended period of quiet, the SpamAssassin project is back and has rededicated itself to the task of keeping junk out of our inboxes.
This is good to see! Also, newsy thread: https://news.ycombinator.com/item?id=18458212(tags: spamassassin open-source oss anti-spam)
Google ‘betrays patient trust’ with DeepMind Health move | Technology | The Guardian
Now that Streams is a Google product itself, that promise appears to have been broken, says privacy researcher Julia Powles: “Making this about semantics is a sleight of hand. DeepMind said it would never connect Streams with Google. The whole Streams app is now a Google product. That is an atrocious breach of trust, for an already beleaguered product.” A DeepMind spokesperson emphasised that the core of the promise remains intact: “All patient data remains under our partners’ strict control, and all decisions about its use lie with them. This data remains subject to strict audit and access controls and its processing remains subject to both our contracts and data protection legislation. The move to Google does not affect this.”
(tags: google deepmind health nhs data-protection privacy healthcare)
Category: Uncategorized
Jeff Bezos is wrong, tech workers are not bullies
I decided to leave my job as a staff engineer at Google because of Project Maven, and because I believe that the artificial intelligence ethical guidelines they published afterwards were not strict enough: they allowed surveillance within “internationally accepted norms”. I am now joining forces with current and former Google employees who also opposed Maven and the Dragonfly search engine. We do not wish to be complicit in human rights violations and we believe that workers, and the public, deserve a voice. We support employees at Amazon, Microsoft, Salesforce, McKinsey and Deloitte who have similarly stood up to their employers. We also have a right to not contribute to killing. Most workers at Google or Amazon did not join those companies to work on military applications. Both companies are international employers with engineering offices across the world, and many of their workers are neither US citizens nor residents. I worked as an engineer in Google’s European headquarters in Dublin, Ireland. To me, the US military is not our military (as Google Cloud chief executive Diane Greene referred to it in a blog post), nor is it a force we should automatically support as a matter of patriotism. As an engineer, I believe it is my responsibility to speak up for human rights and accountable decision making. As an industry, we in technology cannot compromise our principles or allow ourselves to be bullied by billionaires who stand to be enriched by our silence.
(tags: project-maven dragonfly google amazon surveillance us-politics politics ai silicon-valley ethics work life)
HTTP-over-QUIC to be renamed HTTP/3
Decent newsy comment thread about HTTP/3, QUIC, and how the modern internet treats IP protocols
(tags: ip protocols http http3 quic networking internet newsy)
Brits getting into the online-disinfo game
Disinformation and deception have been a part of warfare for thousands of years, but across the world, something new was starting to happen. Information has long been used to support combat operations, but now combat was seen to taking place primarily, sometimes exclusively, through it. From being a tool of warfare, each military began to realise that the struggle with, over and through information was what war itself actually was about. And it wasn’t confined to Russia, China or anyone else. A global informational struggle has broken out. Dozens of countries are already doing it. And these are just the campaigns that we know about.
(tags: disinfo fake-news uk british-army military memes infowar)
Some notes on running new software in production
This is really good — how to approach new infrastructure/software dependencies in production with reliability and uptime in mind. (via Tony Finch)
(tags: reliability uptime slas kubernetes envoy outages runbooks ops)
Specification gaming examples in AI
A good list of examples where machine learning systems “figure out” how to cheat their fitness function, e.g.:
‘Creatures bred for speed grow really tall and generate high velocities by falling over’
(tags: ai funny humor spreadsheets machine-learning ml fitness-functions)
What if the Placebo Effect Isn’t a Trick? – The New York Times
It is not possible to assay levels of COMT directly in a living brain, but there is a snippet of the genome called rs4680 that governs the production of the enzyme, and that varies from one person to another: One variant predicts low levels of COMT, while another predicts high levels. When Hall analyzed the I.B.S. patients’ DNA, she found a distinct trend. Those with the high-COMT variant had the weakest placebo responses, and those with the opposite variant had the strongest. These effects were compounded by the amount of interaction each patient got: For instance, low-COMT, high-interaction patients fared best of all, but the low-COMT subjects who were placed in the no-treatment group did worse than the other genotypes in that group. They were, in other words, more sensitive to the impact of the relationship with the healer. The discovery of this genetic correlation to placebo response set Hall off on a continuing effort to identify the biochemical ensemble she calls the placebome — the term reflecting her belief that it will one day take its place among the other important “-omes” of medical science, from the genome to the microbiome. The rs4680 gene snippet is one of a group that governs the production of COMT, and COMT is one of a number of enzymes that determine levels of catecholamines, a group of brain chemicals that includes dopamine and epinephrine. (Low COMT tends to mean higher levels of dopamine, and vice versa.) Hall points out that the catecholamines are associated with stress, as well as with reward and good feeling, which bolsters the possibility that the placebome plays an important role in illness and health, especially in the chronic, stress-related conditions that are most susceptible to placebo effects.
(tags: placebo comt health healthcare medicine enzymes brain)
Deadlines, lies and videotape: The tale of a gRPC bug
HostedGraphite decided to use gRPC as an internal inter-service protocol and ran into a basic protocol bug — it does not default to using an application-level keepalive on the TCP channel so can block indefinitely if sending-side buffers fill up. Always use application-level keepalives and don’t trust TCP
(tags: tcp protocols keepalive grpc rpc architecture networking)
Sci-Fi Writer Greg Egan and 4chan anon Math Whiz Advance Permutation Problem | Quanta Magazine
On September 16, 2011, an anime fan posted a math question to the online bulletin board 4chan about the cult classic television series ‘The Melancholy of Haruhi Suzumiya’. Season one of the show, which involves time travel, had originally aired in non-chronological order, and a re-broadcast and a DVD version had each further rearranged the episodes. Fans were arguing online about the best order to watch the episodes, and the 4chan poster wondered: If viewers wanted to see the series in every possible order, what is the shortest list of episodes they’d have to watch? In less than an hour, an anonymous person offered an answer — not a complete solution, but a lower bound on the number of episodes required. The argument, which covered series with any number of episodes, showed that for the 14-episode first season of Haruhi, viewers would have to watch at least 93,884,313,611 episodes to see all possible orderings. “Please look over [the proof] for any loopholes I might have missed,” the anonymous poster wrote. The proof slipped under the radar of the mathematics community for seven years — apparently only one professional mathematician spotted it at the time, and he didn’t check it carefully. But in a plot twist last month, the Australian science fiction novelist Greg Egan proved a new upper bound on the number of episodes required. Egan’s discovery renewed interest in the problem and drew attention to the lower bound posted anonymously in 2011. Both proofs are now being hailed as significant advances on a puzzle mathematicians have been studying for at least 25 years.
(tags: mathematics internet math greg-egan anime bizarre 4chan superpermutation permutation proofs)
How do you populate your development databases?
Lots of comments pro/anti copying from production
(tags: database data testing system-tests dev)
-
To sum it up: A historically unreliable narrator who works for a conspiracy website tweets out a video in order to show alleged bad behavior on the part of a journalist. The clip goes viral. The White House picks up and disseminates that video and uses it as proof to ban the journalist from reporting at the White House. Outraged journalists decry the White House’s use of a video taken from a historically unreliable narrator. Then, users attempt to debunk the video as “actual fake news.” Others, unclear if the video is fake, urge caution, suggesting the media may be jumping the gun. An argument breaks out over the intricate technical details of doctoring a clip. The entire ordeal is a near perfect example of a scenario disinformation experts have predicted and warned of, where the very threat of video manipulation can lead to a blurring of reality. “These technological underpinnings [of AI and photoshop, and editing programs lead] to the increasing erosion of trust,” computational propaganda researcher Renee DiResta told BuzzFeed News in early 2018. “It makes it possible to cast aspersions on whether videos — or advocacy for that matter — are real.”
(tags: twitter disinfo disinformation buzzfeed video reality fake-news photoshop)
-
A neat kragen hack:
Report wireless signal strength using audio synthesis. This quick kludge is useful for figuring out where the dead spots in your wireless network coverage are. They’re where the pitch goes high and maybe you start getting clicks from audio buffer underruns. Example audio output can be found at
Underground Utility Colour Codes Explained
Underground utility colour codes are used to differentiate and identify underground utilities to protect it from damage during excavation. There are different types of utilities and in order to tell them apart coloured lines, flags or sometimes both are used. They help mark the location and indicate the type of utility that is buried underground.
I had no idea! (via Tony Finch)(tags: utilities power underground excavation digging repair urban)
Meet the Irish master codebreaker you never heard of
In 1943, one of Nazi Germany’s most notorious communication codes was broken by a mild-mannered librarian and family man from Abbeyfeale in west Limerick, Richard Hayes. His day job was director of the National Library of Ireland, but during the Emergency, he secretly led a covert team of cryptanalysts, working feverishly on the infamous “Görtz Cipher” – a fiendish Nazi code that had stumped many of the greatest code-breaking minds at Bletchley Park, the centre of British wartime cryptography. Astonishingly, the feat was accomplished not in one of the huts at Bletchley Park, but in a now derelict building colloquially known as “The Red House” on Montpellier Hill in Dublin. If this wasn’t an amazing enough feat, Hayes also broke a complex microdot enciphering system which had baffled the American OSS, as well an enciphering system used by the dreaded Sicherheitsdienst (or SD), the intelligence section of the SS. The breaking of the latter code was instrumental in the Allied victory at the Battle of the Bulge. Few individuals can be said to have altered the course of the second World War on their own, but Hayes was one such individual – and he did it not with a gun, but with a detailed knowledge of the complex algorithms and mathematical permutations involved in the art of cryptography.
(via Colm)(tags: via:colmmacc crypto codes hermann-gortz wwii dublin ireland richard-hayes cryptanalysis)
Physical spam using cheap Bluetooth beacons spotted in the wild
Buzz Andersen on Twitter: “Crazy thing: my friends who run a high traffic coffee shop just found this thing under their bar. It turns out to be a super cheap Chinese Bluetooth LE beacon.… https://t.co/pecNBCR86u”
-
‘We recently migrated SaleMove infrastructure from self-managed Kubernetes clusters running on AWS to using Amazon Elastic Container Service for Kubernetes (EKS). There were many surprises along the way to getting our EKS setup ready for production. This post covers some of these gotchas (others may already be fixed or are not likely to be relevant for a larger crowd) and is meant to be used as a reference when thinking of running EKS in production.’
-
These pervasive exhortations to individual action — in corporate ads, school textbooks, and the campaigns of mainstream environmental groups, especially in the west — seem as natural as the air we breathe. But we could hardly be worse-served. While we busy ourselves greening our personal lives, fossil fuel corporations are rendering these efforts irrelevant. The breakdown of carbon emissions since 1988? A hundred companies alone are responsible for an astonishing 71%. You tinker with those pens or that panel; they go on torching the planet. The freedom of these corporations to pollute – and the fixation on a feeble lifestyle response – is no accident. It is the result of an ideological war, waged over the last 40 years, against the possibility of collective action. Devastatingly successful, it is not too late to reverse it. The political project of neoliberalism, brought to ascendence by Thatcher and Reagan, has pursued two principal objectives. The first has been to dismantle any barriers to the exercise of unaccountable private power. The second had been to erect them to the exercise of any democratic public will. […] At the very moment when climate change demands an unprecedented collective public response, neoliberal ideology stands in the way. Which is why, if we want to bring down emissions fast, we will need to overcome all of its free-market mantras.
(tags: politics environment neoliberalism future climate-change green)
Block Advertising on your Network with Pi-hole and Raspberry Pi
A good walkthrough of the Pi-Hole network-wide adblocker install and operation
October 21 post-incident analysis | The GitHub Blog
A network outage caused a split-brain scenario, and their failover system allowed writes to occur in both regional databases. Once the outage was repaired it was impossible to reconcile writes in an automated fashion as a result. Embarrassingly, this exact scenario was called out in their previous blog post about their Raft-based failover system at https://githubengineering.com/mysql-high-availability-at-github/ — “In a data center isolation scenario, and assuming a master is in the isolated DC, apps in that DC are still able to write to the master. This may result in state inconsistency once network is brought back up. We are working to mitigate this split-brain by implementing a reliable STONITH from within the very isolated DC. As before, some time will pass before bringing down the master, and there could be a short period of split-brain. The operational cost of avoiding split-brains altogether is very high.” Failover is hard.
(tags: github fail outages failover replication consensus ops)
-
Analyses open source code repos on github, running static code analysis to find “zero-days and other critical bugs”.
(tags: security code-review static-code-analysis coding lgtm github)
cool twitter image compression hack
D?v?d ?uc??n?n on Twitter:
Assuming this all works out, the image in this tweet is also a valid ZIP archive, containing a multipart RAR archive, containing the complete works of Shakespeare. This technique also survives twitter’s thumbnailer :P
(via JK)(tags: via:johnke cool compression image jpeg twitter)
A Dark Consensus About Screens and Kids Begins to Emerge in Silicon Valley – The New York Times
John Lilly, a Silicon Valley-based venture capitalist with Greylock Partners and the former C.E.O. of Mozilla, said he tries to help his 13-year-old son understand that he is being manipulated by those who built the technology. “I try to tell him somebody wrote code to make you feel this way — I’m trying to help him understand how things are made, the values that are going into things and what people are doing to create that feeling,” Mr. Lilly said. “And he’s like, ‘I just want to spend my 20 bucks to get my Fortnite skins.’”
(tags: kids technology education parenting screentime apps tech phones)
Your Kid’s Apps Are Crammed With Ads – The New York Times
In apps marketed for children 5 and under in the Google Play store, there were pop-up ads with disturbing imagery. There were ads that no child could reasonably be expected to close out of, and which, when triggered, would send a player into more ads. Dancing treasure chests would give young players points for watching video ads, potentially endlessly. The vast majority of ads were not marked at all. Characters in children’s games gently pressured the kids to make purchases, a practice known as host-selling, banned in children’s TV programs in 1974 by the Federal Trade Commission. At other times an onscreen character would cry if the child did not buy something. “The first word that comes to mind is furious,” said Dr. Radesky, an assistant professor of developmental behavioral pediatrics at the University of Michigan Medical School. “I’m a researcher. I want to stay objective. We started this study really just trying to look at distraction. My frustrated response is about all the surprising, potentially deceptive stuff we found.”
brb, installing Pi-Hole.(tags: children kids ads advertising apps android google ftc games iap)
Epic twitter thread from @colmmacc explaining why client certs and mutual-auth TLS are TERRIBAD
Ok. tweet thread time! Too long ago I promised to write a screed explaining how much I hated mutual-auth TLS and why. I got distracted, and I wasn’t happy with the writing, so here it is in tweet thread form instead! But basically: Client certs and Mutual-Auth TLS is TERRIBAD. When I say TERRIBAD, I mean that unless you’ve got the resources of a big security dept and folks who comb threat models for a living, using clients certs and mutual auth probably materially lessens your security. That’s NUTS!
(source: https://twitter.com/colmmacc/status/1057017343438540801 )(tags: terribad rants twitter threads tls ssl authentication mtls security)
-
0 authentication on some setup APIs, including ‘delete the current wifi network config’ — pretty major lack of security
(tags: auth security fail google google-home)
-
similar to the Google Home hackable APIs, it seems Chromecasts are easily hacked/scripted with no auth
“The first AI portrait in Christie’s” was mostly output from someone else’s open-source code
The print was created by Obvious, a trio of 25-year-old French students whose goal is to “explain and democratize” AI through art. Over the past year, they’ve made a series of portraits depicting members of the fictional Belamy family, amplifying their work through attention-grabbing press releases. But insiders say the code used to generate these prints is mostly the work of another artist and programmer: 19-year-old Robbie Barrat, a recent high school graduate who shared his algorithms online via an open-source license. The members of Obvious don’t deny that they borrowed substantially from Barrat’s code, but until recently, they didn’t publicize that fact either. This has created unease for some members of the AI art community, which is open and collaborative and taking its first steps into mainstream attention.[…] Jason Bailey, a digital art blogger who runs the site Artnome, says that what Obvious has done is far from unusual. “It’s almost weekly in digital art that someone takes some open code and tweaks it and sells it,” he tells The Verge. But the prominence of this auction and the fact that Obvious, not Barrat, has received the attendant prestige and attention does complicate the matter. “There’s a lot of stuff you can do that’s legal, but that makes you sort of a jerk,” adds Bailey. “If I was Robbie, I’d be pretty miffed, and Obvious said they owe him a great deal of credit.” Barrat says he holds no grudges at all and is mostly annoyed that the auction might give outsiders the wrong impression about AI art. “I’m more concerned about the fact that actual artists using AI are being deprived of the spotlight,” he says. “It’s a very bad first impression for the field to have.”
(tags: ai art graphics history open-source ownership copyright obvious robbie-barrat digital)
-
‘Tries to move K8s Pods from on-demand to spot instances’:
K8s Spot rescheduler is a tool that tries to reduce load on a set of Kubernetes nodes. It was designed with the purpose of moving Pods scheduled on AWS on-demand instances to AWS spot instances to allow the on-demand instances to be safely scaled down (By the Cluster Autoscaler). In reality the rescheduler can be used to remove load from any group of nodes onto a different group of nodes. They just need to be labelled appropriately. For example, it could also be used to allow controller nodes to take up slack while new nodes are being scaled up, and then rescheduling those pods when the new capacity becomes available, thus reducing the load on the controllers once again.
(tags: k8s kubernetes aws scaling spot-instances ops)
-
Air quality sensor board for Raspberry Pis, with a good quality self-calibrating NDIR CO2 sensor
(tags: co2 air quality monitoring metrics health home raspberry-pi hardware to-get)
Motorola and iFixit—A Match Made in Mobile
This is awesome.
Motorola is setting an example for major manufacturers to embrace a more open attitude towards repair. If you’re a Motorola customer, you can now either send in your broken device directly to Motorola for repair—or you can fix it yourself with the highest quality parts and tools, plus a free step-by-step guide, all included in our official Motorola OEM Fix Kits.
Excellent Twitter thread from colmmacc on how s2n avoids protocol-state errors
using a linearized set of state transitions, and Cryptol and SAW to perform verification of the TLS state machine
(tags: cryptol saw formal-verification twitter threads colmmacc security s2n)
Remapping the silly double-S key on a Macbook running 10.12 Sierra
the latest hacky workaround for this stupid Apple keyboard layout bullshit
Republicans Are Adopting the Proud Boys
Extreme-right factions of the GOP happily taking on a brownshirt paramilitary gang. This is scary shit
(tags: fascism america politics proud-boys paramilitary republicans gop)
LOLWUT: a piece of art inside a database command –
Redis now includes a reference to “Schotter”, by Georg Nees, one of the earliest pieces of computer art. Nice one antirez :)
-
great T-shirt — adding to the shopping list for the next Threadless trip
(tags: threadless to-get tee-shirts thorazos)
Every fantasy map / Boing Boing
Every fantasy map, ever
Lawsuit: “pivoting to video” was a disaster led by Facebook’s cooked viewing data / Boing Boing
The company decided to “obfuscate the fact that we screwed up the math” by quietly retiring the erroneous metrics and replacing them with corrected metrics under a new name. For instance, Average Duration of Video Viewed would be replaced with Average Watch Time. […] In August 2016, Facebook began reaching out privately to select, large advertisers, telling them that Facebook had “recently discovered a discrepancy” in the video ad average view metrics. Facebook pushed that message even as personnel internally emphasized that “we didn’t recently discover a discrepancy.” (emphasis added).
-
If Bitcoin were to cease trading tomorrow, 0.5% of the world’s electricity demand would simply disappear. This is roughly equivalent to the output of ten coal-fired power plants, emitting 50 million tonnes of CO2 per year – which would cover one year’s worth of the carbon emission cuts required to limit temperature rises this century to 2C. It is not a solution by itself, but it would be a good year’s work. Bitcoin is made from ashes, and if ashes were legal tender, humanity would burn everything in sight and call it progress.
(tags: environment bitcoin ecology future earth cryptocurrencies pow electricity climate-change)
-
Well this is terrifying.
Can civilisation prolong its life until the end of this century? “It depends on what we are prepared to do.” He fears it will be a long time before we take proportionate action to stop climatic calamity. “Standing in the way is capitalism. Can you imagine the global airline industry being dismantled when hundreds of new runways are being built right now all over the world? It’s almost as if we’re deliberately attempting to defy nature. We’re doing the reverse of what we should be doing, with everybody’s silent acquiescence, and nobody’s batting an eyelid.”
(tags: climate capitalism environment future scary mayer-hillman)
Querying OpenStreetMap Buildings with AWS Athena – door2door Engineering
Well this is damn cool:
AWS not only made OpenStreetMap planet data available on S3, but it also made it query-able with Athena. Pretty cool, no? Now, in theory, you can just construct an SQL query, send it to Athena, and then do whatever you want with the results. No more: Updating OSM planet data yourself; it gets updated on AWS whenever OSM publishes it, once a week. Transforming the data into a query-able format; Athena handles that for you. Query/request frequency limits (it’s still AWS though, so other limitations might apply ???? ????) At door2door, we had a pretty straight-forward use-case for this: we needed to get buildings in specific regions based only on where they were, and transform those buildings into GeoJSON that we can attach our data to, and visualize on the front-end on top of our base map.
(tags: athena osm buildings aws geodata mapping maps door2door cool hacks)
Opinion | When Your Boss Is an Algorithm – The New York Times
I have learned that drivers at ride-hailing companies may have the freedom and flexibility of gig economy work, but they are still at the mercy of a boss — an algorithmic boss. Data and algorithms are presented as objective, neutral, even benevolent: Algorithms gave us super-convenient food delivery services and personalized movie recommendations. But Uber and other ride-hailing apps have taken the way Silicon Valley uses algorithms and applied it to work, and that’s not always a good thing.
(tags: algorithms uber gig-economy work)
A Soyuz “ballistic re-entry” which subjected the crew to 21 g
At the time when the safety system initiated separation the spacecraft was already pointed downward toward Earth, which accelerated its descent significantly. Instead of the expected acceleration in such an emergency situation of 15 g (147 m/s²), the cosmonauts experienced up to 21.3 g (209 m/s²).[2] Despite very high overloading, the capsule’s parachutes opened properly and slowed the craft to a successful landing after a flight of only 21 minutes.
(tags: spaceflight soyuz accidents history cosmonauts)
How To Survive The Coming Century
New Scientist article from 2009 with a rather terrifying map of the 4-degrees-warmer Earth
(tags: earth new-scientist climate fear)
IPCC 1.5 degrees target requires massive carbon dioxide removal technology efforts
The grimmest prognosis in the draft report is in the details of the effort it would take to actually limit warming to 1.5°C. Countries won’t just have to give up fossil fuels and stop emitting greenhouse gases; they’ll have to pull carbon dioxide straight out of the air. “All pathways that limit global warming to 1.5°C with limited or no overshoot project the use of carbon dioxide removal (CDR),” according to the report. And not just a little, but a lot, upward of 1,000 gigatons of carbon dioxide removed from the atmosphere by the end of the century. This will require machines that scrub carbon dioxide out of the air as well as biofuels coupled with carbon capture and sequestration. These tactics have their own energy demands and environmental drawbacks, and we may not be able to deploy them in time. “CDR deployment of several hundreds of [gigatons of CO2] is subject to multiple feasibility and sustainability constraints,” according to the IPCC report.
(tags: cdr co2 greenhouse-gases climate-change technology ipcc un)
The 100 best restaurants, cafes and places to eat in Ireland
the Irish Times latest foodie list, via Aoife McElwain
(tags: irish-times food cafes eating dublin foodie restaurants)
A Controversial Virus Study Shows Flaws in How Science Is Done – The Atlantic
Absent clearer guidelines, the burden falls on the scientific enterprise to self-regulate—and it isn’t set up to do that well. Academia is intensely competitive, and “the drivers are about getting grants and publications, and not necessarily about being responsible citizens,” says Filippa Lentzos from King’s College London, who studies biological threats. This means that scientists often keep their work to themselves for fear of getting scooped by their peers. Their plans only become widely known once they’ve already been enacted, and the results are ready to be presented or published. This lack of transparency creates an environment where people can almost unilaterally make decisions that could affect the entire world. Take the horsepox study [the main topic of this article]. Evans was a member of a World Health Organization committee that oversees smallpox research, but he only told his colleagues about the experiment after it was completed. He sought approval from biosafety officers at his university, and had discussions with Canadian federal agencies, but it’s unclear if they had enough ethical expertise to fully appreciate the significance of the experiment. “It’s hard not to feel like he opted for agencies that would follow the letter of the law without necessarily understanding what they were approving,” says Kelly Hills, a bioethicist at Rogue Bioethics. She also sees a sense of impulsive recklessness in the interviews that Evans gave earlier this year. Science reported that he did the experiment “in part to end the debate about whether recreating a poxvirus was feasible.” And he told NPR that “someone had to bite the bullet and do this.” To Hills, that sounds like I did it because I could do it. “We don’t accept those arguments from anyone above age 6,” she says.
(tags: the-atlantic science news smallpox horsepox diseases danger risk academia papers publish-or-perish bioethics ethics biology genetics)
Amazon scraps secret AI recruiting tool that showed bias against women | Reuters
Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry. […] Amazon’s system taught itself that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter.’
nice demo of algorithmic bias right there. Worrying that there are plenty of other places carrying on with the concept though….(tags: algorithmic-bias amazon hiring resumes bias feminism machine-learning ml)
Wild camping spots – Google My Maps
A GMaps version of the list of wild camping spots around Ireland
When It’s Too Late to Stop Fascism, According to Stefan Zweig | The New Yorker
The destruction of a symbolic edifice—a blaze that caused no loss of life—became the pretext for the government to begin terrorizing its own civilian population. That fateful conflagration took place less than thirty days after Hitler became Chancellor. The excruciating power of Zweig’s memoir lies in the pain of looking back and seeing that there was a small window in which it was possible to act, and then discovering how suddenly and irrevocably that window can be slammed shut.
(tags: fascism europe stefan-zweig politics history germany nazis)
What to Look for in Java 8 Code
Great advice from Trisha Gee on J8 features and how to code-review code which uses them
(tags: java8 java code-review coding)
Wild Camping Spots map Ireland & Northern Ireland
looks like a decent map (even if they can’t use GMaps)
(tags: wild-camping camping campsites ireland tourism)
archive.org C64 software library
Pretty amazing that these can be credibly emulated in a browser now!
(tags: c=64 history commodore vice web internet-archive gaming games)
Kubernetes: The Surprisingly Affordable Platform for Personal Projects
At the beginning of the year I spent several months deep diving on Kubernetes for a project at work. As an all-inclusive, batteries-included technology for infrastructure management, Kubernetes solves many of the problems you’re bound to run into at scale. However popular wisdom would suggest that Kubernetes is an overly complex piece of technology only really suitable for very large clusters of machines; that it carries a large operational burden and that therefore using it for anything less than dozens of machines is overkill. I think that’s probably wrong. Kubernetes makes sense for small projects and you can have your own Kubernetes cluster today for as little as $5 a month.
(via Tony Finch)(tags: via:fanf deployment howto kubernetes ops projects hacks clustering)
This is how cyber attackers stole £2.26m from Tesco Bank customers | ZDNet
What a shitshow.
Poor design of Tesco Bank debit cards played a significant role in creating security vulnerabilities that led to thousands of customers having their accounts emptied. One of these involved the PAN numbers — the 16-digit card number sequence used to identify all debit cards. Tesco Bank inadvertently issued debit cards with sequential PAN numbers. This increased the likelihood that the attackers would find the next PAN number in the sequence. It took 21 hours after the attack began before Tesco Bank’s Fraud Strategy Team was informed about the incident. Only after what the FCA describes as a “series of errors” — including Tesco Bank’s Financial Crime Operations Team sending an email to the wrong address, instead of making a phone call as procedure requires — was the fraud team made aware of the attack. In all that time, nothing had been done to stop the attacks, with fraudulent transactions continuing to siphon money from accounts as the bank received more and more calls from worried customers.
(tags: tesco fail tesco-bank banking pan-numbers debit-cards security fraud uk)
Running high-scale web applications on Amazon EC2 Spot Instances
AppNext’s setup looks like quite good practice for a CPU-bound fleet
(tags: appnext spot-instances ec2 scalability aws ops architecture)
Amazon S3 Announces Selective Cross-Region Replication Based on Object Tags
Amazon S3 Cross-Region Replication (CRR) now supports object filtering based on S3 object tags. This allows you to identify individual objects using S3 object tags for automatic replication across AWS Regions for compliance and/or data protection. CRR is an Amazon S3 feature that automatically replicates every object uploaded to an S3 bucket to a destination bucket in a different AWS Region that you choose. S3 object tags are key-value pairs applied to S3 objects that allow you to better organize, secure, and manage your data stored in S3. By using S3 object tags to determine which objects to replicate using CRR, you now have fine grained control to selectively replicate your storage to another AWS Region to backup critical data for compliance and disaster recovery.
Estonia sues Gemalto for 152 mln euros over ID card flaws
Estonia’s Police and Border Guard Board (PPA) said in a statement Gemalto had created private key codes for individual cards, leaving the government IDs vulnerable to external cyber attack, rather than embedding it on the card’s chip as promised. “It turned out that our partner had violated this principle for years, and we see this as a very serious breach of contract,” PPA’s deputy director-general Krista Aas said in the statement.
If true, this is a big problem…(tags: gemalto fail security smartcards estonia chip-cards)
Defcon Voting Village report: Bug in one system could “flip Electoral College” | Ars Technica
ES&S strike again:
Today, six prominent information-security experts who took part in DEF CON’s Voting Village in Las Vegas last month issued a report on vulnerabilities they had discovered in voting equipment and related computer systems. One vulnerability they discovered—in a high-speed vote-tabulating system used to count votes for entire counties in 23 states—could allow an attacker to remotely hijack the system over a network and alter the vote count, changing results for large blocks of voters. “Hacking just one of these machines could enable an attacker to flip the Electoral College and determine the outcome of a presidential election,” the authors of the report warned. The machine in question, the ES&S M650, is used for counting both regular and absentee ballots. The device from Election Systems & Software of Omaha, Nebraska, is essentially a networked high-speed scanner like those used for scanning standardized-test sheets, usually run on a network at the county clerk’s office. Based on the QNX 4.2 operating system—a real-time operating system developed and marketed by BlackBerry, currently up to version 7.0—the M650 uses Iomega Zip drives to move election data to and from a Windows-based management system. It also stores results on a 128-megabyte SanDisk Flash storage device directly mounted on the system board. The results of tabulation are output as printed reports on an attached pin-feed printer. The report authors—Matt Blaze of the University of Pennsylvania, Jake Braun of the University of Chicago, David Jefferson of the Verified Voting Foundation, Harri Hursti and Margaret MacAlpine of Nordic Innovation Labs, and DEF CON founder Jeff Moss—documented dozens of other severe vulnerabilities found in voting systems. They found that four major areas of “grave and undeniable” concern need to be addressed urgently. One of the most critical is the lack of any sort of supply-chain security for voting machines—there is no way to test the machines to see if they are trustworthy or if their components have been modified.
(tags: fail security evoting vote-tabulation us-politics voting-machines)
How Triplebyte solved its office Wi-Fi problems
This is good general wi-fi infrastructure advice for home use too
randomised prefixes in S3 are no longer necessary to improve performance
This S3 request rate performance increase removes any previous guidance to randomize object prefixes to achieve faster performance. That means you can now use logical or sequential naming patterns in S3 object naming without any performance implications.
Having said that, it sounds like they may still help to a degree anyway.(tags: s3 coding architecture aws tips)
Do not fall into Oracle’s Java 11 trap
The key part of the terms is as follows: You may not: use the Programs for any data processing or any commercial, production, or internal business purposes other than developing, testing, prototyping, and demonstrating your Application; The trap is as follows: Download Oracle JDK (because that is what you’ve always done, and it is what the web-search tells you); Use it in production (because you didn’t realise the license changed); Get a nasty phone call from Oracle’s license enforcement teams demanding lots of money In other words, Oracle can rely on inertia from Java developers to cause them to download the wrong (commercial) release of Java. Unless you read the text/warnings/legalese very carefully you might not even realise Oracle JDK is now commercial, and that you are therefore liable to pay Oracle for Java.
(tags: java licensing openjdk open-source oracle software jdk jre)
-
lots of nice graphs and dataviz around Dublin Bikes usage
Common Cyborg | Jillian Weise | Granta
Fantastic essay:
When I tell people I am a cyborg, they often ask if I have read Donna Haraway’s ‘A Cyborg Manifesto’. Of course I have read it. And I disagree with it. The manifesto, published in 1985, promised a cyberfeminist resistance. The resistance would be networked and coded by women and for women to change the course of history and derange sexism beyond recognition. Technology would un-gender us. Instead, it has been so effective at erasing disabled women that even now, in conversation with many feminists, I am no longer surprised that disability does not figure into their notions of bodies and embodiment. Haraway’s manifesto lays claim to cyborgs (‘we are all cyborgs’) and defines the cyborg unilaterally through metaphor. To Haraway, the cyborg is a matter of fiction, a struggle over life and death, a modern war orgy, a map, a condensed image, a creature without gender. The manifesto coopts cyborg identity while eliminating reference to disabled people on which the notion of the cyborg is premised. Disabled people who use tech to live are cyborgs. Our lives are not metaphors.
(Via Tony Finch)(tags: via:dotat cyborg technology feminism essay disability tech jillian-weise granta)
25 Years of WIRED Predictions: Why the Future Never Arrives
These early views of the sharing economy were accurate depictions of the moment, but poor visions of the future. Within a few short years, many of those Uber drivers would be stuck paying off their cars in sub-minimum-wage jobs with no benefits. What began as an earnest insight about bits and atoms quickly turned into an arbitrage opportunity for venture capitalists eager to undercut large, lucrative markets by skirting regulations. To meet the growth and monetization demands of investors, yesterday’s sharing economy became today’s gig economy.
(tags: advertising future technology futurism predictions wired web2.0 history 1990s 2000s)
Customer Service Data Vulnerability, Intercom [Fixed]
good demo of the danger of unrestricted API keys
(tags: api-keys security intercom fail dropmobility web apis)
Peter Flynn caused the first 404
Now that’s a great bit of web trivia :) “[UCC’s] first webmaster was the first person to ever break a link on the web, when he moved the location of a webpage on UCC’s servers without telling TimBL. Such a change resulted in the need to error-handle such an occurrence, and the 404 was born”
(tags: 404 history http web peter-flynn ucc irish-web trivia)
Credit reference agency Equifax fined for security breach
The ICO fines Equifax £500K, the maximum amount possible under the old Data Protection Act (via Privacy Kit)
(tags: via:privacy-kit ico equifax privacy data-protection uk penalties law)
Muting some magic keywords fixes the Twitter timeline
Apparently, turning off some of the shittier recent features:
Muting suggest_recycled_tweet_inline and suggest_activity_tweet actually has fixed my timeline. It’s all chronological and there are barely any “x and y liked” tweets.
(tags: twitter feature-creep muting hacks)
-
TIL! In other words spelling identifiers-like-this, Lisp style
(tags: kebab-case case lisp identifiers coding terminology)
Cindy Sridharan on Twitter: NanoLog by Ousterhout et al.
– just formatting a log typically takes on the order of 1µs! – nanolog achieves high throughput by shifting work out of runtime hot path into compilation + post-execution phases
Basically records symbolic form of logs, and uses a post-processor after the fact to generate readable text.(tags: logging ops coding performance)
Surprisingly Little Evidence for the Accepted Wisdom About Teeth – The New York Times
Turns out there is little evidence for many dental practices:
A systematic review in 2011 concluded that, in adults, toothbrushing with flossing versus toothbrushing alone most likely reduced gingivitis, or inflammation of the gums. But there was really weak evidence that it reduced plaque in the short term. There was no evidence that it reduced cavities. That’s pretty much what we learned recently.
(tags: teeth dentistry dental health medicine statistics science)
Google spent $60 million on building Content ID
That’s how much it costs to build a not-particularly-accurate UGC copyright filter:
Google’s new report takes aim at this claim. It asserts that Content ID is a highly effective solution, with over 98 percent of copyright management on YouTube happening through Content ID, and just 2 percent coming from humans filing copyright removal notices. Google also says the music industry opts to monetize more than 95 percent of its copyright claims, meaning they leave the videos up on the service. It claims a whopping half of the music industry’s YouTube revenue comes from fan content — covers, remixes, dance versions, etc. — claimed via Content ID. The report also puts a hard figure on how much Google has spent so far on Content ID: $60 million.
(tags: filtering copyright eu article-13 copyfight content-id google web ugc)
-
Very interesting! This paper and the one at https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0126438 discuss the increasing evidence that some kinds of IBS may be caused by post-infection autoimmune activity triggered by a gastroenteritis infection — this matches the thing which put me on a restricted diet a few years ago.
(tags: digestion ibs medicine health diet fodmap gastroenteritis papers)
-
Five or six years ago, around the time most people seemed to be spending almost all of their time on the internet, I began to notice a particular kind of online phenomenon, one that I did not have a terminology for. I started to call these moments “artefacts”, borrowing a term from photography that describes the machine-created distortions and ghosts that corrupt digital imagery. “An unintended alteration in data” is one definition, but this new kind of “artefact” was expanding beyond sporadic instances and becoming a persistent sub-theme in discourse at large. The result was a type of semiotic collapse, one that first found its fullest expression in the absurdity of the 2016 presidential campaign, when news stories fabricated in Macedonia found a wider reach than The Washington Post. Countermeasures to interference in the coming 2018 congressional election look ineffectual, perhaps deliberately so.
The British Airways Breach: How Magecart Claimed 380,000 Victims
very detailed and pretty fiendish
(tags: analysis security attacks magecart british-airways web javascript)
UIDAI’s Aadhaar Software Hacked, ID Database Compromised, Experts Confirm
The authenticity of the data stored in India’s controversial Aadhaar identity database, which contains the biometrics and personal information of over 1 billion Indians, has been compromised by a software patch that disables critical security features of the software used to enrol new Aadhaar users, a three month-long investigation by HuffPost India reveals. The patch—freely available for as little as Rs 2,500 (around $35)— allows unauthorised persons, based anywhere in the world, to generate Aadhaar numbers at will, and is still in widespread use. This has significant implications for national security at a time when the Indian government has sought to make Aadhaar numbers the gold standard for citizen identification, and mandatory for everything from using a mobile phone to accessing a bank account.
(tags: security aadhaar identity india privacy databases data-privacy)
Troy Hunt: The Effectiveness of Publicly Shaming Bad Security
Now I don’t know how much of this change was due to my public shaming of their security posture, maybe they were going to get their act together afterward anyway. Who knows. However, what I do know for sure is that I got this DM from someone not long after that post got media attention (reproduced with their permission): Hi Troy, I just want to say thanks for your blog post on the Natwest HTTPS issue you found that the BBC picked up on. I head up the SEO team at a Media agency for a different bank and was hitting my head against a wall trying to communicate this exact thing to them after they too had a non secure public site separate from their online banking. The quote the BBC must have asked from them prompted the change to happen overnight, something their WebDev team assured me would cost hundreds of thousands of pounds and at least a year to implement! I was hitting my head against the desk for 6 months before that so a virtual handshake of thanks from my behalf! Thanks!
(tags: business internet security social-media shame troy-hunt bad-press spin shaming)
Software as Craft: software delivery and open source in a Cloud & Enterprise world
Niall Murphy sends this on:
Microsoft is very pleased to welcome Maggie Pint and Dr. Nicole Forsgren to our new campus, to talk about open source and the deep connections between how software is written, and how successful it is. For those of you who are not aware, Maggie Pint is a software engineering lead in Azure’s Production Infrastructure Engineering (PIE) organization. Maggie’s team works on improving the engineering systems experience for Microsoft’s web-focused developers. She co-ordinates open source and inner source education and execution through Azure PIE. Outside of her day job, Maggie maintains the popular Moment.js JavaScript library, and is the JS Foundation’s delegate to TC39, the standards committee for JavaScript. She is passionate about dogs, coffee, the JavaScript language, and helping others live open source values in their day-to-day work. Dr. Nicole Forsgren is the co-founder and Chief Scientist of the DevOps Research and Assessment joint venture with Jez Humble and Gene Kim, also well-known leaders in the DevOps community. She is best known as a co-author of Accelerate: The Science of Lean Software and DevOps and lead investigator for the largest-scale DevOps studies undertaken to date. She is also member of the ACM Queue editorial board, a research affiliate for a number of universities, and earned her PhD in Management Information Systems from the University of Arizona. This event comprises two public technical talks, with an intended audience of a few hundred software and systems professionals, including technical managers and SREs.
(tags: software coding open-source microsoft maggie-pint nicole-forsgren azure)
gRPC On HTTP/2: Engineering A Robust, High Performance Protocol
Decent writeup on how gRPC uses HTTP/2 efficiently
‘The Internet of Garbage’ by Sarah Jeong
Sarah Jeong’s 2015 book is now free: ‘I think The Internet of Garbage still provides a useful framework to begin to talk about our new dystopia, and it continues to be surprisingly relevant in many ways. But I wrote the book with a tone of optimism I did not feel even at the time, hoping that by reaching the well-meaning policy teams across Silicon Valley, I might be able to spark change for the better. Not only did that change never quite solidify, but the coordinated, orchestrated harassment campaigns of Gamergate that I very briefly touch on in Chapter Two have since overtaken our national political and cultural conversations. These twisted knots of lies, deflection, and rage are not just some weird and terrible online garbage. They shadow executive orders, court rulings, even the newly appointed judiciary. They will haunt us for years to come. We are all victims of fraud in the marketplace of ideas. I hope that in the very near future, I will be putting out a second edition of The Internet of Garbage. In that future edition, I hope to grapple with advertising incentives, engagement traps, international propaganda wars, the American crisis in free speech coinciding with the rise of platform power, and search engine optimization as the new paradigm of speech. In the meantime, I am putting out The Internet of Garbage 1.5 as an interim edition. I wish it were more helpful in our present reality. But as imperfect a tool as it is, I figure we all need as much help as we can get. ‘
(tags: dystopia fake-news internet spam harrassment abuse twitter gamergate politics books free to-read)
-
cOAlition S signals the commitment to implement, by 1 January 2020, the necessary measures to fulfil its main principle: “By 2020 scientific publications that result from research funded by public grants provided by participating national and European research councils and funding bodies, must be published in compliant Open Access Journals or on compliant Open Access Platforms.” The 11 national research funding organisations that form cOAlition S have agreed to implement the 10 principles of Plan S in a coordinated way, together with the European Commission including the ERC. Other research funders from across the world, both public and private, are invited to join cOAlition S.
I am extremely happy to see SFI on this list! (Via Cathal Garvey)(tags: sfi ireland funding science open-access open papers journals via:cathalgarvey)