Category: Uncategorized

Troubleshooting Problems With Native (Off-Heap) Memory in Java Applications

quite good advice on dealing with memory problems caused by off-heap DirectByteBuffers in java 8. 'Furthermore, the JDK caches one or more DirectByteBuffers per thread, and by default, there is no limit on the number or size of these buffers. As a result, if a Java app creates many threads that perform I/O using HeapByteBuffers, and/or these buffers are big, the JVM process may end up using a lot of additional native memory that looks like a leak'. 'java.lang.OutOfMemoryError: Direct buffer memory' is the indicative error message.

(tags: java off-heap buffers memory memory-leaks gc jdk ops)

Syndrome K

The Fake Disease That Saved Lives:

On October 1943, the Nazis raided a Jewish ghetto in Rome, and many Jews fled to Fatebenefratelli, where Borromeo admitted them as “patients”. The refugees were given a new [fake] fatal disease—Syndrome K—in order to identify them from the actual patients.

(tags: wwii syndrome-k fakes diseases rome giovanni-borromeo hospitals jews resistance)

cel-go

The Go implementation of the Common Expression Language (CEL). CEL is a non-Turing complete language designed to be portable and fast. It is well suited to embedded applications expression evaluation with familiar syntax and features, protocol buffer support, and not needing the sandboxing needed for a runtime like JavaScript or Lua.

(tags: scripting golang go cel languages coding configuration config embedded)

XXH3

'a cross-over inspired by many other great hash algorithms, which proves substantially faster than existing variants of xxHash, across basically all dimensions.'

(tags: hashing algorithms xxhash xxh3 checksums performance)

2-hour-long meetings can impair cognitive functioning

'Study shoes three people in a conference room over 2 hours can result in a Co2 level that can impair cognitive functioning. Ie. If you’re making decisions at the end of the meeting, you’re mentally less qualified to do so.' Well, I'd say that fatigue could also result in this, but it's interesting to see how unhealthy the typical office environment can be. (via Jeff Dean)

(tags: via:jeffdean meetings work offices brain co2 cognition)

The Oxygen of Amplification

Offering extremely candid comments from mainstream journalists, this report provides a snapshot of an industry [news media] caught between the pressure to deliver page views, the impulse to cover manipulators and “trolls,” and the disgust (expressed in interviewees’ own words) of accidentally propagating extremist ideology. After reviewing common methods of “information laundering” of radical and racist messages through the press, Phillips uses journalists’ own words to propose a set of editorial “better practices” intended to reduce manipulation and harm. As social and digital media are leveraged to reconfigure the information landscape, Phillips argues that this new domain requires journalists to take what they know about abuses of power and media manipulation in traditional information ecosystems; and apply and adapt that knowledge to networked actors, such as white nationalist networks online.

(tags: media news harassment nazis fascism overton-window journalism racism press)

Ash Sarkar on how to counter the new right

'a) Acknowledge that the fascist threat has changed. It's political operations are far more nebulous and diffuse; it works in political institutions and dark corners of the internet; it will adopt and distort liberal tropes and talking points. b) Deal with the fact that traditional forms of policing will be of little effectiveness in countering it. Those with the most power to inhibit the dissemination of far-right and racist ideology are the digital platforms they rely on: reddit, Twitch, YouTube, Twitter, Facebook. c) Transform current affairs media. For too long, producers and editors have taken the alt-right at their word, and framed issues as free speech/limits of offensive humour. That must change. Unless you're willing to do rigorous research first, don't commission the debate. d) Overhaul the teaching of PSHE & Citizenship in education to prepare young people for the desensitising and extreme content they will see online. Create space for healthy debate and discussion in respectful environments. Don't let groomers take advantage of their curiousity. e) Get a very big bin, and put Melanie Phillips, Rod Liddle, and Douglas Murray in it. Then fire the bin into outer space.'

(tags: alt-right fascism media politics internet social-media twitter reddit ash-sarkar)

Why Do so Many Egyptian Statues Have Broken Noses? - Artsy

wow, TIL. 'The ancient Egyptians, it’s important to note, ascribed important powers to images of the human form. They believed that the essence of a deity could inhabit an image of that deity, or, in the case of mere mortals, part of that deceased human being’s soul could inhabit a statue inscribed for that particular person. These campaigns of vandalism were therefore intended to “deactivate an image’s strength,” as Bleiberg put it.'

(tags: egypt culture art history noses)

Several Boeing 737 Max 8 pilots in U.S. complained about suspected safety flaw | Airlines | Dallas News

'one captain calling the flight manual "inadequate and almost criminally insufficient"' 'The disclosures found by The News reference problems during Boeing 737 Max 8 flights with an autopilot system, and they were all during takeoff and nose-down situations while trying to gain altitude.'

(tags: boeing planes safety autopilots 737max)

What (And Why) I’m Stockpiling For Brexit

Jack Monroe's shopping list. may be coming in handy soon, unfortunately

(tags: brexit fuck stockpiling prep food tins)

The "Tragedy of the Commons" was invented by a white supremacist based on a false history, and it's toxic bullshit / Boing Boing

Hardin's paper starts with a history of the English Commons -- publicly held lands that were collectively owned and managed -- and the claim that commons routinely fell prey to the selfish human impulse to overgraze your livestock on public land (and that even non-selfish people would overgraze their animals because they knew that their more-selfish neighbors would do so even if they didn't). But this isn't what actually happened to the Commons: they were stable and well-managed until other factors (e.g. rich people trying to acquire even more land) destabilized them. Hardin wasn't just inventing false histories out of a vacuum. He was, personally, a nasty piece of work: a white supremacist and eugenicist, and the Tragedy of the Commons paper is shot through with this vile ideology, arguing that poor people should not be given charity lest they breed beyond their means (Hardin also campaigned against food aid). Hardin was a director of the Federation for American Immigration Reform and the white nationalist Social Contract Press, and co-founded anti-immigrant groups like Californians for Population Stabilization and The Environmental Fund.

(tags: commons capitalism racism garrett-hardin tragedy-of-the-commons politics privatization public-ownership)

How Theranos used the USPTO to defraud investors and patients

When legendary grifter Elizabeth Holmes was 19 years old, she conceived of a medical device that could perform extensive diagnostics in an eyeblink from only a single drop of blood; she had no idea how such a device would work or whether it was even possible, but that didn't stop her from drawing up a patent application for her "invention" and repeatedly submitting to the patent office until, eventually, she was awarded a patent for what amounted to a piece of uninspiring design fiction. For Holmes, the patent was key to convincing investors, partners, and patients that her massive, years-long fraud (a company called "Theranos" bilked investors out of hundreds of millions of dollars) was legit; the USPTO helped her out by trumpeting the importance of patents to "inventors" like Holmes, comparing her to Benjamin Franklin in their public communications. Patents are only supposed to be issued for devices with "utility" -- that is, they have to actually work before you can get a patent for them. But it's been decades since the USPTO has paid meaningful attention to this criterion when evaluating applications, handing out patents for imaginary "inventions" to con artists, delusional hucksters, and other "inventors" who are willing to pay the filing fees that keep the lights on at the Patent Office. And since most people only have a vague idea of the rigor used in patent examination, these patents for design fiction can be used as impressive "proof" when crooks set out to deceive their marks. [....] 'More than a decade after Holmes’ first patent application, Theranos had still not managed to build a reliable blood-testing device. By then the USPTO had granted it hundreds of patents. Holmes had been constructing a fantasy world from the minute she started writing her first application, and the agency was perfectly happy to play along.'

(tags: fraud patents uspto theranos inventions boing-boing)

National Climate Assessment: How to deal with despair over climate change - Vox

The dominant narrative around climate change tells us that it’s our fault. We left the lights on too long, didn’t close the refrigerator door, and didn’t recycle our paper. I’m here to tell you that is bullshit. ... Don’t give in to that shame. It’s not yours. The oil and gas industry is gaslighting you. That same IPCC report revealed that a mere 100 companies are responsible for 71 percent of global climate emissions. These people are locking you and everything you love into a tomb. You have every right to be pissed all the way off. And we have to make them hear about it.

(tags: climate climate-change anger capitalism ipcc fossil-fuels future)

camelcamelcamel, a free Amazon price tracker

'Our free Amazon price tracker monitors millions of products and alerts you when prices drop, helping you decide when to buy.' Supports amazon.co.uk, handily

(tags: amazon shopping deals buying money)

Why do remote meetings suck so much?

Unstructured, "caucus"-style meetings suck particularly badly for remote workers.

When audio/visual delays exacerbate the caucus problem for people who always get the floor in meetings, it looks to them like a new problem. It’s not new; it’s just normally experienced by people in meetings with lower caucus scores. Leadership doesn’t notice because people in leadership positions tend to have higher caucus scores, and being in a position of leadership also tends to boost your caucus score (basically because people interrupt you less). But that’s a weakness of the way we identify decision-makers: good ideas come from everywhere, and especially from people who do a lot of thinking and observing before they say anything. Making meetings more accessible to remote employees doesn’t just make meetings more accessible to remote employees; it makes meetings more accessible to everyone. 

(tags: team meetings remote communication management caucus-score remote-work)

Why did Sumerians use the sexagesimal system?

TIL that 60 is countable using the fingers of both hands, and this ancient Mesopotamian counting technique is still used in India, Pakistan, Indochina, Afghanistan, Iran, Turkey, Syria and Egypt apparently

(tags: sexagesimal 12 60 counting fingers history sumerian mesopotamia)

Algorithms aren’t racist. Your skin is just too dark.

More than a few observers have recommended that instead of pointing out failures, I should simply make sure I use additional lighting. Silence is not the answer. The suggestion to get more lights to increase illumination in an already lit room is a stop gap solution. Suggesting people with dark skin keep extra lights around to better illuminate themselves misses the point. Should we change ourselves to fit technology or make technology that fits us?

(tags: ethics racism ai algorithms cameras melanin skin video videoconferencing)

"Understanding Real-World Concurrency Bugs in Go" (paper)

'Go advocates for the usage of message passing as the means of inter-thread communication and provides several new concurrency mechanisms and libraries to ease multi-threading programming. It is important to understand the implication of these new proposals and the comparison of message passing and shared memory synchronization in terms of program errors, or bugs. Unfortunately, as far as we know, there has been no study on Go’s concurrency bugs. In this paper, we perform the first systematic study on concurrency bugs in real Go programs. We studied six popular Go software including Docker, Kubernetes, and gRPC. We analyzed 171 concurrency bugs in total, with more than half of them caused by non-traditional, Go-specific problems. Apart from root causes of these bugs, we also studied their fixes, performed experiments to reproduce them, and evaluated them with two publicly-available Go bug detectors. Overall, our study provides a better understanding on Go’s concurrency models and can guide future researchers and practitioners in writing better, more reliable Go software and in developing debugging and diagnosis tools for Go.' (via Bill de hOra)

(tags: via:dehora golang go concurrency bugs lint synchronization threading threads bug-detection)

Paper: Hyperscan: A Fast Multi-pattern Regex Matcher for Modern CPUs

a software based, large-scale regex matcher designed to match multiple patterns at once (up to tens of thousands of patterns at once) and to ‘stream‘ (that is, match patterns across many different ‘stream writes’ without holding on to all the data you’ve ever seen). To my knowledge this makes it unique. RE2 is software based but doesn’t scale to large numbers of patterns; nor does it stream (although it could). It occupies a fundamentally different niche to Hyperscan; we compared the performance of RE2::Set (the RE2 multiple pattern interface) to Hyperscan a while back. Most back-tracking matchers (such as libpcre) are one pattern at a time and are inherently incapable of streaming, due to their requirement to backtrack into arbitrary amounts of old input.

(tags: regex regular-expressions algorithms hyperscan sensory-networks regexps simd nfa)

Thought-provoking thread on Facebook/YouTube content moderation

Extremely thought-provoking thread on the horrors of Facebook/YouTube content moderation, from Andrew Strait:

My time doing this work convinced me there is no ultimate mitigation measure for the mental harm it causes. Automation is not a silver bullet - it requires massive labeled data sets by moderators on a continuing basis to ensure accuracy and proper model fit. There are steps to make this process less worse, but IMO it all comes back to a basic question - what technologies are worth the incredible human suffering and cost that moderators will inevitably experience? Is image search worth it? Is YouTube? Is Facebook? I don't have an answer. But these platforms create the need for this kind of horrific work and that must be considered at the forefront of design and deployment of any platform, not as an afterthought.

(tags: horror moderation youtube facebook video content mental-health andrew-strait image-search images labelling google)

Argo Workflows & Pipelines

Nice new workflow system built on Kubernetes and Docker

(tags: k8s kubernetes docker containers workflow pipelines architecture batch nightly-jobs ops)

databricks/devbox

interesting, a synchronization daemon from Databricks which they use to synch up dev repos with a remote "devbox" in EC2 for heavyweight compilation

(tags: remote-compiles compiling devbox databricks coding tools dev ec2)

The Woolsey fire started at the contaminated Santa Susana Field Laboratory

'The speculative and non-credible have now happened. A fire burned through most of the Santa Susana Field Laboratory, a site contaminated with radioactivity and toxic chemicals allowed by decades of shoddy environmental controls.' This site is legendary as a place where dangerous, toxic and radioactive chemicals were experimented with - indeed it was where the 1959 sodium reactor partial meltdown occurred. I wouldn't want to have been downwind of this fire...

(tags: health santa-susana-field-laboratory nuclear fast-breeders atomic-energy meltdowns history toxic-waste california woolsey-fire wildfires)

'digital health will lead to forms of enslavement we can barely imagine'

Author and Consultant Gastroenterologist Dr. Seamus O'Mahony:

Perhaps most alarming of all is his analysis of the future of the world of digital health - "Anyone with a smartphone will be monitoring themselves, or - more likely - will be monitored by some external agency. Health and life insurance companies will offer financial inducements to people to be monitored, and big corporations will undoubtedly make the wearing of health-tracking devices mandatory. The danger of all of this is that in countries where health care is paid for by insurance, a new underclass of uninsured people will emerge. Digital health," he points out, "is presented as something empowering, but the reality is that it will lead to forms of enslavement that we can barely imagine. Facebook and Google have shown how easily people hand over their privacy and personal data in return for a few shiny trinkets. They have also shown how this personal data can be monetised."

(tags: health medicine tracking privacy insurance surveillance data)

Cocktail similarity by Levenshtein distance

Love it. 'I was recently figuring out a minimum-viable bar setup for making cocktails at home, and a system for memorizing & recording recipes. When I started writing down the first basic ingredients, I started noticing that cocktails are very close to each other - if you ignore fruit rinds and ice and such, an Americano is a Negroni with soda water instead of gin. An Old Fashioned is a Manhattan with sugar instead of vermouth. So I wondered - what’s a cocktail edit distance?'

(tags: edit-distance levenshtein-distance algorithms visualization cocktails d3 recipes booze)

Pinterest blocks vaccine-related searches in bid to fight anti-vaxx propaganda | Technology | The Guardian

The phenomenon on display in the Facebook search result screenshots is known in technology circles as a “data void”, after a paper by the Data & Society founder and researcher danah boyd. For certain search terms, boyd explains, “the available relevant data is limited, non-existent, or deeply problematic”. In the case of vaccines, the fact that scientists and doctors are not producing a steady stream of new digital content about settled science has left a void for conspiracy theorists and fraudsters to fill with fear-mongering propaganda and misinformation. [...] Pinterest has responded by building a “blacklist” of “polluted” search terms. “We are doing our best to remove bad content, but we know that there is bad content that we haven’t gotten to yet,” explained Ifeoma Ozoma, a public policy and social impact manager at Pinterest. “We don’t want to surface that with search terms like ‘cancer cure’ or ‘suicide’. We’re hoping that we can move from breaking the site to surfacing only good content. Until then, this is preferable.”

(tags: data-voids danah-boyd pinterest antivax vaccination misinformation disinfo vaccines truth blacklisting)

The log/event processing pipeline you can't have - apenwarr

So good. Apenwarr knows how to design a system.

Simple things don't break. Our friends on the "let's use structured events to make metrics" team streamed those events straight into a database, and it broke all the time, because databases have configuration options and you inevitably set those options wrong, and it'll fall over under heavy load, and you won't find out until you're right in the middle of an emergency and you really want to see those logs. Or events.

(tags: logging scalability klog kernel log-processing events embedded ops)

[1902.04023] Computing Extremely Accurate Quantiles Using t-Digests

'We present on-line algorithms for computing approximations of rank-based statistics that give high accuracy, particularly near the tails of a distribution, with very small sketches. Notably, the method allows a quantile q to be computed with an accuracy relative to max(q,1?q) rather than absolute accuracy as with most other methods. This new algorithm is robust with respect to skewed distributions or ordered datasets and allows separately computed summaries to be combined with no loss in accuracy. An open-source Java implementation of this algorithm is available from the author. Independent implementations in Go and Python are also available.' (via Tony Finch)

(tags: java go python open-source quantiles percentiles approximation statistics sketching algorithms via:fanf)

81 Megapixel image of the moon

I took nearly 50,000 images of the night sky to make an 81 Megapixel image of Tuesday's moon. Uncompressed image linked in the comments. [OC]

(tags: via:elliot art moon astronomy photography hd)

ColouriseSG

'Colourise your black and white photos - A deep learning colouriser prototype specifically for old Singaporean photos.'

(tags: color colour photography ai deep-learning ml colouriser colourisation singapore history black-and-white photos)

The Soothing Promise of Our Own Artisanal Internet | WIRED

"Eat independent sites, mostly not Facebook" as Anil Dash puts it. This is pretty much how I use Mastodon fwiw

(tags: facebook twitter internet web social-media mastodon fediverse anil-dash)

igloohome Smart Mortise

Now this is clever. A "smart lock" which doesn't require access to the internet; uses crypto to allow generation of transient time-limited access codes which you can give to other people, and a TOTP-style algorithm to generate one-time access codes. (via threeze)

(tags: via:threeze smart-locks locks iot crypto igloohome homes)

Functional 3D Printing

nifty subreddit for useful 3D print models

(tags: 3d-printing 3d printing gadgets maker)

The curious case of disappearing buses

Nice investigation into some dodgy pseudo-real-time bus location data in the Bristol real time passenger information system (via Tony Finch)

So what have we learned? One thing we are sure is that data of different qualities – genuinely real-time, pseudo real-time (Type 2 and Type 1), and non-real-time (scheduled) data – all present in the data stream. Among these the most interesting are Type 2 pseudo real-time data. They appear to be the root cause of the phenomenon of disappearing buses. Type 2 pseudo-real-time data are not totally bogus. One possible explanation of their existence can be this. The bus company has limited but not full tracking information on some of their buses. For example, it may know the location of a bus only when the bus leaves the bus terminal. Instead of not showing any data at all about the bus, the bus company uses interpolation to predict the locations of the bus, and reports these as if those are real-time data.

(tags: via:fanf bristol buses public-transport rtpi estimation open-data)

Blockchain: What's Not To Like?

'We're in a period when blockchain or "Distributed Ledger Technology" is the Solution to Everything™, so it is inevitable that it will be proposed as the solution to the problems of academic communication and digital preservation. These proposals typically assume, despite the evidence, that real-world blockchain implementations actually deliver the theoretical attributes of decentralization, immutability, anonymity, security, scalability, sustainability, lack of trust, etc. The proposers appear to believe that Satoshi Nakamoto revealed the infallible Bitcoin protocol to the world on golden tablets; they typically don't appreciate or cite the nearly three decades of research and implementation that led up to it. This talk will discuss the mis-match between theory and practice in blockchain technology, and how it applies to various proposed applications of interest to the CNI audience.'

(tags: talks bitcoin blockchain icos ethereum dao security)

Attack of the week: searchable encryption and the ever-expanding leakage function

In all seriousness: database encryption has been a controversial subject in our field. I wish I could say that there’s been an actual debate, but it’s more that different researchers have fallen into different camps, and nobody has really had the data to make their position in a compelling way. There have actually been some very personal arguments made about it. The schools of thought are as follows: The first holds that any kind of database encryption is better than storing records in plaintext and we should stop demanding things be perfect, when the alternative is a world of constant data breaches and sadness. To me this is a supportable position, given that the current attack model for plaintext databases is something like “copy the database files, or just run a local SELECT * query”, and the threat model for an encrypted database is “gain persistence on the server and run sophisticated statistical attacks.” Most attackers are pretty lazy, so even a weak system is probably better than nothing. The countervailing school of thought has two points: sometimes the good is much worse than the perfect, particularly if it gives application developers an outsized degree of confidence of the security that their encryption system is going to provide them. If even the best encryption protocol is only throwing a tiny roadblock in the attacker’s way, why risk this at all? Just let the database community come up with some kind of ROT13 encryption that everyone knows to be crap and stop throwing good research time into a problem that has no good solution. I don’t really know who is right in this debate. I’m just glad to see we’re getting closer to having it.

(tags: cryptography attacks encryption database crypto security storage ppi gdpr search databases via:ecksor)

Live Transcribe

Google's new live transcription app -- 'see instant captions anywhere. Whether you’re ordering a coffee or meeting someone new, Live Transcribe helps you communicate in the moment.' If this works, it'd be fantastic for the deaf and hard of hearing... nifty!

(tags: android google deaf hearing transcription accessibility)

Marc Brooker on leases

Good advice from Marc Brooker on using leases as a way to handle leader election in a distributed system: 'Leases are a nice primitive because they are easy to understand, easy (if subtle) to implement correctly, require very little co-ordination, optimistic, and don't require much load on the strongly consistent service.'

(tags: leases primitives distributed-systems distcomp networking coding marc-brooker algorithms)

using BGP to compute best paths across the London Underground

this is deeply silly, but also very impressive (via Tony Finch)

(tags: via:fanf bgp networking internet routing london-underground trains)

hrvach/fpg1

a PDP-1 implementation in FPGA:

DEC PDP-1 is a computer designed and produced in 1959. Considering the pace of change in computing, that might seem like the prehistoric age. However, it is also surprisingly modern and proves a point that the basic concepts still withstand the test of time. This project is trying to re-create this computer in FPGA and enable running the first real computer game, SpaceWar!, on a modern display and gamepad. It is designed to run on the MiSTer platform, a retro gaming system based on the Terasic DE10-Nano FPGA board. The implementation is done in Verilog, a hardware description language specifying the structure and behavior of digital logic circuits. This is not a software emulation because there is no CPU executing it. Since this is my first Verilog project ever and its purpose was to teach myself about FPGA, don't expect too much. Beginners often make mistakes or break best practice. Please keep this in mind. Advice and suggestions are welcome!

(tags: pdp1 history computers spacewar fpga hardware verilog)

Revealed: The dark-money Brexit ads flooding social media | openDemocracy

Oh god, here we go again. Facebook ads need to be regulated.

Over the last four months, the People’s Vote and Best for Britain campaigns spent £266,369 and £183,943, respectively. Neither of these anti-Brexit groups is fully transparent either: both publish some details about themselves, such as addresses, but do not publish full details of all funders and donors. During the same time period, Britain’s Future [which does not declare its funders and has no published address] has spent more than £200,000 on Facebook ads. While anti-Brexit spending has slowed down in recent weeks, however, adverts pushing a ‘no deal’ Brexit have spiked. Britain’s Future has spent more than £110,000 on Facebook ads since mid-January. It is not clear where the money for this huge ad push has come from.

(tags: facebook advertising transparency brexit ads political-ads uk)

An In-Depth Guide to Nginx Metrics

decent list of what nginx offers in terms of instrumentation

(tags: nginx metrics ops graphing scalyr)

Google Hired Gig Economy Workers for Project Maven

Other tech giants are reportedly interested in engaging the military as it continues to deploy artificial intelligence technology. Much larger machine-learning projects may require vastly new engagement from gig economy workers, who may unknowingly engage in the work. “Workers absolutely should have the right to know what they are working on, and especially when moral or politically controversial activities are involved,” said Juliet Schor, a sociology professor at Boston College, in an email to The Intercept. “It’s a basic dimension of democracy, which should not stop at either the factory or the platform ‘door.’ For too long, the country has tolerated erosion of basic civil rights in the workplace, as corporations assume ever-more control over their workforces. It’s time to win them back.”

(tags: google project-maven ai training labelling work ethics military)

Understanding the bin, sbin, usr/bin , usr/sbin split

omg. /usr/bin came about because Ken Thompson and Dennis Ritchie ran out of disk space on the root volume. Mind = blown

You know how Ken Thompson and Dennis Ritchie created Unix on a PDP-7 in 1969? Well around 1971 they upgraded to a PDP-11 with a pair of RK05 disk packs (1.5 megabytes each) for storage. When the operating system grew too big to fit on the first RK05 disk pack (their root filesystem) they let it leak into the second one, which is where all the user home directories lived (which is why the mount was called /usr). They replicated all the OS directories under there (/bin, /sbin, /lib, /tmp...) and wrote files to those new directories because their original disk was out of space. When they got a third disk, they mounted it on /home and relocated all the user directories to there so the OS could consume all the space on both disks and grow to THREE WHOLE MEGABYTES (ooooh!).

(tags: filesystem unix history ken-thompson dennis-ritchie disk-space usr)

NYC cops now using Chinese "Sky Net" video surveillance systems

This is absolutely scary. Systematic surveillance:

The surveillance tools are identical to those used in Sky Net in China, the largest video surveillance system on Earth, Chinese government research institutes and a company involved in the project said. At a time when China and the United States are locked in a rivalry on several fronts including trade and technology, Hikvision – the world’s largest surveillance technology company, which is state-owned and based in Hangzhou in eastern China – has supplied the equipment and software used by an American force that polices a population of about 8.6 million people. It has been claimed that Hikvision’s system can accurately identify faces regardless of race, whereas some Western-developed technology had previously been more accurate for white people than for black citizens – although the NYPD has not discussed its reasons for using the Chinese technology. The Sky Net programme, now renamed Pingan Chengshi, or Safe Cities, claimed to have connected 170 million cameras across China last year. By 2020, another 400 million units will be installed, it said, casting a watchful eye on every two citizens. Beijing plans to be able to identify anyone, anytime, anywhere in China within three seconds.

(tags: surveillance new-york nyc skynet china cctv hikvision)

Write tests. Not too many. Mostly integration. – kentcdodds

Nice short summary of Kent Dodds' approach to testing, which I mostly agree with :)

(tags: integration coding testing unit-tests integration-tests system-tests)

One Of The Biggest At-Home DNA Testing Companies Is Working With The FBI

Family Tree reveal that they are providing access to customer-submitted DNA records:

“We are nearing a de-facto national DNA database,” Natalie Ram, an assistant law professor at the University of Baltimore who specializes in bioethics and criminal justice, told BuzzFeed News. “We don’t choose our genetic relatives, and I cannot sever my genetic relation to them. There’s nothing voluntary about that.” Others aired similar concerns. “I would be very against Family Tree DNA allowing law enforcement to have open access to their DNA database,” Debbie Kennett, a British genealogy enthusiast and honorary research associate at University College London said. “I don’t think it’s right for law enforcement to use a database without the informed consent of the consumer.”

(tags: biometrics privacy dna family-tree via:antonio-regalado genealogy data-protection fbi us)

remote AC control for a Nissan Leaf using a Google Home

'OK Google, heat up the car' - nifty

(tags: ok-google google google-home nissan-leaf cars heating gadgets home)

The right-wing history of the urban models which inspired SimCity

Largely forgotten now, Jay Forrester’s Urban Dynamics put forth the controversial claim that the overwhelming majority of American urban policy was not only misguided but that these policies aggravated the very problems that they were intended to solve. In place of Great Society-style welfare programs, Forrester argued that cities should take a less interventionist approach to the problems of urban poverty and blight, and instead encourage revitalization indirectly through incentives for businesses and for the professional class. Forrester’s message proved popular among conservative and libertarian writers, Nixon Administration officials, and other critics of the Great Society for its hands-off approach to urban policy. This outlook, supposedly backed up by computer models, remains highly influential among establishment pundits and policymakers today.

(tags: simulation cities society politics history simcity games jay-forrester will-wright sociology)

The 26,000-Year Astronomical Monument Hidden in Plain Sight at the Hoover Dam

This is amazing! I wish I'd noticed it when I visited Hoover Dam.

The center of the circle traced by the axial precession is actually the massive flag pole in the center of the plaza. This axial circle is prominently marked around the pole, and the angle of Polaris was depicted as precisely as possible to show where it would have been on the date of the dam’s opening. Hansen used the rest of the plaza floor to show the location of the planets visible that evening, and many of the bright stars that appear in the night sky at that location. By combining planet locations with the angle of precession, we are able to pinpoint the time of the dam’s completion down to within a day. We are now designing a similar system?—?though with moving parts?—?in the dials of the 10,000 Year Clock. It is likely that at least major portions of the Hoover Dam will still be in place hundreds of thousands of years from now. Hopefully the Clock will still be ticking and Hansen’s terrazzo floor will still be there, even if it continues to baffle visitors.

(tags: hoover-dam history precession astronomy long-now polaris vega thuban stars clocks)

Facial Recognition Is the Perfect Tool for Oppression

'We believe facial recognition technology is the most uniquely dangerous surveillance mechanism ever invented. It’s the missing piece in an already dangerous surveillance infrastructure, built because that infrastructure benefits both the government and private sectors. And when technologies become so dangerous, and the harm-to-benefit ratio becomes so imbalanced, categorical bans are worth considering. The law already prohibits certain kinds of dangerous digital technologies, like spyware. Facial recognition technology is far more dangerous. It’s worth singling out, with a specific prohibition on top of a robust, holistic, value-based, and largely technology-neutral regulatory framework. Such a layered system will help avoid regulatory whack-a-mole where lawmakers are always chasing tech trends. Surveillance conducted with facial recognition systems is intrinsically oppressive. The mere existence of facial recognition systems, which are often invisible, harms civil liberties, because people will act differently if they suspect they’re being surveilled. Even legislation that holds out the promise of stringent protective procedures won’t prevent chill from impeding crucial opportunities for human flourishing by dampening expressive and religious conduct.'

(tags: tech surveillance facial-recognition faces oppression future chilling-effects)

Amendment-apocalypse: Spineless MPs just voted against reality

... in a way, it was typical May tactics. She prioritised vague promises over content. She sabotaged something - anything - in order to fight another day. She made promises she could not keep on issues she knew to be false. Once again, she said anything, anything at all, to survive just a little longer. [...] it has significant medium-term implications too. Firstly, it shows why the backstop was needed in the first place. This country has become an unreliable negotiating partner. It will demand something one day then seek to detonate it the next. The events in the Commons today actually had the ironic effect of reaffirming to the EU the need for the backstop insurance policy. On a broader level, we are about to go around the world asking for trade deals. But we're seen, by everyone, on the largest stage imaginable, to be fundamentally politically insane. We've gone mad and everyone is looking. This is as bleak a day as we have had in the entire Brexit process. All roads now seem blocked. MPs won't back an extension to Article 50. They won't back May's deal. And they won't back no-deal. They've opted for fairy tales over action. Things are looking very bad indeed.

(tags: brexit politics uk eu backstop)

Security Things to Consider When Your Apartment Goes ‘Smart’

Good advice, and I'd be pretty unhappy about this if it happened to me too.

If you’re a tenant in the US, it’s very likely that a management-provided smart home system is headed your way in the near future. Carefully evaluate your family’s personal threat model, and consider the plausible digital ways which these systems could be exploited. Spend some time reading into the vendor. Respectfully and courteously encourage your property management company and their smart system vendor to adopt industry best practices in securing smart hubs physically and digitally, the networks they are connected to, and and resident data at rest and in transit in their infrastructure. Request your property managers clearly and decisively address privacy concerns such as data ownership and resale in writing. If solid answers in writing don’t assuage legitimate concerns, consider politely seeking an option to opt-out – and make your threat model clear to them, if you’re in a sensitive situation.

(tags: locks iot security internetofshit tenancy renting smart-hubs smart-homes smart-locks)

Daniel Edward Heffernan’s Map of Dublin, 1861

These maps are really remarkable work:

Heffernan’s map of Dublin, published 1 May 1861, is both highly unusual and very fine. No other map of the time gives such a bird’s eye, almost axonometric, view of the city’s edifices. It shows in exceptional elevational detail both a number of the city’s principal public buildings, but also its less vaunted institutions; prisons, hospitals, workhouses etc., all within the municipal boundary set by the two canals.

(tags: daniel-heffernan history 19th-century dublin ireland maps)

In praise of the sci-fi corridor

A lovely appraisal of classic 1970s SF set design

(tags: sets sf scifi movies corridors)

We may finally know what causes Alzheimer’s – and how to stop it

This is amazing:

If you bled when you brushed your teeth this morning, you might want to get that seen to. We may finally have found the long-elusive cause of Alzheimer’s disease: Porphyromonas gingivalis, the key bacteria in chronic gum disease. That’s bad, as gum disease affects around a third of all people. But the good news is that a drug that blocks the main toxins of P. gingivalis is entering major clinical trials this year, and research published today shows it might stop and even reverse Alzheimer’s. There could even be a vaccine.

(tags: via:johnlooney gingivitis alzheimers brain health medicine teeth)

Brexit: debunking "trading on WTO terms"

a favourite brexiteer talking point demolished

(tags: wto trade brexit uk)

research!rsc: Our Software Dependency Problem

The kind of critical examination of specific dependencies that I outlined in this article is a significant amount of work and remains the exception rather than the rule. But I doubt there are any developers who actually make the effort to do this for every possible new dependency. I have only done a subset of them for a subset of my own dependencies. Most of the time the entirety of the decision is “let’s see what happens.” Too often, anything more than that seems like too much effort. But the Copay and Equifax attacks are clear warnings of real problems in the way we consume software dependencies today. We should not ignore the warnings. I offer three broad recommendations. * Recognize the problem. If nothing else, I hope this article has convinced you that there is a problem here worth addressing. We need many people to focus significant effort on solving it. * Establish best practices for today. We need to establish best practices for managing dependencies using what’s available today. This means working out processes that evaluate, reduce, and track risk, from the original adoption decision through to production use. In fact, just as some engineers specialize in testing, it may be that we need engineers who specialize in managing dependencies. * Develop better dependency technology for tomorrow. Dependency managers have essentially eliminated the cost of downloading and installing a dependency. Future development effort should focus on reducing the cost of the kind of evaluation and maintenance necessary to use a dependency. For example, package discovery sites might work to find more ways to allow developers to share their findings. Build tools should, at the least, make it easy to run a package’s own tests. More aggressively, build tools and package management systems could also work together to allow package authors to test new changes against all public clients of their APIs. Languages should also provide easy ways to isolate a suspect package.

(tags: dependencies software coding work)

PhoneNumbers.ie

'Do you want to find out information about an unknown caller?' -- reputation service for random callers in Ireland. Very useful to find out if others have received scammy calls from a given number

(tags: reputation phone telephone callers scams phishing ireland)

The Plot Against George Soros

The anti-Soros campaign was entirely artificial, as a means to elect Orban in Hungary:

Orbán was busy creating a new, more dramatic story of the nation. Hungary, which had collaborated with the Nazis, was painted as a victim, surrounded by external enemies, under perpetual siege, first from the Ottomans, then the Nazis, and later the Communists. Hungary’s mission was clear: to defend against its enemies, and to preserve Christianity against encroaching Islam and secular forces. Against this backdrop, Finkelstein had an epiphany. What if the veil of the conspiracy were to be lifted and a shadowy figure appear, controlling everything? The puppet master. Someone who not only controlled the “big capital” but embodied it. A real person. A Hungarian. Strange, yet familiar. That person was Soros, Finkelstein told Birnbaum. Birnbaum was mesmerized: Soros was the perfect enemy. [....] Despite everything that followed, Birnbaum is proud of the campaign against Soros: “Soros was a perfect enemy. It was so obvious. It was the simplest of all products, you just had to pack it and market it.” The product was so good, it sold itself and went global. In 2017, Italians started talking about Soros-financed immigrant boats arriving on the shores. In the US, some people suspected Soros was behind the migrant caravan entering from Central America. A Polish member of parliament called Soros the “most dangerous man in the world.” Putin referred dismissively to Soros during a press conference with Trump in Helsinki. Trump even claimed that the demonstrations against Supreme Court candidate Brett Kavanaugh were sponsored by Soros. Today Finkelstein and Birnbaum’s work in Hungary has echoes everywhere. Birnbaum denied the suggestion that he had run anti-Soros campaigns outside of Hungary. But perhaps he didn’t have to. Anyone could pick up the ideas and run with them. Finkelstein and Birnbaum had turned Soros into a meme. Right-wing sites like Breitbart, or the Kremlin-controlled Russia Today, could simply adopt the Hungarian campaign, translate it into other languages, and feed it with local arguments. If right-wing movements want to campaign today, they can source Soros material from the internet. Anti-Soros material is a globalized, freely available, and adaptable open-source weapon. Birnbaum said it was the common denominator of the nationalist movement.

(tags: george-soros conspiracies george-birnbaum antisemitism hungary arthur-finkelstein politics campaigning)

Stack Overflow: How We Do Monitoring - 2018 Edition

interesting to see how the other half lives, as Stack Overflow is a .NET shop

(tags: logging monitoring stack-overflow dotnet ops metrics)

'The goal is to automate us': welcome to the age of surveillance capitalism

“Surveillance capitalism,” she writes, “unilaterally claims human experience as free raw material for translation into behavioural data. Although some of these data are applied to service improvement, the rest are declared as a proprietary behavioural surplus, fed into advanced manufacturing processes known as ‘machine intelligence’, and fabricated into prediction products that anticipate what you will do now, soon, and later. Finally, these prediction products are traded in a new kind of marketplace that I call behavioural futures markets. Surveillance capitalists have grown immensely wealthy from these trading operations, for many companies are willing to lay bets on our future behaviour.” While the general modus operandi of Google, Facebook et al has been known and understood (at least by some people) for a while, what has been missing – and what Zuboff provides – is the insight and scholarship to situate them in a wider context. She points out that while most of us think that we are dealing merely with algorithmic inscrutability, in fact what confronts us is the latest phase in capitalism’s long evolution – from the making of products, to mass production, to managerial capitalism, to services, to financial capitalism, and now to the exploitation of behavioural predictions covertly derived from the surveillance of users.

(tags: advertising technology surveillance facebook google adtech capitalism business)

Instance Price Guide

Nicely done EC2 instance price comparison site, with spot instance and reserved pricing discounts taken into account; possibly better than ec2instances.info

(tags: ec2 instances aws pricing hosts spot-instances money)

mbasic.facebook.com

Super-basic HTML-only view of Facebook -- ugly but blisteringly fast with no HTML5 crapola

(tags: light facebook basic html javascript)

Opinion | The Malign Incompetence of the British Ruling Class - The New York Times

Pankaj Mishra wastes no time getting the boot into the ineptitude of the ruling-class Brexiteers. it's glorious

Politicians and journalists in Ireland are understandably aghast over the aggressive ignorance of English Brexiteers. Business people everywhere are outraged by their cavalier disregard for the economic consequences of new borders. But none of this would surprise anyone who knows of the unconscionable breeziness with which the British ruling class first drew lines through Asia and Africa and then doomed the people living across them to endless suffering.

(tags: britain india brexit ireland pakistan asia partition history colonialism pankaj-mishra)

_Amazon Aurora: On Avoiding Distributed Consensus for I/Os, Commits, and Membership Changes_, SIGMOD '18

One of the more novel differences between Aurora and other relational databases is how it pushes redo processing to a multi-tenant scale-out storage service, purpose-built for Aurora. Doing so reduces networking traffic, avoids checkpoints and crash recovery, enables failovers to replicas without loss of data, and enables fault-tolerant storage that heals without database involvement. Traditional implementations that leverage distributed storage would use distributed consensus algorithms for commits, reads, replication, and membership changes and amplify cost of underlying storage. In this paper, we describe how Aurora avoids distributed consensus under most circumstances by establishing invariants and leveraging local transient state. Doing so improves performance, reduces variability, and lowers costs.

(tags: papers toread aurora amazon aws pdf scalability distcomp state sql mysql postgresql distributed-consensus)

On the association between adolescent well-being and digital technology use

Lies, damn lies, and statistics. 'if we believe screens are destroying a generation [of kids] that would mean that so are potatoes, having asthma, not drinking milk, going to movies, music, religion, being tall, biking, and wearing glasses' [...] 'The take home from this new study is the evidence that smart phones are destroying a generation is not any stronger than potatoes and eyeglasses are destroying a generation. The moral panic surrounding the fear of screens is simply not supported by good science.'

(tags: potatoes funny glasses asthma milk movies music religion cycling screens screentime kids teenagers wellbeing mental-health)

Serverless Computing: One Step Forward, Two Steps Back - Speaker Deck

So much agreement with this slide deck, particularly the list of limitations of current FaaS: 15 min lifetimes; I/O bottlenecks; no inbound network comms; no specialized hardware; and the general horribleness of using DynamoDB or S3 state as a platform for distributed computing protocols.

(tags: faas lambda serverless fail slides architecture aws dynamodb s3 cloud)

Some facts on immigration to Ireland

Handy to have to hand next time right-wing talking points emerge:

Let’s summarise: Ireland has a relatively high level of non-citizens in its population. But this is down to the high level of UK citizens and citizens from other English-speaking countries (US, Canada, Australia and New Zealand). Ireland has significantly fewer non-citizens from outside the English-speaking world than high-income EU countries. The proportion of non-citizens has remained stable over the last 10 years (i.e. there is no ‘surge’). Non-citizens in Ireland are more integrated into the labour market than any other high-income EU country – that is, there is lower unemployment among non-citizens. So much for the ‘sponging-off-the-state’ argument. We have had far fewer asylum-seekers and we grant asylum to far fewer than most other high-income EU countries. The claims of the Far Right and their allies collapse when we look to reality. 

(tags: immigration facts statistics ireland asylum-seekers)

The Embroidered Computer

an exploration into using historic gold embroidery materials and knowledge to craft a programmable 8 bit computer. Solely built from a variety of metal threads, magnetic, glass and metal beads, and being inspired by traditional crafting routines and patterns, the piece questions the appearance of current digital and electronic technologies surrounding us, as well as our interaction with them. Technically, the piece consists of (textile) relays, similar to early computers before the invention of semiconductors. Visually, the gold materials, here used for their conductive properties, arranged into specific patterns to fulfil electronic functions, dominate the work. Traditionally purely decorative, their pattern here defines they function. They lay bare core digital routines usually hidden in black boxes. Users are invited to interact with the piece in programming the textile to compute for them.

(tags: electronics computers computing art embroidery craft gold 8-bit)

Crunching The Numbers: How Much Will Your Electric Car Really Cost? | CleanTechnica

some good real-world measurements from Germany. Ireland's residential electricity costs are roughly comparable to the Netherlands I think in price/kWh

(tags: evs cars driving costs home household electricity)

Apache Iceberg (incubating)

Coming to presto soon apparently....

Iceberg tracks individual data files in a table instead of directories. This allows writers to create data files in-place and only adds files to the table in an explicit commit. Table state is maintained in metadata files. All changes to table state create a new metadata file and replace the old metadata with an atomic operation. The table metadata file tracks the table schema, partitioning config, other properties, and snapshots of the table contents. The atomic transitions from one table metadata file to the next provide snapshot isolation. Readers use the latest table state (snapshot) that was current when they load the table metadata and are not affected by changes until they refresh and pick up a new metadata location.

(tags: presto storage s3 hive iceberg apache asf data architecture)

certain Irish surnames inherited ‘the cure’

This is quite an odd superstition -- the belief that people with a certain surname could cure a specific ailment.

Wicklow-born reader Mattie Lennon reminds me that certain Irish families used to hold monopolies over the treatment of individual disorders, based solely on their surnames. Mattie knows this from experience. As a child in the 1950s, he contracted shingles. And it was an article of faith then that anyone by the name of Keogh could cure that painful condition. There were no mysterious herbal concoctions involved: the power resided in their veins, literally. Thus a man named Darby Keogh was called, “bled his fingers, mixed the blood with holy water, and applied it”. Scoff all you like readers, but “it cured my shingles”, says Mattie,

(tags: health history superstitions ireland wicklow quackery the-cure)

PRNG-predicting bot ascends in Nethack in 7 minutes 15 seconds

This is a really amazing hack!

So, we could now go from starting inventory to RNG-seed in no time, but we still had to ascend. While we could write a simple “wish”-bot, get +127 Magicbane and then ascend by hand from there, achieving both fastest realtime and lowest turn count in the same game would not be easy. First idea was to start a NAO game, fetch the seed, then saving and perfecting the seed offline. It was quickly ruled out though as the game is reseeded each time it is started (i.e. when you load your game). Loooong story short, we wrote a bot. You had to play the first turns (offline) and move her to a non-magic fountain located next to a wall. If you died, no big deal, just retry on the same seed. This is why SWAGGINZZZ stood still for 6 minutes, we had absolutely horrible RNG when trying to get the specific fountain needed on dlvl2. The fountain is required for wishes. The wall is required to be able to offset the random state without advancing the game state – every time the character attempts to walk into a wall, it calls random() without wasting any in-game time. From the fountain, the bot ascends completely on her own.

(tags: nethack random prng games hacks cheats nao amazing)

glibc changed their UTF-8 character collation ordering across versions, breaking postgres

This is terrifying:

Streaming replicas—and by extension, base backups—can become dangerously broken when the source and target machines run slightly different versions of glibc. Particularly, differences in strcoll and strcoll_l leave "corrupt" indexes on the slave. These indexes are sorted out of order with respect to the strcoll running on the slave. Because postgres is unaware of the discrepancy is uses these "corrupt" indexes to perform merge joins; merges rely heavily on the assumption that the indexes are sorted and this causes all the results of the join past the first poison pill entry to not be returned. Additionally, if the slave becomes master, the "corrupt" indexes will in cases be unable to enforce uniqueness, but quietly allow duplicate values.

(tags: postgresql scary ops glibc collation utf-8 characters indexing sorting replicas postgres)

A UK police force is dropping tricky cases on advice of an algorithm

Wow, this is a terrible idea. It will definitely launder existing human bias into its decisions.

However, because the technology bases its predictions on past investigations, any biases contained in those decisions may be reinforced by the algorithm. For example, if there are areas that don’t have CCTV and police frequently decided not to pursue cases there, people in those places could be disadvantaged. “When we train algorithms on the data on historical arrests or reports of crime, any biases in that data will go into the algorithm and it will learn those biases and then reinforce them,” says Joshua Loftus at Stanford University in California. [...] Police forces only ever know about crimes they detect or have reported to them, but plenty of crime goes unreported, especially in communities that have less trust in the police. This means the algorithms are making predictions based on a partial picture. While this sort of bias is hard to avoid, baking it into an algorithm may make its decisions harder to hold to account compared with an officer’s. John Phillips, superintendent at Kent Police, says that for the types of crimes that EBIT is being used for, under-reporting isn’t an issue and so shouldn’t affect the tool’s effectiveness.

(tags: bias machine-learning ml ai cctv police uk kent policing)

How Millennials Became The Burnout Generation

That realization recast my recent struggles: Why can’t I get this mundane stuff done? Because I’m burned out. Why am I burned out? Because I’ve internalized the idea that I should be working all the time. Why have I internalized that idea? Because everything and everyone in my life has reinforced it — explicitly and implicitly — since I was young. Life has always been hard, but many millennials are unequipped to deal with the particular ways in which it’s become hard for us.

(tags: burnout life work workaholism millenials anxiety)

White Matter Integrity in Obstructive Sleep Apnea before and after Treatment

Sleep Apnea causes physical damage to the brain, particularly in white matter fibre integrity. Thankfully, it is reversible.

Results in pre-treatment OSA patients showed impairments in most cognitive areas, mood and sleepiness that were associated with diffuse reduction of WM fiber integrity reflected by diminished fractional anisotropy (FA) and mean diffusivity (MD) in multiple brain areas. After 3 months of CPAP, only limited changes of WM were found. However, over the course of 12 months CPAP treatment, an almost complete reversal of WM abnormalities in all the affected regions was observed in patients who were compliant with treatment. Significant improvements involving memory, attention, and executive-functioning paralleled WM changes after treatment.

(tags: brain brain-damage sleep sleep-apnea snoring medicine papers)

"Cracking the Coding Interview" handouts

amazing handouts for interviewees looking to do well in technical interviews; there are 3, coding, PM, and soft-skills-oriented. I'm on the other side of the table these days but these are fantastic

(tags: interviews coding career interviewing hiring)

AWS Service SLAs

The goal of this page is to high-light the lack of coverage AWS provides for its services across different security factors. These limitations are not well-understood by many. Further, the "Y" fields are meant to indicate that this service has any capability for the relevant factor. In many cases, this is not full coverage for the service, or there are exceptions or special cases.

(tags: amazon aws services slas ops reliability)

Trek10 | From relational DB to single DynamoDB table: a step-by-step exploration

tl;dr:

Is modeling my relational database in a single DynamoDB table really a good idea? About a year ago, I wrote a fairly popular article called “Why DynamoDB isn’t for everyone”. Many of the technical criticisms of DynamoDB I put forth at that time (lack of operational controls such as backup/restore; a persistent problem with hot keys) have since been partially or fully resolved due to a truly awe-inspiring run of feature releases from the DynamoDB team. However, the central argument of that article remains valid: DynamoDB is a powerful tool when used properly, but if you don’t know what you’re doing it’s a deceptively user-friendly guide into madness. And the further you stray into esoteric applications like relational modeling, the more sure you’d better be that you know what you’re getting into. Especially with SQL-friendly “serverless” databases like Amazon Aurora hitting their stride, you have a lot of fully-managed options with a smaller learning curve.

(tags: dynamodb databases storage nosql sql relational aws relations)

Give Up the Ghost: A Backdoor by Another Name | Just Security

Now that GCHQ have asked for this, I suspect plenty of other government bodies around the world will be looking for similar.

They’re talking about adding a “feature” that would require the user’s device to selectively lie about whether it’s even employing end-to-end encryption, or whether it’s leaking the conversation content to a third (secret) party. Is the security code displayed by your device a mathematical representation of the two keys involved, or is it a straight-up lie? Furthermore, what’s to guarantee that the method used by governments to insert the “ghost” key into a conversation without alerting the users won’t be exploited by bad actors? Despite the GCHQ authors’ claim, the ghost will require vendors to disable the very features that give our communications systems their security guarantees in a way that fundamentally changes the trust relationship between a service provider and its users. Software and hardware companies will never be able to convincingly claim that they are being honest about what their applications and tools are doing, and users will have no good reason to believe them if they try. And, as we’ve seen already seen, GCHQ will not be the only agency in the world demanding such extraordinary access to billions of users’ software. Australia was quick to follow the UK’s lead, and we can expect to see similar demands, from Brazil and the European Union to Russia and China. (Note that this proposal would be unconstitutional were it proposed in the United States, which has strong protections against governments forcing actors to speak or lie on its behalf.) We must reject GCHQ’s newest “ghost” proposal for what it is: a mandated encryption backdoor that weakens the security properties of encrypted messaging systems and fundamentally compromises user trust.

(tags: crypto ghost gchq security backdoors uk)

FFmpeg, SOX, Pandoc and RSVG for AWS Lambda

OK-ish way to add dependencies to your Lambda containers:

The basic AWS Lambda container is quite constrained, and until recently it was relatively difficult to include additional binaries into Lambda functions. Lambda Layers make that easy. A Layer is a common piece of code that is attached to your Lambda runtime in the /opt directory. You can reuse it in many functions, and deploy it only once. Individual functions do not need to include the layer code in their deployment packages, which means that the resulting functions are smaller and deploy faster. For example, at MindMup, we use Pandoc to convert markdown files into Word documents. The actual lambda function code is only a few dozen lines of JavaScript, but before layers, each deployment of the function had to include the whole Pandoc binary, larger than 100 MB. With a layer, we can publish Pandoc only once, so we use significantly less overall space for Lambda function versions. Each code change now requires just a quick redeployment.

(tags: serverless lambda dependencies deployment packaging ops)

Allthefood

Decent new Dublin food blog -- reviews and news. Like a Harbo-free version of Lovin' Dublin

(tags: dublin food eating restaurants reviews)

We’ll Never Know Whether Monorepos Are Better

This makes sense to me where the monorepo-vs-polyrepo argument is concerned -- it's another religious war:

About six months after the project was declared “done” (but there was always more to do, more improvements to make to our homegrown dependency management solution), we had a retrospective meeting. The same engineers who had taken sides, for and against the project, were again assembled to discuss how it went. One of the main opponents went first. “Thank goodness we’re finally having this retrospective,” he said. “I think we can all see that this experiment has been a colossal failure and that it’s time for us to change course and roll back to monorepo.” “What do you mean?” one of the main multirepo advocates replied. “This was one of the best decisions we’ve ever made!” This really shocked me. We had access to all of the data you could possibly want to evaluate the decision. The same engineers working with the same codebase had seen what it was like in the monorepo model and the multirepo model. We knew exactly how much it had actually cost to switch. We had lived with the advantages and disadvantages of both models. But still we couldn’t come to an agreement. That retrospective taught me to be humble in my ambitions to “improve” engineering productivity. There’s no way to measure productivity in software, so there’s no way to know whether controversial, expensive “productivity enhancing” projects actually deliver on their promise, even in hindsight.

(tags: monorepo productivity dev engineering coding polyrepo)

I Accidentally Made Myself Lactose Intolerant With Whole30

A few years back, I had a nasty bout of food poisoning while travelling, which made me lactose-intolerant for several years. Sounds like this may be more common than you'd think, based on this article:

If you haven’t heard of Whole30, some information: It’s a month-long eating plan that aims to help followers hit “the reset button with your health, habits, and relationship with food.” For 30 days, you cut out soy, legumes, grains, sugars, alcohol, and, of course, dairy. [....] When you reach the end of the Whole30, you’re supposed to add the forbidden food groups back into your diet one at a time. The goal is to figure out which foods are making you feel sluggish, bloated, or just generally not great, so you can ostensibly keep on avoiding them forever. I didn’t do that part. I just jumped right back into eating what I wanted — but suddenly nothing was the same. That first bowl of ice cream I’d been looking forward to for weeks was quickly followed by sharp stomach pains and what can best be described (grossly, but accurately) as bubble gut. [....] The good news, according to gastrointestinal specialist Kim Barrett, a professor of medicine at the University of California, San Diego, is that I’m not crazy. The bad news is that dairy no longer agrees with my body’s biology. Turns out, it is possible to suddenly make yourself lactose intolerant. “To some extent, our ability to handle lactose is a use-it-or-lose-it phenomenon,” Barrett says. The body digests lactose — a disaccharide — by using lactase, an enzyme in the small intestine, to break it down into the monosaccharides glucose and galactose, which can then be absorbed. “If you don’t have the [lactose] substrate in the diet, you start to reduce the synthesis of the lactase enzyme to digest it,” Barrett explains. “After a period of completely excluding lactose from the diet, you may not have any of those digestive enzymes present.”

(tags: diet food lactose lactase intolerance whole30 milk cheese)

The Hydrogen Fuel Cell Scam

The need to have a massive and costly buildout of [hydrogen refuelling] infrastructure remains one of the most glaring problems of a supposed transition to a hydrogen economy. The cost of a single hydrogen fueling station is likely to be over $2 million. This is in contrast to the relatively modest $50,000 cost of deploying a high-speed battery-electric car charging station. Another factor here is the reality of putting into place an all-new infrastructure from scratch versus building off of an already existing electrical infrastructure that exists in every developed nation.  The fact that every building, garage, and lamp post in the US is already electrified means we simply have to add one final component to the existing and established network.

(tags: hydrogen green driving cars fuel fossil-fuels decarbonisation)

Don’t buy a 5G smartphone—at least, not for a while | Ars Technica

wow, 5G sounds like it's going to be terrible

(tags: 5g 4g mobile-phones mobile tech hardware radio)

Risky business: linking _Toxoplasma gondii_ infection and entrepreneurship behaviours across individuals and countries | Proceedings of the Royal Society B: Biological Sciences

Using a saliva-based assay, we found that students (n = 1495) who tested IgG positive for Toxoplasma gondii exposure were 1.4× more likely to major in business and 1.7× more likely to have an emphasis in ‘management and entrepreneurship' over other business-related emphases. Among professionals attending entrepreneurship events, T. gondii-positive individuals were 1.8× more likely to have started their own business compared with other attendees (n = 197). Finally, after synthesizing and combining country-level databases on T. gondii infection from the past 25 years with the Global Entrepreneurship Monitor of entrepreneurial activity, we found that infection prevalence was a consistent, positive predictor of entrepreneurial activity and intentions at the national scale, regardless of whether previously identified economic covariates were included. Nations with higher infection also had a lower fraction of respondents citing ‘fear of failure' in inhibiting new business ventures. While correlational, these results highlight the linkage between parasitic infection and complex human behaviours, including those relevant to business, entrepreneurship and economic productivity.

(tags: science biology infection toxoplasmosis parasites humans behaviour entrepreneurs business brains economics)

"Thread on climate change & Irish agri lobbyists"

'The massive levy income raised on beef & dairy gives the IFA a financial incentive to promote beef & dairy expansion and a disincentive to support any fundamental changes to Ireland’s agricultural model, regardless the public interest.'

(tags: dairy beef ireland farming climate-change climate lobbying government)

slides from "Distributed Log-Processing Design Workshop", SRECon Americas 2018

Fantastic presentation discussing the kinds of design criteria used when architecting a large-scale data processing and storage service. Interesting to see some Google terminology, e.g. "dimensioning" -- ballparking the expected scalability numbers, bandwidth, qps, and limits.

(tags: distributed-systems coding design architecture google photon logs log-storage slides srecon)

Zstandard: How Facebook increased compression speed - Facebook Code

very cool; particularly the high level of support for external-dictionary compression

(tags: facebook compression zstd zstandard dictionary-compression)

The Holiday Drink of 2018: Cognac Punch

Looks tasty, albeit pricey and strong:

London Punch House Punch By David Wondrich Ingredients: 4 Lemons 6 oz (3/4 cup) White sugar 20 oz Hennessy Privilege VSOP Cognac 6 oz Jamaican rum 1 quart (4 cups) Cold water Glass: Punch cup Garnish: Freshly grated nutmeg Directions: The day before you serve the punch, fill a quart container with water and put it in the freezer. Peel 4 lemons in long spirals and put the peels into a mason jar with 6 oz (3/4 cup) of white sugar. Seal the jar, shake it and let sit overnight. Two hours before you serve the punch, unseal the mason jar, add 6 oz (3/4) cup fresh-squeezed, strained lemon juice, reseal and shake until all the sugar has dissolved. Refrigerate. To assemble the punch, unmold the ice block and place it in a 1-gallon punch bowl. Shake the contents of the mason jar and pour it into the punch bowl unstrained, peels and all. Add 20 oz (2-and-a-half cups) Hennessy Privilege VSOP Cognac and 6 oz (3/4 cup) Jamaican rum. Stir. Add 1 quart (4 cups) cold water. Stir again and grate nutmeg on top. Using a punch ladle, drape a few ends of the spiral lemon peels over the rim of the bowl. Grate nutmeg over the top.

(tags: punch cognac hennessy recipes rum)

CRDTs in Production

at PayPal (via Carlos Baquero)

(tags: via:xmal crdts distributed-systems distcomp prod coding data-structures)

Remove Background from Image

A fully automated background-removal tool. Nicely done

(tags: images editing tools background graphics)

Google used a Baidu front-end to scrape user searches without consent

The engineers used the data they pulled from [acquired Baidu front-end site] 265.com to learn about the kinds of things that people located in mainland China routinely search for in Mandarin. This helped them to build a prototype of Dragonfly. The engineers used the sample queries from 265.com, for instance, to review lists of websites Chinese people would see if they typed the same word or phrase into Google. They then used a tool they called “BeaconTower” to check whether any websites in the Google search results would be blocked by China’s internet censorship system, known as the Great Firewall. Through this process, the engineers compiled a list of thousands of banned websites, which they integrated into the Dragonfly search platform so that it would purge links to websites prohibited in China, such as those of the online encyclopedia Wikipedia and British news broadcaster BBC. Under normal company protocol, analysis of people’s search queries is subject to tight constraints and should be reviewed by the company’s privacy staff, whose job is to safeguard user rights. But the privacy team only found out about the 265.com data access after The Intercept revealed it, and were “really pissed,” according to one Google source.

(tags: china search tech google privacy baidu interception censorship great-firewall dragonfly)

awslabs/amazon-kinesis-scaling-utils

The Kinesis Scaling Utility is designed to give you the ability to scale Amazon Kinesis Streams in the same way that you scale EC2 Auto Scaling groups – up or down by a count or as a percentage of the total fleet. You can also simply scale to an exact number of Shards. There is no requirement for you to manage the allocation of the keyspace to Shards when using this API, as it is done automatically. You can also deploy the Web Archive to a Java Application Server, and allow Scaling Utils to automatically manage the number of Shards in the Stream based on the observed PUT or GET rate of the stream.

(tags: kinesis scaling scalability shards sharding ops)

The “Bart” — sudden hundreds-of-Bitcoin pumps or dumps, to burn the margin traders

Finance journalists need to stop treating crypto as an efficient market that responds to concerns. It’s a thinly-traded unregulated playground for whales, out to wreck the margin traders. A $400 dip in fifteen minutes is not a “market signal” — it’s a deliberate dump to manipulate the price. Though there’s still downward pressure on the price — all the suckers from the bubble have gone home, so they’re not buying … but the miners still have to sell coins for actual money to pay for their electricity. And even more so now that the price of mining one bitcoin is at — or above — what you could get for selling that bitcoin. So one minute you’ll see a sudden $100 increase in the price that cost 130 BTC of dollars — and those are actual dollars going in — followed the next minute by a matching $100 drop that came from selling only 30 BTC. It’s much easier to drop the price than raise it.

(tags: investing bitcoin via:harikunzru cryptocurrency pump-and-dump barts finance margin-traders currencies fraud)

GDPR Subject Access Request helper for Dublin Bus vs bike incidents

'A new tool has been created to assist people who want to request video footage of incidents involving Dublin Bus under GDPR rules.'

(tags: dublin-bus bus bike cycling dublin cameras gdpr sars)

8 Myths of Abortion in Ireland

from the Abortion Rights Campaign's "8 Days - 8 Reasons to Repeal the 8th" myth-busting series in 2014.

(tags: arc abortion ireland politics mythbusters repealthe8th)

Notebookcheck's Top 10 Tablets under 250 Euros

a decent list of low-end tablets, given the season

(tags: tablets devices hardware android gadgets xmas)

A primer on privacy as "contextual integrity"

A primer on privacy as "contextual integrity" and why privacy notices on mobile platforms (both Android and iOS) are insufficient for attaining informed consent. If your doctor asked for permission to collect your medical history, you would probably say yes. However, if that doctor asked to collect your medical history to give to marketers for advertising purposes, you would probably decline. The difference is, in the first case, you're making assumptions about how the data will be used based on who is making the request. Knowing just the type of data requested and the requester are insufficient to make an informed decision: people also consider the purpose and other constraints (e.g., will data be resold, stored securely, etc.), which are equally important factors.

(tags: privacy context understanding mobile data-protection permission opt-in)

Makisu

Uber’s Fast, Reliable Docker Image Builder for Apache Mesos and Kubernetes.

we built our own image building tool, Makisu, a solution that allows for more flexible, faster container image building at scale. Specifically, Makisu: requires no elevated privileges, making the build process portable. uses a distributed layer cache to improve performance across a build cluster. provides flexible layer generation, preventing unnecessary files in images. is Docker-compatible, supporting multi-stage builds and common build commands.

(tags: makisu docker containers ops build mesos kubernetes building)

Billionaires Are the Leading Cause of Climate Change

That's largely because there is no "free market" incentive to prevent disaster. An economic environment where a company is only considered viable if it's constantly expanding and increasing its production can't be expected to pump its own brakes over something as trivial as pending global catastrophe. Instead, market logic dictates that rather than take the financial hit that comes with cutting profits, it's more reasonable to find a way to make money off the boiling ocean. Nothing illustrates this phenomenon better than the burgeoning climate-change investment industry. According to Bloomberg, investors are looking to make money off of everything from revamped food production to hotels for people fleeing increasingly hurricane-ravaged areas. A top JP Morgan Asset investment strategist advised clients that sea-level rise was so inevitable that there was likely a lot of opportunity for investing in sea-wall construction.

(tags: capitalism environment politics future climate-change doom)

Irish agricultural CO2 emissions actually INCREASED by 2.9% last year

Irish EPA: agriculture greenhouse gas emissions increased by 2.9% in 2017 “The most significant drivers are higher dairy cow numbers (+3.1%) which reflects national plans to expand milk production”

(tags: epa ireland co2 greenhouse-gases emissions green farming agriculture)

"Bring home the bacon" debate is a case study in how stupid we've all become

Well said, that Carl Kinsella.

How are we supposed to focus on the things that matter when media organisations refuse to? Ivan Yates and Matt Cooper ran a segment on whether or not we've reached political correctness. Niall Boylan had a discussion about whether or not it's okay to ban phrases that nobody really wants to ban [as PETA has suggested]. As a matter of pure coincidence, other reports from outlets like the Daily Mail and Today FM reported that the HSE was going to ban its employees from using phrases like "love," "pet" or "dear." That wasn't true either. Again, it was one line in a huge report that asked the question as to whether or not it was the best course of action to use such terms. It wasn't mentioned again. Unlike phrases about meat, this actually matters. This is the kind of thing that could actually change how people are treated in hospital. But again, it wasn't true. The public – which has lost its capacity to verify what is true and what is not — was being lied to by the media. That's how it works now.

(tags: media clickbait fake-news news today-fm daily-mail hse niall-boylan peta controversy political-correctness ivan-yates matt-cooper)

Rudy Giuliani doesn't understand how links work

As waxy noted: 'this might be funny if he wasn't Trump's cybersecurity advisor'.

Twitter allowed someone to invade my text with a disgusting anti-President message. The same thing-period no space-occurred later and it didn’t happen. Don’t tell me they are not committed cardcarrying anti-Trumpers. Time Magazine also may fit that description. FAIRNESS PLEASE

(tags: edge-cases bugs twitter regexps regular-expressions links urls us-politics trump rudy-giuliani security funny)

3D models by DH_Age Sheela-na-Gig3D Project (@DH_Age) - Sketchfab

These are fantastic -- 3D scans of Sheela-na-Gig carvings around Ireland from 3D Sheela, an Irish based research initiative 'focusing on the digital documentation and analysis of Ireland's Sheela-na-Gig catalogue' (NSFW)

(tags: 3d sheela-na-gigs history carving nsfw models photogrammetry)

Chester Beatty Digital Collections

'Explore online access to our remarkable treasures, through this searchable database of digitised artworks and manuscripts', from the Chester Beatty museum's collection. Licensing isn't fully open though -- 'Images and PDF's are provided for personal research and scholarship.'

(tags: chester-beatty museums history archaeology artifacts art manuscripts)

Convert an RSS feed to ActivityPub

for autoposting to Mastodon or similar. Via Nelson

(tags: via:nelson mastodon rss blogging scripts)

A Serverless Journey: AWS Lambda Under the Hood (SRV409-R1) - AWS re:…

Marc Brooker on Lambda internals

(tags: lambda aws marc-brooker ops slides)

Nice heatmap visualisation of mortality ratios between men and women

'Age and period specific mortality ratios of females and males (a "Lexis-surface" in demographers parlance)' (via apenwarr)

(tags: via:apenwarr mortality heatmaps dataviz visualisation colouring lexis-surfaces)

House Democrats Sent A New Letter To Jeff Bezos About Amazon’s Facial Recognition Tool

The House Democrats’ questions focus largely on possible embedded bias in Amazon Rekognition, including how the tool’s accuracy breaks down by race, gender, ethnicity, and age. Also of particular concern is whether Amazon will build privacy protections into its facial recognition system and how it will ensure it is not abused for secret government surveillance. [....] Meanwhile, Jeff Bezos has yet to address mounting criticism of Amazon’s Rekognition technology by Amazon employees, shareholders, and civil rights groups. In November, Amazon executives defended the company’s controversial facial recognition technology at an all-hands staff meeting after employees raised civil rights concerns about the tech’s potential misuse. “It’s hard to trust that harm and abuse can be prevented if it is only post-mortem and through the Terms of Service,” an Amazon employee who requested anonymity told BuzzFeed News at the time.

(tags: rekognition aws privacy data-protection surveillance amazon us-politics civil-rights)

Google Tried to Patent My Work After a Job Interview

I looked up the patent application and luckily, this time the patent application was still being reviewed by the patent examiner.  It had not issued! The provisional was filed August 29, 2014, months after my first interview and visit back in March 2014.  Two of the inventors listed were the same people who had interviewed me. 

(tags: patents software-patents google dirty-tricks interviewing ip mit medialab paper jie-qi)

Crowdfunding Backer Patented My Project

I had thought thought all along that if we published everything openly, it wouldn’t be possible for someone else to patent stuff that’s already all over the web.  But I was wrong. Despite tons of prior art out on the web, in academic research papers and even for sale that are LED stickers, the patent examiner missed it and deemed the LED sticker patent “new” and “non-obvious.”   How could that happen?  The sad truth is that patents are approved all that time that probably shouldn’t be. [...] Can’t you invalidate the patent? Unfortunately, once a patent gets issued things get much more complicated and expensive. The cheapest option would be for us to go to the USPTO with our prior art list and invalidate the patent though a process called an inter partes review (IPR).  But such a process typically costs between $300,000 and $600,000 to file due to legal fees. In fact, it’s much more than the cost of getting the patent in the first place! So for now, we’ve decided not to go down this route.

(tags: patents led-stickers uspto prior-art invention crowdfunding)

PRDD - Performance-Review Driven Development

'If the way to get promoted is to launch a shiny new product, then your most senior people will be the best at finding shiny new products to launch, even if that's not the right technical decision to make.' (from a newsy thread about Twitter's latest messaging system switch)

(tags: newsy messaging infrastructure twitter kafka pubsub ops architecture prdd performance-reviews)

Presto Performance for Ad Hoc Workloads on AWS Instance Types

good benchmark/review of instance types from Qubole

(tags: qubole presto performance benchmarks ops aws instances ec2)

Party Parrot as a Service

Enter an image URL and it'll generate an animated GIF of the party parrot version

(tags: party-parrot gifs funny slack emojis)

event-stream vulnerability explained - Zach Schneider

This was an incredibly clever attack, very reminiscent of this blog post from January about how a similar attack might work. The attacker covered their tracks well — the code and commit log on GitHub all tell an innocuous and fairly common story (a new maintainer joins a project, adds a feature, and then tweaks the implementation of their feature a bit). Other than the warning signs about flatmap-stream (new package, no contributors or download activity), the attack was virtually undetectable. And indeed, it wasn’t discovered for over two months — it was only found because the attacker made a tiny mistake and used the deprecated crypto.createDecipher rather than crypto.createDecipheriv, which raised a suspicious deprecation warning in another library that consumes event-stream. Unfortunately, this genre of attack isn’t going away anytime soon. JavaScript is the most popular language right now and it’s not really close, meaning it will continue to be an attractive target for hackers. JavaScript also has relatively few standard-library convenience features compared to other languages, which encourages developers to import them from npm packages instead — this, along with other cultural factors, means that JavaScript projects tend to have massive dependency trees.

(tags: npm malware bitcoin security javascript event-stream flatmap-stream hacks)

Leeroy Jenkins was taking the piss

I had no idea!

"We didn’t think anyone would believe it was real, we thought it was so obviously satire."

(tags: leeroy-jenkins warcraft gaming memes satire videos at-least-i-have-chicken)

cryptoshredding

A way to handle immutable blockchains and GDPR: "have an encryption key for each user (stored outside of this ledger) and encrypt all PII with that key. Throw away the key if the user wants you to delete their data."

(tags: pii gdpr privacy data-protection crypto cryptoshredding deletion coding)

flatmap-stream NPM package backdoor incident

Good twitter thread with background on the incident. 2,000,000 downloads per week, used by many other core libs. It appears the attacker persuaded the (overloaded) legit maintainer to hand over ownership then backdoored the package in order to attack copay-dash, a cryptocurrency wallet app.

(tags: cryptocurrency npm packages open-source twitter flatmap-stream packaging security backdoors)

Antoin O Lachtnain on Twitter regarding GDPR and DNA kits

'It provides legal protections. But the problem is that those protections may not be practically effective. The other problem is the effect that you, as a consenting adult may have on other people who are related to you but who haven't consented.' -- useful thread

(tags: gdpr privacy data-protection dna)

boundary-layer

Declarative Airflow Workflows in YAML, from Etsy

(tags: airflow python batch cron etl)

Linkerd 2.0

The 2.0 release of Linkerd brings two very significant changes. First, we’ve completely rewritten Linkerd to be orders of magnitude faster and smaller than Linkerd 1.x. Linkerd 2.0’s data plane is comprised of ultralight Rust proxies which consume around 10mb of RSS and have a p99 latency of <1ms. Linkerd’s minimalist control plane (written in Go) is similarly designed for speed and low resource footprint. Second, with the 2.0 release, Linkerd moves beyond the service mesh model to be something not more, but less: Linkerd 2.0 at its core is a service sidecar, running on a single service without requiring cluster-wide installation. This means that if you’re a developer or service owner who doesn’t have access to the whole Kubernetes cluster, you can run Linkerd on your service and get: Instant Grafana dashboards of your service’s success rates, latencies, and throughput; A topology graph of incoming and outgoing dependencies; A live view of requests being made to your service; Improved, latency-aware load balancing; … and much more.

(tags: linkerd mesh networking services architecture sidecars)

MuMufication

the act of having a small portion of your cremated remains fired in a Brickof Mu. MuMufied is what you will be after the act of MuMufication has been carried out. What you get in the here and now is a Brick of Mu and a signed and stamped Certificate of MuMufication. What you get after you die is 23 grams of yours cremated remains fired in your Brick of Mu, which will then be laid to rest on The People’s Pyramid come the following Toxteth Day of the Dead on 23rd of November.

(tags: klf mu mumufication 23 eris discordianism religion death toxteth liverpool cremation pyramids)

Introducing Predictive Scaling for Amazon EC2 in AWS Auto Scaling

Predictive Scaling predicts future traffic based on daily and weekly trends, including regularly-occurring spikes, and provisions the right number of EC2 instances in advance of anticipated changes. Provisioning the capacity just in time for an impending load change makes Auto Scaling faster than ever before. Predictive Scaling’s machine learning algorithms detect changes in daily and weekly patterns, automatically adjusting their forecasts. This removes the need for manual adjustment of Auto Scaling parameters over time, making Auto Scaling simpler to configure and consume. Auto Scaling enhanced with Predictive Scaling delivers faster, simpler, and more accurate capacity provisioning to our customers.

(tags: aws amazon scaling autoscaling predictive-scaling ml ec2 asg)

Awful AI

Artificial intelligence in its current state is unfair, easily susceptible to attacks and notoriously difficult to control. Nevertheless, more and more concerning uses of AI technology are appearing in the wild. This list aims to track all of them. We hope that Awful AI can be a platform to spur discussion for the development of possible contestational technology (to fight back!).

(tags: ai algorithms ethics technology machine-learning)

Some notes about HTTP/3

Robert Graham from ErrataSec on QUIC aka HTTP/3: 'Google (pbuh) has both the most popular web browser (Chrome) and the two most popular websites (#1 Google.com #2 Youtube.com). Therefore, they are in control of future web protocol development.' Faster connection setup and latency; better bandwidth negotiation when using multiplexing; user-mode stacks by building on UDP and using recvmmsg(); and better mobile support for roaming IPs.

(tags: google http3 quic protocols ip)

Traditional Chinese medicine origins: Mao invented it but didn’t believe in it

Mikulski and the rest of the Senate may be surprised to learn that they were repeating 60-year-old justifications of Chinese medicine put forward by Chairman Mao. Unlike Mikulski, however, Mao was under no illusion that Chinese medicine—a key component of naturopathic education—actually worked. In The Private Life of Chairman Mao, Li Zhisui, one of Mao’s personal physicians, recounts a conversation they had on the subject. Trained as an M.D. in Western medicine, Li admitted to being baffled by ancient Chinese medical books, especially their theories relating to the five elements. It turns out his employer also found them implausible. 

(tags: medicine tcm mao history china health naturopathy)

Java's ByteBuffer native memory "leak"

Well this is suboptimal:

The Java NIO APIs use ByteBuffers as the source and destination of I/O calls, and come in two flavours. Heap ByteBuffers wrap a byte[] array, allocated in the garbage collected Java heap. Direct ByteBuffers wrap memory allocated outside the Java heap using malloc. Only "native" memory can be passed to operating system calls, so it won't be moved by the garbage collector. This means that when you use a heap ByteBuffer for I/O, it is copied into a temporary direct ByteBuffer. The JDK caches one temporary buffer per thread, without any memory limits. As a result, if you call I/O methods with large heap ByteBuffers from multiple threads, your process can use a huge amount of additional native memory, which looks like a native memory leak. This can cause your process to unexpectedly run into memory limits and get killed.

(tags: jvm performance java memory leaks bytebuffers netty threads coding bugs)

The Time Our Provider Screwed Us

Good talk (with transcript) from Paul Biggar about what happened when CircleCI had a massive security incident, and how Jesse Robbins helped them do incident response correctly. 'On the left, Jesse pointed out that we needed an incident commander. That’s me, Paul. And this is very good, because I was a big proponent, I think lots of were around the 2013 mark, of flat organizational structures, and so I hadn’t really got a handle of this whole being in charge thing. The fact that someone else came in and said, “No, no, no, you are in charge”: extremely useful. And he also laid out the order of our priorities. Number one priority; safety of customers. Number two priority: communicate with customers. Number three priority: recovery of service. I think a reasonable person could have put those in a different order, especially under the pressure and time constraints of the potential company-ending situation. So I was very happy to have those in order. If this is ever going to happen to you, I’d memorize them, maybe put it on an index card in your pocket, in case this ever happens. The last thing he said is to make sure that we log everything, that we go slow, and that we code review and communicate. His point there is that if we’re going to bring our site back up, if we’re going to do all the things that we need to do in order to save our business and do the right thing for our customers and all that, we can’t be making quick, bad decisions. You can’t just upload whatever code is on your computer now, because I have to do this now, I have to fix it. So we set up a Slack channel … This was pre-Slack; it was a HipChat channel, where all of our communications went. Every single communication that we had about this went in that chatroom. Which came in extremely useful the next day, when I had to write a blog post that detailed exactly what had happened and all the steps that we did to fix it and remediate this, and I had an exact time stamps of all the things that had happened.'

(tags: incidents incident-response paul-biggar circleci security communication outages)

Deep learning can "discover" new knowledge from scans/images

Amazing paper:

Here, we show that deep learning can extract new knowledge from retinal fundus images. Using deep-learning models trained on data from 284,335 patients and validated on two independent datasets of 12,026 and 999 patients, we predicted cardiovascular risk factors not previously thought to be present or quantifiable in retinal images, such as age (mean absolute error within 3.26 years), gender (area under the receiver operating characteristic curve (AUC)?=?0.97), smoking status (AUC?=?0.71), systolic blood pressure (mean absolute error within 11.23?mmHg) and major adverse cardiac events (AUC?=?0.70). We also show that the trained deep-learning models used anatomical features, such as the optic disc or blood vessels, to generate each prediction.

(tags: deep-learning data analysis ml machine-learning health medicine papers)

OpsMop

'a next-generation, no-compromise automation system'.

Uses: Web-scale configuration management of all Linux/Unix systems; Application deployment; Immutable systems build definition; Maintaining stateful services such as database and messaging platforms; Automating one-off tasks & processes; Deployment and management of the undercloud. Features: Python 3 DSL; Declarative resource model with imperative capabilities; Type / Provider plugin seperation; Implicit ordering (with handler notification); Formalized “Plan” vs “Apply” evaluation stages; Early validation prior to runtime; Programatically scoped variables; Strong object-orientation

(tags: opsmop ops configuration-management deployment build)

The JVM in Docker 2018

Later JDK versions have made it far easier to run a JVM application in a Linux container. The memory support means that if you relied on JVM ergonomics before than you can do the same inside a container where as previously you had to override all memory related settings. The CPU support for containers needs to be carefully evaluated for your application and environment. If you’ve previously set low cpu_shares in environments like Kubernetes to increase utilisation while relying on using up unused cycles then you might get a shock.

(tags: jvm docker kubernetes linux containers ops)

This Week’s Giza Pyramid Discovery Was Already Built Into Assassin’s Creed Origins

As a kid, I was Egypt-obsessed -- this game is hitting the spot

(tags: assassins-creed egypt history pyramids architecture)

Tuning Spark Back Pressure by Simulation

Interesting, Spark uses a PID controller algorithm to manage backpressure:

Spark back pressure, which can be enabled by setting spark.streaming.backpressure.enabled=true, will dynamically resize batches so as to avoid queue build up. It is implemented using a Proportional Integral Derivative (PID) algorithm. This algorithm has some interesting properties, including the lack of guarantee of a stable fixed point. This can manifest itself not just in transient overshoot, but in a batch size oscillating around a (potentially optimal) constant throughput. The overshoot incurs latency; the undershoot costs throughput. Catastrophic overshoot leading to OOM is possible in degenerate circumstances (you need to choose the parameters quite deviously to cause this to happen). Having witnessed undershoot and slow recovery in production streaming jobs, I decided to investigate further by testing the algorithm with a simulator.

(tags: backpressure streaming queueing pid-controllers algorithms congestion-control)

New – EC2 Auto Scaling Groups With Multiple Instance Types & Purchase Options | AWS News Blog

Basically getting EC2 Fleet's featureset into ASGs, good news

(tags: ec2 fleet asg ops architecture cost-control)

SpamAssassin is back [LWN.net]

The SpamAssassin 3.4.2 release was the first from that project in well over three years. At the 2018 Open Source Summit Europe, Giovanni Bechis talked about that release and those that will be coming in the near future. It would seem that, after an extended period of quiet, the SpamAssassin project is back and has rededicated itself to the task of keeping junk out of our inboxes.

(tags: spamassassin open-source oss anti-spam)

Google 'betrays patient trust' with DeepMind Health move | Technology | The Guardian

Now that Streams is a Google product itself, that promise appears to have been broken, says privacy researcher Julia Powles: “Making this about semantics is a sleight of hand. DeepMind said it would never connect Streams with Google. The whole Streams app is now a Google product. That is an atrocious breach of trust, for an already beleaguered product.” A DeepMind spokesperson emphasised that the core of the promise remains intact: “All patient data remains under our partners’ strict control, and all decisions about its use lie with them. This data remains subject to strict audit and access controls and its processing remains subject to both our contracts and data protection legislation. The move to Google does not affect this.”

(tags: google deepmind health nhs data-protection privacy healthcare)

Jeff Bezos is wrong, tech workers are not bullies

I decided to leave my job as a staff engineer at Google because of Project Maven, and because I believe that the artificial intelligence ethical guidelines they published afterwards were not strict enough: they allowed surveillance within “internationally accepted norms”. I am now joining forces with current and former Google employees who also opposed Maven and the Dragonfly search engine. We do not wish to be complicit in human rights violations and we believe that workers, and the public, deserve a voice. We support employees at Amazon, Microsoft, Salesforce, McKinsey and Deloitte who have similarly stood up to their employers. We also have a right to not contribute to killing. Most workers at Google or Amazon did not join those companies to work on military applications. Both companies are international employers with engineering offices across the world, and many of their workers are neither US citizens nor residents. I worked as an engineer in Google's European headquarters in Dublin, Ireland. To me, the US military is not our military (as Google Cloud chief executive Diane Greene referred to it in a blog post), nor is it a force we should automatically support as a matter of patriotism. As an engineer, I believe it is my responsibility to speak up for human rights and accountable decision making. As an industry, we in technology cannot compromise our principles or allow ourselves to be bullied by billionaires who stand to be enriched by our silence.

(tags: project-maven dragonfly google amazon surveillance us-politics politics ai silicon-valley ethics work life)

HTTP-over-QUIC to be renamed HTTP/3

Decent newsy comment thread about HTTP/3, QUIC, and how the modern internet treats IP protocols

(tags: ip protocols http http3 quic networking internet newsy)

Brits getting into the online-disinfo game

Disinformation and deception have been a part of warfare for thousands of years, but across the world, something new was starting to happen. Information has long been used to support combat operations, but now combat was seen to taking place primarily, sometimes exclusively, through it. From being a tool of warfare, each military began to realise that the struggle with, over and through information was what war itself actually was about. And it wasn’t confined to Russia, China or anyone else. A global informational struggle has broken out. Dozens of countries are already doing it. And these are just the campaigns that we know about.

(tags: disinfo fake-news uk british-army military memes infowar)

Some notes on running new software in production

This is really good -- how to approach new infrastructure/software dependencies in production with reliability and uptime in mind. (via Tony Finch)

(tags: reliability uptime slas kubernetes envoy outages runbooks ops)

Specification gaming examples in AI

A good list of examples where machine learning systems "figure out" how to cheat their fitness function, e.g.:

'Creatures bred for speed grow really tall and generate high velocities by falling over'

(tags: ai funny humor spreadsheets machine-learning ml fitness-functions)

What if the Placebo Effect Isn’t a Trick? - The New York Times

It is not possible to assay levels of COMT directly in a living brain, but there is a snippet of the genome called rs4680 that governs the production of the enzyme, and that varies from one person to another: One variant predicts low levels of COMT, while another predicts high levels. When Hall analyzed the I.B.S. patients’ DNA, she found a distinct trend. Those with the high-COMT variant had the weakest placebo responses, and those with the opposite variant had the strongest. These effects were compounded by the amount of interaction each patient got: For instance, low-COMT, high-interaction patients fared best of all, but the low-COMT subjects who were placed in the no-treatment group did worse than the other genotypes in that group. They were, in other words, more sensitive to the impact of the relationship with the healer. The discovery of this genetic correlation to placebo response set Hall off on a continuing effort to identify the biochemical ensemble she calls the placebome — the term reflecting her belief that it will one day take its place among the other important “-omes” of medical science, from the genome to the microbiome. The rs4680 gene snippet is one of a group that governs the production of COMT, and COMT is one of a number of enzymes that determine levels of catecholamines, a group of brain chemicals that includes dopamine and epinephrine. (Low COMT tends to mean higher levels of dopamine, and vice versa.) Hall points out that the catecholamines are associated with stress, as well as with reward and good feeling, which bolsters the possibility that the placebome plays an important role in illness and health, especially in the chronic, stress-related conditions that are most susceptible to placebo effects.

(tags: placebo comt health healthcare medicine enzymes brain)

Deadlines, lies and videotape: The tale of a gRPC bug

HostedGraphite decided to use gRPC as an internal inter-service protocol and ran into a basic protocol bug -- it does not default to using an application-level keepalive on the TCP channel so can block indefinitely if sending-side buffers fill up. Always use application-level keepalives and don't trust TCP

(tags: tcp protocols keepalive grpc rpc architecture networking)

Sci-Fi Writer Greg Egan and 4chan anon Math Whiz Advance Permutation Problem | Quanta Magazine

On September 16, 2011, an anime fan posted a math question to the online bulletin board 4chan about the cult classic television series 'The Melancholy of Haruhi Suzumiya'. Season one of the show, which involves time travel, had originally aired in non-chronological order, and a re-broadcast and a DVD version had each further rearranged the episodes. Fans were arguing online about the best order to watch the episodes, and the 4chan poster wondered: If viewers wanted to see the series in every possible order, what is the shortest list of episodes they’d have to watch? In less than an hour, an anonymous person offered an answer — not a complete solution, but a lower bound on the number of episodes required. The argument, which covered series with any number of episodes, showed that for the 14-episode first season of Haruhi, viewers would have to watch at least 93,884,313,611 episodes to see all possible orderings. “Please look over [the proof] for any loopholes I might have missed,” the anonymous poster wrote. The proof slipped under the radar of the mathematics community for seven years — apparently only one professional mathematician spotted it at the time, and he didn’t check it carefully. But in a plot twist last month, the Australian science fiction novelist Greg Egan proved a new upper bound on the number of episodes required. Egan’s discovery renewed interest in the problem and drew attention to the lower bound posted anonymously in 2011. Both proofs are now being hailed as significant advances on a puzzle mathematicians have been studying for at least 25 years.

(tags: mathematics internet math greg-egan anime bizarre 4chan superpermutation permutation proofs)

How do you populate your development databases?

Lots of comments pro/anti copying from production

(tags: database data testing system-tests dev)

Welcome To The Dystopia: People Are Arguing Whether This Trump Press Conference Video Is Doctored Or Not

To sum it up: A historically unreliable narrator who works for a conspiracy website tweets out a video in order to show alleged bad behavior on the part of a journalist. The clip goes viral. The White House picks up and disseminates that video and uses it as proof to ban the journalist from reporting at the White House. Outraged journalists decry the White House's use of a video taken from a historically unreliable narrator. Then, users attempt to debunk the video as "actual fake news." Others, unclear if the video is fake, urge caution, suggesting the media may be jumping the gun. An argument breaks out over the intricate technical details of doctoring a clip. The entire ordeal is a near perfect example of a scenario disinformation experts have predicted and warned of, where the very threat of video manipulation can lead to a blurring of reality. "These technological underpinnings [of AI and photoshop, and editing programs lead] to the increasing erosion of trust,” computational propaganda researcher Renee DiResta told BuzzFeed News in early 2018. “It makes it possible to cast aspersions on whether videos — or advocacy for that matter — are real.”

(tags: twitter disinfo disinformation buzzfeed video reality fake-news photoshop)

wifiscan.py

A neat kragen hack:

Report wireless signal strength using audio synthesis. This quick kludge is useful for figuring out where the dead spots in your wireless network coverage are. They’re where the pitch goes high and maybe you start getting clicks from audio buffer underruns. Example audio output can be found at . This is based on a brilliant hack by Seth Schoen, in which he generated a tone controlled by the signal strength indication from his wireless card in order to get a sort of 21st-century software Theremin. This adds some envelope modulation so that it’s useful as a network diagnostic tool rather than a musical instrument. It depends on PulseAudio and the interface to the usual set of Linux commands: pacat, ping, and iwconfig. (I should really just read /proc/net/wireless instead of depending on iwconfig.)

(tags: kragen wifi scanning audio sound signal linux)

Underground Utility Colour Codes Explained

Underground utility colour codes are used to differentiate and identify underground utilities to protect it from damage during excavation. There are different types of utilities and in order to tell them apart coloured lines, flags or sometimes both are used. They help mark the location and indicate the type of utility that is buried underground.

(tags: utilities power underground excavation digging repair urban)

Meet the Irish master codebreaker you never heard of

In 1943, one of Nazi Germany’s most notorious communication codes was broken by a mild-mannered librarian and family man from Abbeyfeale in west Limerick, Richard Hayes. His day job was director of the National Library of Ireland, but during the Emergency, he secretly led a covert team of cryptanalysts, working feverishly on the infamous “Görtz Cipher” – a fiendish Nazi code that had stumped many of the greatest code-breaking minds at Bletchley Park, the centre of British wartime cryptography. Astonishingly, the feat was accomplished not in one of the huts at Bletchley Park, but in a now derelict building colloquially known as “The Red House” on Montpellier Hill in Dublin. If this wasn’t an amazing enough feat, Hayes also broke a complex microdot enciphering system which had baffled the American OSS, as well an enciphering system used by the dreaded Sicherheitsdienst (or SD), the intelligence section of the SS. The breaking of the latter code was instrumental in the Allied victory at the Battle of the Bulge. Few individuals can be said to have altered the course of the second World War on their own, but Hayes was one such individual – and he did it not with a gun, but with a detailed knowledge of the complex algorithms and mathematical permutations involved in the art of cryptography.

(tags: via:colmmacc crypto codes hermann-gortz wwii dublin ireland richard-hayes cryptanalysis)

Physical spam using cheap Bluetooth beacons spotted in the wild

Buzz Andersen on Twitter: "Crazy thing: my friends who run a high traffic coffee shop just found this thing under their bar. It turns out to be a super cheap Chinese Bluetooth LE beacon.… https://t.co/pecNBCR86u"

(tags: bluetooth spam via:twitter alibaba eddystone ads)

Productionproofing EKS

'We recently migrated SaleMove infrastructure from self-managed Kubernetes clusters running on AWS to using Amazon Elastic Container Service for Kubernetes (EKS). There were many surprises along the way to getting our EKS setup ready for production. This post covers some of these gotchas (others may already be fixed or are not likely to be relevant for a larger crowd) and is meant to be used as a reference when thinking of running EKS in production.'

(tags: eks aws docker kubernetes k8s ops prod)

Neoliberalism has conned us into fighting climate change as individuals | Martin Lukacs | Environment | The Guardian

These pervasive exhortations to individual action — in corporate ads, school textbooks, and the campaigns of mainstream environmental groups, especially in the west — seem as natural as the air we breathe. But we could hardly be worse-served. While we busy ourselves greening our personal lives, fossil fuel corporations are rendering these efforts irrelevant. The breakdown of carbon emissions since 1988? A hundred companies alone are responsible for an astonishing 71%. You tinker with those pens or that panel; they go on torching the planet. The freedom of these corporations to pollute – and the fixation on a feeble lifestyle response – is no accident. It is the result of an ideological war, waged over the last 40 years, against the possibility of collective action. Devastatingly successful, it is not too late to reverse it. The political project of neoliberalism, brought to ascendence by Thatcher and Reagan, has pursued two principal objectives. The first has been to dismantle any barriers to the exercise of unaccountable private power. The second had been to erect them to the exercise of any democratic public will. [...] At the very moment when climate change demands an unprecedented collective public response, neoliberal ideology stands in the way. Which is why, if we want to bring down emissions fast, we will need to overcome all of its free-market mantras.

(tags: politics environment neoliberalism future climate-change green)

Block Advertising on your Network with Pi-hole and Raspberry Pi

A good walkthrough of the Pi-Hole network-wide adblocker install and operation

(tags: pi-hole ads blocking ops home raspberry-pi)

October 21 post-incident analysis | The GitHub Blog

A network outage caused a split-brain scenario, and their failover system allowed writes to occur in both regional databases. Once the outage was repaired it was impossible to reconcile writes in an automated fashion as a result. Embarrassingly, this exact scenario was called out in their previous blog post about their Raft-based failover system at https://githubengineering.com/mysql-high-availability-at-github/ -- "In a data center isolation scenario, and assuming a master is in the isolated DC, apps in that DC are still able to write to the master. This may result in state inconsistency once network is brought back up. We are working to mitigate this split-brain by implementing a reliable STONITH from within the very isolated DC. As before, some time will pass before bringing down the master, and there could be a short period of split-brain. The operational cost of avoiding split-brains altogether is very high." Failover is hard.

(tags: github fail outages failover replication consensus ops)

LGTM

Analyses open source code repos on github, running static code analysis to find "zero-days and other critical bugs".

(tags: security code-review static-code-analysis coding lgtm github)

cool twitter image compression hack

D?v?d ?uc??n?n on Twitter:

Assuming this all works out, the image in this tweet is also a valid ZIP archive, containing a multipart RAR archive, containing the complete works of Shakespeare. This technique also survives twitter's thumbnailer :P

(tags: via:johnke cool compression image jpeg twitter)

A Dark Consensus About Screens and Kids Begins to Emerge in Silicon Valley - The New York Times

John Lilly, a Silicon Valley-based venture capitalist with Greylock Partners and the former C.E.O. of Mozilla, said he tries to help his 13-year-old son understand that he is being manipulated by those who built the technology. “I try to tell him somebody wrote code to make you feel this way — I’m trying to help him understand how things are made, the values that are going into things and what people are doing to create that feeling,” Mr. Lilly said. “And he’s like, ‘I just want to spend my 20 bucks to get my Fortnite skins.’”

(tags: kids technology education parenting screentime apps tech phones)

Your Kid’s Apps Are Crammed With Ads - The New York Times

In apps marketed for children 5 and under in the Google Play store, there were pop-up ads with disturbing imagery. There were ads that no child could reasonably be expected to close out of, and which, when triggered, would send a player into more ads. Dancing treasure chests would give young players points for watching video ads, potentially endlessly. The vast majority of ads were not marked at all. Characters in children’s games gently pressured the kids to make purchases, a practice known as host-selling, banned in children’s TV programs in 1974 by the Federal Trade Commission. At other times an onscreen character would cry if the child did not buy something. “The first word that comes to mind is furious,” said Dr. Radesky, an assistant professor of developmental behavioral pediatrics at the University of Michigan Medical School. “I’m a researcher. I want to stay objective. We started this study really just trying to look at distraction. My frustrated response is about all the surprising, potentially deceptive stuff we found.”

(tags: children kids ads advertising apps android google ftc games iap)

Epic twitter thread from @colmmacc explaining why client certs and mutual-auth TLS are TERRIBAD

Ok. tweet thread time! Too long ago I promised to write a screed explaining how much I hated mutual-auth TLS and why. I got distracted, and I wasn't happy with the writing, so here it is in tweet thread form instead! But basically: Client certs and Mutual-Auth TLS is TERRIBAD. When I say TERRIBAD, I mean that unless you've got the resources of a big security dept and folks who comb threat models for a living, using clients certs and mutual auth probably materially lessens your security. That's NUTS!

(tags: terribad rants twitter threads tls ssl authentication mtls security)

Google Home (in)Security

0 authentication on some setup APIs, including 'delete the current wifi network config' -- pretty major lack of security

(tags: auth security fail google google-home)

ChromeCast HTTP APIs

similar to the Google Home hackable APIs, it seems Chromecasts are easily hacked/scripted with no auth

(tags: auth http apis security chromecast google)

"The first AI portrait in Christie’s" was mostly output from someone else's open-source code

The print was created by Obvious, a trio of 25-year-old French students whose goal is to “explain and democratize” AI through art. Over the past year, they’ve made a series of portraits depicting members of the fictional Belamy family, amplifying their work through attention-grabbing press releases. But insiders say the code used to generate these prints is mostly the work of another artist and programmer: 19-year-old Robbie Barrat, a recent high school graduate who shared his algorithms online via an open-source license. The members of Obvious don’t deny that they borrowed substantially from Barrat’s code, but until recently, they didn’t publicize that fact either. This has created unease for some members of the AI art community, which is open and collaborative and taking its first steps into mainstream attention.[...] Jason Bailey, a digital art blogger who runs the site Artnome, says that what Obvious has done is far from unusual. “It’s almost weekly in digital art that someone takes some open code and tweaks it and sells it,” he tells The Verge. But the prominence of this auction and the fact that Obvious, not Barrat, has received the attendant prestige and attention does complicate the matter. “There’s a lot of stuff you can do that’s legal, but that makes you sort of a jerk,” adds Bailey. “If I was Robbie, I’d be pretty miffed, and Obvious said they owe him a great deal of credit.” Barrat says he holds no grudges at all and is mostly annoyed that the auction might give outsiders the wrong impression about AI art. “I’m more concerned about the fact that actual artists using AI are being deprived of the spotlight,” he says. “It’s a very bad first impression for the field to have.”

(tags: ai art graphics history open-source ownership copyright obvious robbie-barrat digital)

pusher/k8s-spot-rescheduler

'Tries to move K8s Pods from on-demand to spot instances':

K8s Spot rescheduler is a tool that tries to reduce load on a set of Kubernetes nodes. It was designed with the purpose of moving Pods scheduled on AWS on-demand instances to AWS spot instances to allow the on-demand instances to be safely scaled down (By the Cluster Autoscaler). In reality the rescheduler can be used to remove load from any group of nodes onto a different group of nodes. They just need to be labelled appropriately. For example, it could also be used to allow controller nodes to take up slack while new nodes are being scaled up, and then rescheduling those pods when the new capacity becomes available, thus reducing the load on the controllers once again.

(tags: k8s kubernetes aws scaling spot-instances ops)

LiV Pi

Air quality sensor board for Raspberry Pis, with a good quality self-calibrating NDIR CO2 sensor

(tags: co2 air quality monitoring metrics health home raspberry-pi hardware to-get)

Motorola and iFixit—A Match Made in Mobile

This is awesome.

Motorola is setting an example for major manufacturers to embrace a more open attitude towards repair. If you’re a Motorola customer, you can now either send in your broken device directly to Motorola for repair—or you can fix it yourself with the highest quality parts and tools, plus a free step-by-step guide, all included in our official Motorola OEM Fix Kits.

(tags: motorola repair ifixit hardware mobile)

Excellent Twitter thread from colmmacc on how s2n avoids protocol-state errors

using a linearized set of state transitions, and Cryptol and SAW to perform verification of the TLS state machine

(tags: cryptol saw formal-verification twitter threads colmmacc security s2n)

Remapping the silly double-S key on a Macbook running 10.12 Sierra

the latest hacky workaround for this stupid Apple keyboard layout bullshit

(tags: apple keyboards europe fail hidutil macosx)

Republicans Are Adopting the Proud Boys

Extreme-right factions of the GOP happily taking on a brownshirt paramilitary gang. This is scary shit

(tags: fascism america politics proud-boys paramilitary republicans gop)

LOLWUT: a piece of art inside a database command -

Redis now includes a reference to "Schotter", by Georg Nees, one of the earliest pieces of computer art. Nice one antirez :)

(tags: antirez coding art georg-nees redis lolwut)

Protective Hand | Thorazos

great T-shirt -- adding to the shopping list for the next Threadless trip

(tags: threadless to-get tee-shirts thorazos)

Every fantasy map / Boing Boing

Every fantasy map, ever

(tags: maps funny fantasy sf)

Lawsuit: "pivoting to video" was a disaster led by Facebook's cooked viewing data / Boing Boing

The company decided to "obfuscate the fact that we screwed up the math" by quietly retiring the erroneous metrics and replacing them with corrected metrics under a new name. For instance, Average Duration of Video Viewed would be replaced with Average Watch Time. [...] In August 2016, Facebook began reaching out privately to select, large advertisers, telling them that Facebook had "recently discovered a discrepancy" in the video ad average view metrics. Facebook pushed that message even as personnel internally emphasized that "we didn't recently discover a discrepancy." (emphasis added).

(tags: facebook metrics video fail lawsuits content)

Bitcoin must die

If Bitcoin were to cease trading tomorrow, 0.5% of the world’s electricity demand would simply disappear. This is roughly equivalent to the output of ten coal-fired power plants, emitting 50 million tonnes of CO2 per year – which would cover one year’s worth of the carbon emission cuts required to limit temperature rises this century to 2C. It is not a solution by itself, but it would be a good year’s work. Bitcoin is made from ashes, and if ashes were legal tender, humanity would burn everything in sight and call it progress.

(tags: environment bitcoin ecology future earth cryptocurrencies pow electricity climate-change)

'We're doomed': Mayer Hillman on the climate reality no one else will dare mention | Environment | The Guardian

Well this is terrifying.

Can civilisation prolong its life until the end of this century? “It depends on what we are prepared to do.” He fears it will be a long time before we take proportionate action to stop climatic calamity. “Standing in the way is capitalism. Can you imagine the global airline industry being dismantled when hundreds of new runways are being built right now all over the world? It’s almost as if we’re deliberately attempting to defy nature. We’re doing the reverse of what we should be doing, with everybody’s silent acquiescence, and nobody’s batting an eyelid.”

(tags: climate capitalism environment future scary mayer-hillman)

Querying OpenStreetMap Buildings with AWS Athena – door2door Engineering

Well this is damn cool:

AWS not only made OpenStreetMap planet data available on S3, but it also made it query-able with Athena. Pretty cool, no? Now, in theory, you can just construct an SQL query, send it to Athena, and then do whatever you want with the results. No more: Updating OSM planet data yourself; it gets updated on AWS whenever OSM publishes it, once a week. Transforming the data into a query-able format; Athena handles that for you. Query/request frequency limits (it’s still AWS though, so other limitations might apply ???? ????) At door2door, we had a pretty straight-forward use-case for this: we needed to get buildings in specific regions based only on where they were, and transform those buildings into GeoJSON that we can attach our data to, and visualize on the front-end on top of our base map.

(tags: athena osm buildings aws geodata mapping maps door2door cool hacks)

Opinion | When Your Boss Is an Algorithm - The New York Times

I have learned that drivers at ride-hailing companies may have the freedom and flexibility of gig economy work, but they are still at the mercy of a boss — an algorithmic boss.  Data and algorithms are presented as objective, neutral, even benevolent: Algorithms gave us super-convenient food delivery services and personalized movie recommendations. But Uber and other ride-hailing apps have taken the way Silicon Valley uses algorithms and applied it to work, and that’s not always a good thing.

(tags: algorithms uber gig-economy work)

A Soyuz "ballistic re-entry" which subjected the crew to 21 g

At the time when the safety system initiated separation the spacecraft was already pointed downward toward Earth, which accelerated its descent significantly. Instead of the expected acceleration in such an emergency situation of 15 g (147 m/s²), the cosmonauts experienced up to 21.3 g (209 m/s²).[2] Despite very high overloading, the capsule's parachutes opened properly and slowed the craft to a successful landing after a flight of only 21 minutes.

(tags: spaceflight soyuz accidents history cosmonauts)

How To Survive The Coming Century

New Scientist article from 2009 with a rather terrifying map of the 4-degrees-warmer Earth

(tags: earth new-scientist climate fear)

IPCC 1.5 degrees target requires massive carbon dioxide removal technology efforts

The grimmest prognosis in the draft report is in the details of the effort it would take to actually limit warming to 1.5°C. Countries won’t just have to give up fossil fuels and stop emitting greenhouse gases; they’ll have to pull carbon dioxide straight out of the air. “All pathways that limit global warming to 1.5°C with limited or no overshoot project the use of carbon dioxide removal (CDR),” according to the report. And not just a little, but a lot, upward of 1,000 gigatons of carbon dioxide removed from the atmosphere by the end of the century. This will require machines that scrub carbon dioxide out of the air as well as biofuels coupled with carbon capture and sequestration. These tactics have their own energy demands and environmental drawbacks, and we may not be able to deploy them in time. “CDR deployment of several hundreds of [gigatons of CO2] is subject to multiple feasibility and sustainability constraints,” according to the IPCC report.

(tags: cdr co2 greenhouse-gases climate-change technology ipcc un)

The 100 best restaurants, cafes and places to eat in Ireland

the Irish Times latest foodie list, via Aoife McElwain

(tags: irish-times food cafes eating dublin foodie restaurants)

A Controversial Virus Study Shows Flaws in How Science Is Done - The Atlantic

Absent clearer guidelines, the burden falls on the scientific enterprise to self-regulate—and it isn’t set up to do that well. Academia is intensely competitive, and “the drivers are about getting grants and publications, and not necessarily about being responsible citizens,” says Filippa Lentzos from King’s College London, who studies biological threats. This means that scientists often keep their work to themselves for fear of getting scooped by their peers. Their plans only become widely known once they’ve already been enacted, and the results are ready to be presented or published. This lack of transparency creates an environment where people can almost unilaterally make decisions that could affect the entire world. Take the horsepox study [the main topic of this article]. Evans was a member of a World Health Organization committee that oversees smallpox research, but he only told his colleagues about the experiment after it was completed. He sought approval from biosafety officers at his university, and had discussions with Canadian federal agencies, but it’s unclear if they had enough ethical expertise to fully appreciate the significance of the experiment. “It’s hard not to feel like he opted for agencies that would follow the letter of the law without necessarily understanding what they were approving,” says Kelly Hills, a bioethicist at Rogue Bioethics. She also sees a sense of impulsive recklessness in the interviews that Evans gave earlier this year. Science reported that he did the experiment “in part to end the debate about whether recreating a poxvirus was feasible.” And he told NPR that “someone had to bite the bullet and do this.” To Hills, that sounds like I did it because I could do it. “We don’t accept those arguments from anyone above age 6,” she says.

(tags: the-atlantic science news smallpox horsepox diseases danger risk academia papers publish-or-perish bioethics ethics biology genetics)

Amazon scraps secret AI recruiting tool that showed bias against women | Reuters

Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry. […] Amazon’s system taught itself that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter.’

(tags: algorithmic-bias amazon hiring resumes bias feminism machine-learning ml)

Wild camping spots - Google My Maps

A GMaps version of the list of wild camping spots around Ireland

(tags: gmaps google mapping wild-camping camping ireland)

When It’s Too Late to Stop Fascism, According to Stefan Zweig | The New Yorker

The destruction of a symbolic edifice—a blaze that caused no loss of life—became the pretext for the government to begin terrorizing its own civilian population. That fateful conflagration took place less than thirty days after Hitler became Chancellor. The excruciating power of Zweig’s memoir lies in the pain of looking back and seeing that there was a small window in which it was possible to act, and then discovering how suddenly and irrevocably that window can be slammed shut.

(tags: fascism europe stefan-zweig politics history germany nazis)

What to Look for in Java 8 Code

Great advice from Trisha Gee on J8 features and how to code-review code which uses them

(tags: java8 java code-review coding)

Wild Camping Spots map Ireland & Northern Ireland

looks like a decent map (even if they can't use GMaps)

(tags: wild-camping camping campsites ireland tourism)

archive.org C64 software library

Pretty amazing that these can be credibly emulated in a browser now!

(tags: c=64 history commodore vice web internet-archive gaming games)

Kubernetes: The Surprisingly Affordable Platform for Personal Projects

At the beginning of the year I spent several months deep diving on Kubernetes for a project at work. As an all-inclusive, batteries-included technology for infrastructure management, Kubernetes solves many of the problems you're bound to run into at scale. However popular wisdom would suggest that Kubernetes is an overly complex piece of technology only really suitable for very large clusters of machines; that it carries a large operational burden and that therefore using it for anything less than dozens of machines is overkill. I think that's probably wrong. Kubernetes makes sense for small projects and you can have your own Kubernetes cluster today for as little as $5 a month.

(tags: via:fanf deployment howto kubernetes ops projects hacks clustering)

This is how cyber attackers stole £2.26m from Tesco Bank customers | ZDNet

What a shitshow.

Poor design of Tesco Bank debit cards played a significant role in creating security vulnerabilities that led to thousands of customers having their accounts emptied. One of these involved the PAN numbers -- the 16-digit card number sequence used to identify all debit cards. Tesco Bank inadvertently issued debit cards with sequential PAN numbers. This increased the likelihood that the attackers would find the next PAN number in the sequence. It took 21 hours after the attack began before Tesco Bank's Fraud Strategy Team was informed about the incident. Only after what the FCA describes as a "series of errors" -- including Tesco Bank's Financial Crime Operations Team sending an email to the wrong address, instead of making a phone call as procedure requires -- was the fraud team made aware of the attack. In all that time, nothing had been done to stop the attacks, with fraudulent transactions continuing to siphon money from accounts as the bank received more and more calls from worried customers.

(tags: tesco fail tesco-bank banking pan-numbers debit-cards security fraud uk)

Running high-scale web applications on Amazon EC2 Spot Instances

AppNext's setup looks like quite good practice for a CPU-bound fleet

(tags: appnext spot-instances ec2 scalability aws ops architecture)

Amazon S3 Announces Selective Cross-Region Replication Based on Object Tags

Amazon S3 Cross-Region Replication (CRR) now supports object filtering based on S3 object tags. This allows you to identify individual objects using S3 object tags for automatic replication across AWS Regions for compliance and/or data protection. CRR is an Amazon S3 feature that automatically replicates every object uploaded to an S3 bucket to a destination bucket in a different AWS Region that you choose. S3 object tags are key-value pairs applied to S3 objects that allow you to better organize, secure, and manage your data stored in S3. By using S3 object tags to determine which objects to replicate using CRR, you now have fine grained control to selectively replicate your storage to another AWS Region to backup critical data for compliance and disaster recovery.

(tags: aws s3)

Estonia sues Gemalto for 152 mln euros over ID card flaws

Estonia’s Police and Border Guard Board (PPA) said in a statement Gemalto had created private key codes for individual cards, leaving the government IDs vulnerable to external cyber attack, rather than embedding it on the card’s chip as promised. “It turned out that our partner had violated this principle for years, and we see this as a very serious breach of contract,” PPA’s deputy director-general Krista Aas said in the statement.

(tags: gemalto fail security smartcards estonia chip-cards)

Defcon Voting Village report: Bug in one system could “flip Electoral College” | Ars Technica

ES&S strike again:

Today, six prominent information-security experts who took part in DEF CON's Voting Village in Las Vegas last month issued a report on vulnerabilities they had discovered in voting equipment and related computer systems. One vulnerability they discovered—in a high-speed vote-tabulating system used to count votes for entire counties in 23 states—could allow an attacker to remotely hijack the system over a network and alter the vote count, changing results for large blocks of voters. "Hacking just one of these machines could enable an attacker to flip the Electoral College and determine the outcome of a presidential election," the authors of the report warned. The machine in question, the ES&S M650, is used for counting both regular and absentee ballots. The device from Election Systems & Software of Omaha, Nebraska, is essentially a networked high-speed scanner like those used for scanning standardized-test sheets, usually run on a network at the county clerk's office. Based on the QNX 4.2 operating system—a real-time operating system developed and marketed by BlackBerry, currently up to version 7.0—the M650 uses Iomega Zip drives to move election data to and from a Windows-based management system. It also stores results on a 128-megabyte SanDisk Flash storage device directly mounted on the system board. The results of tabulation are output as printed reports on an attached pin-feed printer. The report authors—Matt Blaze of the University of Pennsylvania, Jake Braun of the University of Chicago, David Jefferson of the Verified Voting Foundation, Harri Hursti and Margaret MacAlpine of Nordic Innovation Labs, and DEF CON founder Jeff Moss—documented dozens of other severe vulnerabilities found in voting systems. They found that four major areas of "grave and undeniable" concern need to be addressed urgently. One of the most critical is the lack of any sort of supply-chain security for voting machines—there is no way to test the machines to see if they are trustworthy or if their components have been modified.

(tags: fail security evoting vote-tabulation us-politics voting-machines)

How Triplebyte solved its office Wi-Fi problems

This is good general wi-fi infrastructure advice for home use too

(tags: internet networking wifi ethernet routers ops)

randomised prefixes in S3 are no longer necessary to improve performance

This S3 request rate performance increase removes any previous guidance to randomize object prefixes to achieve faster performance. That means you can now use logical or sequential naming patterns in S3 object naming without any performance implications.

(tags: s3 coding architecture aws tips)

Do not fall into Oracle's Java 11 trap

The key part of the terms is as follows: You may not: use the Programs for any data processing or any commercial, production, or internal business purposes other than developing, testing, prototyping, and demonstrating your Application; The trap is as follows: Download Oracle JDK (because that is what you've always done, and it is what the web-search tells you); Use it in production (because you didn't realise the license changed); Get a nasty phone call from Oracle's license enforcement teams demanding lots of money In other words, Oracle can rely on inertia from Java developers to cause them to download the wrong (commercial) release of Java. Unless you read the text/warnings/legalese very carefully you might not even realise Oracle JDK is now commercial, and that you are therefore liable to pay Oracle for Java.

(tags: java licensing openjdk open-source oracle software jdk jre)

Dublin Bikes Animated

lots of nice graphs and dataviz around Dublin Bikes usage

(tags: bikes cycling dublin dataviz cool)

Common Cyborg | Jillian Weise | Granta

Fantastic essay:

When I tell people I am a cyborg, they often ask if I have read Donna Haraway’s ‘A Cyborg Manifesto’. Of course I have read it. And I disagree with it. The manifesto, published in 1985, promised a cyberfeminist resistance. The resistance would be networked and coded by women and for women to change the course of history and derange sexism beyond recognition. Technology would un-gender us. Instead, it has been so effective at erasing disabled women that even now, in conversation with many feminists, I am no longer surprised that disability does not figure into their notions of bodies and embodiment. Haraway’s manifesto lays claim to cyborgs (‘we are all cyborgs’) and defines the cyborg unilaterally through metaphor. To Haraway, the cyborg is a matter of fiction, a struggle over life and death, a modern war orgy, a map, a condensed image, a creature without gender. The manifesto coopts cyborg identity while eliminating reference to disabled people on which the notion of the cyborg is premised. Disabled people who use tech to live are cyborgs. Our lives are not metaphors.

(tags: via:dotat cyborg technology feminism essay disability tech jillian-weise granta)

25 Years of WIRED Predictions: Why the Future Never Arrives

These early views of the sharing economy were accurate depictions of the moment, but poor visions of the future. Within a few short years, many of those Uber drivers would be stuck paying off their cars in sub-minimum-wage jobs with no benefits. What began as an earnest insight about bits and atoms quickly turned into an arbitrage opportunity for venture capitalists eager to undercut large, lucrative markets by skirting regulations. To meet the growth and monetization demands of investors, yesterday’s sharing economy became today’s gig economy.

(tags: advertising future technology futurism predictions wired web2.0 history 1990s 2000s)

Customer Service Data Vulnerability, Intercom [Fixed]

good demo of the danger of unrestricted API keys

(tags: api-keys security intercom fail dropmobility web apis)

Peter Flynn caused the first 404

Now that's a great bit of web trivia :) "[UCC's] first webmaster was the first person to ever break a link on the web, when he moved the location of a webpage on UCC's servers without telling TimBL. Such a change resulted in the need to error-handle such an occurrence, and the 404 was born"

(tags: 404 history http web peter-flynn ucc irish-web trivia)

Credit reference agency Equifax fined for security breach

The ICO fines Equifax £500K, the maximum amount possible under the old Data Protection Act (via Privacy Kit)

(tags: via:privacy-kit ico equifax privacy data-protection uk penalties law)