The myopia boom seems to be due to spending too much time indoors
via Tony Finch
(tags: eyes health neuroscience science vision nature myopia short-sightedness)
-
Some neat new features for Mark Fletcher’s mailing-lists-as-a-service site: Markdown support, manageable archives (GREAT feature!), subgroups, calendars, files and wiki.
(tags: wiki email mailman mailing-lists mlm markdown mark-fletcher groups.io collaboration)
Justin's Linklog Posts
Stairs to nowhere, trap streets, and other Toronto oddities
‘There’s a set of stairs on Greenwood Avenue that lead nowhere. At the top, a wooden fence at the end of someone’s back yard blocks any further movement, forcing the climber to turn around and descend back to the street. What’s remarkable about the pointless Greenwood stairs, which were built in 1959 as a shortcut to a now-demolished brickyard, is that someone still routinely maintains them: in winter, some kindly soul deposits a scattering of salt lest one of the stairs’ phantom users slip; in summer someone comes with a broom to sweep away leaves. These urban leftovers are lovingly called “Thomassons” after Gary Thomasson, a former slugger for the San Francisco Giants, Oakland As, Yankees, Dodgers, and, most fatefully, the Yomiuri Giants in Tokyo.’
(tags: trap-streets maps ip google via:bldgblog mapping copyright thomassons orphaned-roads)
President’s message gets lost in (automated) translation
In a series of bizarre translations, YouTube’s automated translation service took artistic licence with the [President’s] words of warmth. When the head of state sent St Patrick’s Day greetings to viewers, the video sharing site said US comedian Tina Fey was being “particular with me head”. As President Higgins spoke of his admiration for Irish emigrants starting new communities abroad, YouTube said the President referenced blackjack and how he “just couldn’t put the new iPhone” down. And, in perhaps the most unusual moment, as he talked of people whose hearts have sympathy, the President “explained” he was once on a show “that will bar a gift card”.
(via Daragh O’Brien)(tags: lol president ireland michael-d-higgins automation translation machine-learning via:daraghobrien funny blackjack iphone tina-fey st-patrick fail)
Irish government under fire for turning its back on basic research : Nature News & Comment
Pretty much ALL of Ireland’s research scientists have put their names to an open letter to the Irish government, decrying the state of science funding, published this week in “Nature”. ‘Although total spending on research and development grew through the recession, helped by foreign investments, Ireland’s government has cut state spending on research (see ‘Celtic tiger tamed’). It also prioritized grants in 14 narrow areas — ones in which either large global markets exist, or in which Irish companies are competitive. These include marine renewable energy, smart grids, medical devices and computing. The effect has been to asphyxiate the many areas of fundamental science — including astrophysics, particle physics and areas of the life sciences — that have been deprived of funding, several researchers in Ireland told Nature. “The current policies are having a very significant detrimental effect on the health and viability of the Irish scientific ecosystem,” says Kevin Mitchell, a geneticist who studies the basis of neurological disorders at Trinity College Dublin. “Research that cannot be shoehorned into one of the 14 prioritized areas has been ineligible for most funding,” he says.’ That’s another fine mess Sean Sherlock has gotten us into :(
(tags: sean-sherlock fail ireland research government funding grants science tcd kevin-mitchell life-sciences nature)
Mars One finalist Dr. Joseph Roche rips into the project
So, here are the facts as we understand them: Mars One has almost no money. Mars One has no contracts with private aerospace suppliers who are building technology for future deep-space missions. Mars One has no TV production partner. Mars One has no publicly known investment partnerships with major brands. Mars One has no plans for a training facility where its candidates would prepare themselves. Mars One’s candidates have been vetted by a single person, in a 10-minute Skype interview. “My nightmare about it is that people continue to support it and give it money and attention, and it then gets to the point where it inevitably falls on its face,” said Roche. If, as a result, “people lose faith in NASA and possibly even in scientists, then that’s the polar opposite of what I’m about. If I was somehow linked to something that could do damage to the public perception of science, that is my nightmare scenario.”
(tags: science space mars-one tcd joseph-roche nasa mars exploration scams)
Stu Hood and Brian Degenhardt, Scala at Twitter, SF Scala @Twitter 20150217
‘Stu Hood and Brian Degenhardt talk about the history of Scala at Twitter, from inception until today, covering 2.10 migration, the original Alex Payne’s presentation from way back, pants, and more. The first five years of Scala at Twitter and the years ahead!’ Very positive indeed on the monorepo concept.
(tags: monorepo talks scala sfscala stu-hood twitter pants history repos build projects compilation gradle maven sbt)
demonstration of the importance of server-side request timeouts
from MongoDB, but similar issues often apply in many other TCP/HTTP-based systems
(tags: tcp http requests timeout mongodb reliability safety)
-
an open source stream processing software system developed by Mozilla. Heka is a “Swiss Army Knife” type tool for data processing, useful for a wide variety of different tasks, such as: Loading and parsing log files from a file system. Accepting statsd type metrics data for aggregation and forwarding to upstream time series data stores such as graphite or InfluxDB. Launching external processes to gather operational data from the local system. Performing real time analysis, graphing, and anomaly detection on any data flowing through the Heka pipeline. Shipping data from one location to another via the use of an external transport (such as AMQP) or directly (via TCP). Delivering processed data to one or more persistent data stores.
Via feylya on twitter. Looks potentially nifty(tags: heka mozilla monitoring metrics via:feylya ops statsd graphite stream-processing)
Real World Crypto 2015: Password Hashing according to Facebook
Very interesting walkthrough of how Facebook hash user passwords, including years of accreted practices
(tags: facebook passwords authentication legacy web security)
-
My account got hacked, running up over $600 in charges. Here’s the conclusion after running through the Sony support gauntlet. They can only refund up to $150. I can dispute the charges with my bank, but that will result in my account being banned. I cannot unban my account, and will thus lose my purchases (“but you only have the Last of Us and some of our free games, so it’s not a big deal”) Whomever hacked my account deactivated my PS4, and activated their own. Customer support will only permit one activation every 6 months. I’m locked out of logging into my own account on my PS4 for six months.
(tags: games sony psn playstation fail ps4 hacking security customer-support horror-stories)
Goodbye MongoDB, Hello PostgreSQL
Another core problem we’ve faced is one of the fundamental features of MongoDB (or any other schemaless storage engine): the lack of a schema. The lack of a schema may sound interesting, and in some cases it can certainly have its benefits. However, for many the usage of a schemaless storage engine leads to the problem of implicit schemas. These schemas aren’t defined by your storage engine but instead are defined based on application behaviour and expectations.
Well, don’t say we didn’t warn you ;)(tags: mongodb mysql postgresql databases storage schemas war-stories)
Apple Appstore STATUS_CODE_ERROR causes worldwide service problems
Particularly notable for this horrific misfeature, noted by jgc:
I can’t commit code at CloudFlare because we use two-factor auth for the VPN (and everything else) and non-Apple apps on my iPhone are asking for my iTunes password. Tried airplane mode and apps simply don’t load at all!
That is a _disastrous_ policy choice by Apple. Does this mean Apple can shut down third-party app operation on iOS devices worldwide should they feel like it?(tags: 2fa authy apps ios apple ownership itunes outages appstore fail jgc)
Correcting YCSB’s Coordinated Omission problem
excellent walkthrough of CO and how it affects Yahoo!’s Cloud Storage Benchmarking platform
(tags: coordinated-omission co yahoo ycsb benchmarks performance testing)
Backblaze Vaults: Zettabyte-Scale Cloud Storage Architecture
Backblaze deliver their take on nearline storage: ‘Backblaze’s cloud storage Vaults deliver 99.99999% annual durability, horizontal scalability, and 20 Gbps of per-Vault performance, while being operationally efficient and extremely cost effective. Driven from the same mindset that we brought to the storage market with Backblaze Storage Pods, Backblaze Vaults continue our singular focus of building the most cost-efficient cloud storage around.’
(tags: architecture backup storage backblaze nearline offline reed-solomon error-correction)
Ireland accused of weakening data rules
Privacy campaign group Lobbyplag puts Ireland one of top three offenders in pushing for changes to EU privacy law
(tags: privacy data-protection lobbyplag ireland eu germany lobbying)
-
the stock-photo counterpart to “Women Eating Salad” has been found
Can Spark Streaming survive Chaos Monkey?
good empirical results on Spark’s resilience to network/host outages in EC2
(tags: ec2 aws emr spark resilience ha fault-tolerance chaos-monkey netflix)
-
Concourse is a CI system composed of simple tools and ideas. It can express entire pipelines, integrating with arbitrary resources, or it can be used to execute one-off builds, either locally or in another CI system.
(tags: ci concourse-ci build deployment continuous-integration continuous-deployment devops)
Epsilon Interactive breach the Fukushima of the Email Industry (CAUCE)
Upon gaining access to an ESP, the criminals then steal subscriber data (PII such as names, addresses, telephone numbers and email addresses, and in one case, Vehicle Identification Numbers). They then use ESPs’ mailing facility to send spam; to monetize their illicit acquisition, the criminals have spammed ads for fake Adobe Acrobat and Skype software. On March 30, the Epsilon Interactive division of Alliance Data Marketing (ADS on NASDAQ) suffered a massive breach that upped the ante, substantially. Email lists of at least eight financial institutions were stolen. Thus far, puzzlingly, Epsilon has refused to release the names of compromised clients. […] The obvious issue at hand is the ability of the thieves to now undertake targeted spear-phishing problem as critically serious as it could possibly be.
(tags: cauce epsilon-interactive esp email pii data-protection spear-phishing phishing identity-theft security ads)
In Ukraine, Tomorrow’s Drone War Is Alive Today
Drones, hackerspaces and crowdfunding:
The most sophisticated UAV that has come out of the Ukrainian side since the start of the conflict is called the PD-1 from developer Igor Korolenko. It has a wingspan of nearly 10 feet, a five-hour flight time, carries electro-optical and infrared sensors as well as a video camera that broadcasts on a 128 bit encrypted channel. Its most important feature is the autopilot software that allows the drone to return home in the event that the global positioning system link is jammed or lost. Drone-based intelligence gathering is often depicted as risk-free compared to manned aircraft or human intelligence gathering, but, says Korolenko, if the drone isn’t secure or the signature is too obvious, the human coasts can be very, very high. “Russian military sometimes track locations of ground control stations,” he wrote Defense One in an email. “Therefore UAV squads have to follow certain security measures – to relocate frequently, to move out antennas and work from shelter, etc. As far as I know, two members of UAV squads were killed from mortar attacks after [their] positions were tracked by Russian electronic warfare equipment.”
(via bldgblog)(tags: via:bldgblog war drones uav future ukraine russia tech aircraft pd-1 crowdfunding)
-
a 303 and an 808 in your browser. this is deadly
Ubuntu To Officially Switch To systemd Next Monday – Slashdot
Jesus. This is going to be the biggest shitfest in the history of Linux…
-
A project to reduce systemd to a base initd, process supervisor and transactional dependency system, while minimizing intrusiveness and isolationism. Basically, it’s systemd with the superfluous stuff cut out, a (relatively) coherent idea of what it wants to be, support for non-glibc platforms and an approach that aims to minimize complicated design. uselessd is still in its early stages and it is not recommended for regular use or system integration.
This may be the best option to evade the horrors of systemd. Japan’s Robot Dogs Get Funerals as Sony Looks Away
in July 2014, [Sony’s] repairs [of Aibo robot dogs] stopped and owners were left to look elsewhere for help. The Sony stiff has led not only to the formation of support groups–where Aibo enthusiasts can share tips and help each other with repairs–but has fed the bionic pet vet industry. “The people who have them feel their presence and personality,” Nobuyuki Narimatsu, director of A-Fun, a repair company for robot dogs, told AFP. “So we think that somehow, they really have souls.” While concerted repair efforts have kept many an Aibo alive, a shortage of spare parts means that some of their lives have come to an end.
(tags: sony aibo robots japan dogs pets weird future badiotday iot gadgets)
“Cuckoo Filter: Practically Better Than Bloom”
‘We propose a new data structure called the cuckoo filter that can replace Bloom filters for approximate set membership tests. Cuckoo filters support adding and removing items dynamically while achieving even higher performance than Bloom filters. For applications that store many items and target moderately low false positive rates, cuckoo filters have lower space overhead than space-optimized Bloom filters. Our experimental results also show that cuckoo filters outperform previous data structures that extend Bloom filters to support deletions substantially in both time and space.’
(tags: algorithms paper bloom-filters cuckoo-filters cuckoo-hashing data-structures false-positives big-data probabilistic hashing set-membership approximation)
Amazing cutting from Vanity Fair, 1896, for International Women’s Day
“The sisters make a pretty picture on the platform ; but it is not women of their type who need to assert themselves over Man. However, it amuses them–and others ; and I doubt if the tyrant has much to fear from their little arrows.” Constance Markievicz was one of those sisters, and the other was Eva Gore-Booth.
(tags: markievicz history ireland sligo vanity-fair 19th-century dismissal sexism iwd women)
-
Authy doesn’t come off well here: ‘Authy should have been harder to break. It’s an app, like Authenticator, and it never left Davis’ phone. But Eve simply reset the app on her phone using a mail.com address and a new confirmation code, again sent by a voice call. A few minutes after 3AM, the Authy account moved under Eve’s control.’
(tags: authy security hacking mfa authentication google apps exploits)
Ask the Decoder: Did I sign up for a global sleep study?
How meaningful is this corporate data science, anyway? Given the tech-savvy people in the Bay Area, Jawbone likely had a very dense sample of Jawbone wearers to draw from for its Napa earthquake analysis. That allowed it to look at proximity to the epicenter of the earthquake from location information. Jawbone boasts its sample population of roughly “1 million Up wearers who track their sleep using Up by Jawbone.” But when looking into patterns county by county in the U.S., Jawbone states, it takes certain statistical liberties to show granularity while accounting for places where there may not be many Jawbone users. So while Jawbone data can show us interesting things about sleep patterns across a very large population, we have to remember how selective that population is. Jawbone wearers are people who can afford a $129 wearable fitness gadget and the smartphone or computer to interact with the output from the device. Jawbone is sharing what it learns with the public, but think of all the public health interests or other third parties that might be interested in other research questions from a large scale data set. Yet this data is not collected with scientific processes and controls and is not treated with the rigor and scrutiny that a scientific study requires. Jawbone and other fitness trackers don’t give us the option to use their devices while opting out of contributing to the anonymous data sets they publish. Maybe that ought to change.
(tags: jawbone privacy data-protection anonymization aggregation data medicine health earthquakes statistics iot wearables)
Pinterest’s highly-available configuration service
Stored on S3, update notifications pushed to clients via Zookeeper
A Journey into Microservices | Hailo Tech Blog
Excellent three-parter from Hailo, describing their RabbitMQ+Go-based microservices architecture. Very impressive!
(tags: hailo go microservices rabbitmq amqp architecture blogs)
-
The Large Hadron Migrator is a tool to perform live database migrations in a Rails app without locking.
The basic idea is to perform the migration online while the system is live, without locking the table. In contrast to OAK and the facebook tool, we only use a copy table and triggers. The Large Hadron is a test driven Ruby solution which can easily be dropped into an ActiveRecord or DataMapper migration. It presumes a single auto incremented numerical primary key called id as per the Rails convention. Unlike the twitter solution, it does not require the presence of an indexed updated_at column.
(tags: migrations database sql ops mysql rails ruby lhm soundcloud activerecord)
Biased Locking in HotSpot (David Dice’s Weblog)
This is pretty nuts. If biased locking in the HotSpot JVM is causing performance issues, it can be turned off:
You can avoid biased locking on a per-object basis by calling System.identityHashCode(o). If the object is already biased, assigning an identity hashCode will result in revocation, otherwise, the assignment of a hashCode() will make the object ineligible for subsequent biased locking.
(tags: hashcode jvm java biased-locking locking mutex synchronization locks performance)
A Zero-Administration Amazon Redshift Database Loader – AWS Big Data Blog
nifty!
Archie Markup Language (ArchieML)
ArchieML (or “AML”) was created at The New York Times to make it easier to write and edit structured text on deadline that could be rendered in web pages, or more specifically, rendered in interactive graphics. One of the main goals was to make it easy to tag text as data, without having type a lot of special characters. Another goal was to allow the document to contain lots of notes and draft text that would not be read into the data. And finally, because we make extensive use of Google Documents’s concurrent-editing features — while working on a graphic, we can have several reporters, editors and developers all pouring information into a single document — we wanted to have a format that could survive being edited by users who may never have seen ArchieML or any other markup language at all before.
California Says Motorcycle Lane-Splitting Is Hella Safe
A recent yearlong study by the California Office of Traffic Safety has found motorcycle lane-splitting to be a safe practice on public roads. The study looked at collisions involving 7836 motorcyclists reported by 80 police departments between August 2012 and August 2013. “What we learned is, if you lane-split in a safe or prudent manner, it is no more dangerous than motorcycling in any other circumstance,” state spokesman Chris Cochran told the Sacramento Bee. “If you are speeding or have a wide speed differential (with other traffic), that is where the fatalities came about.”
(tags: lane-splitting cycling motorcycling bikes road-safety driving safety california)
-
Good terminology for this concept:
The try server runs a similar configuration to the continuous integration server, except that it is triggered not on commits but on “try job request”, in order to test code pre-commit.
See also https://wiki.mozilla.org/ReleaseEngineering/TryServer for the Moz take on it.(tags: build ci integration try-server jenkins buildbot chromium development)
-
A Dropwizard Metrics extension to instrument JDBC resources and measure SQL execution times.
(tags: metrics sql jdbc instrumentation dropwizard)
HP is trying to patent Continuous Delivery
This is appalling bollocks from HP:
On 1st March 2015 I discovered that in 2012 HP had filed a patent (WO2014027990) with the USPO for ‘Performance tests in a continuous deployment pipeline‘ (the patent was granted in 2014). [….] HP has filed several patents covering standard Continuous Delivery (CD) practices. You can help to have these patents revoked by providing ‘prior art’ examples on Stack Exchange.
In fairness, though, this kind of shit happens in most big tech companies. This is what happens when you have a broken software patenting system, with big rewards for companies who obtain shitty troll patents like these, and in turn have companies who reward the engineers who sell themselves out to write up concepts which they know have prior art. Software patents are broken by design!(tags: cd devops hp continuous-deployment testing deployment performance patents swpats prior-art)
Exponential Backoff And Jitter
Great go-to explainer blog post for this key distributed-systems reliability concept, from the always-solid Marc Brooker
(tags: marc-brooker distsys networking backoff exponential jitter retrying retries reliability occ)
17 Things Everyone Must Eat In Dublin
actually a fairly sane list of lunchy options — the SMS fish finger butty is a lunch staple for us Swrvers
VividCortex uses K-Means Clustering to discover related metrics
After selecting an interesting spike in a metric, the algorithm can automate picking out a selection of other metrics which spiked at the same time. I can see that being pretty damn useful
(tags: metrics k-means-clustering clustering algorithms discovery similarity vividcortex analysis data)
Alibaba’s cloud service launches in US, wants to rain all over Amazon
server-hosting only for now. Interesting!
Alibaba’s cloud platform already competes with the likes of AWS in China. Aliyun’s Chinese data centers are in Beijing, Hangzhou, Qingdao, Hong Kong, and Shenzhen. “For the time being, we are just testing the water,” Yu said today. That means Aliyun will focus first on Chinese companies doing business in the US. “We know well what Chinese clients need, and now it’s time for us to learn what US clients need,” he added.
-
the following guidelines maximize bandwidth usage: Optimizing the sizes of the file parts, whether they are part of a large file or an entire small file; Optimizing the number of parts transferred concurrently. Tuning these two parameters achieves the best possible transfer speeds to [S3].
-
Excellent web-based ASCII-art editor (via Craig)
(tags: via:craig design ascii diagrams editor ascii-art art asciiflow drawing)
Services Engineering Reading List
good list of papers/articles for fans of scalability etc.
(tags: architecture papers reading reliability scalability articles to-read)
-
nice, free-during-beta Mac app to draw ASCII-art diagrams
-
“Open source APM for Java” — profiling in production, with a demo benchmark showing about a 2% performance impact. Wonder about effects on memory/GC, though
(tags: apm java metrics measurement new-relic profiling glowroot)
“Everything you’ve ever said to Siri/Cortana has been recorded…and I get to listen to it”
This should be a reminder.
At first, I though these sound bites were completely random. Then I began to notice a pattern. Soon, I realized that I was hearing peoples commands given to their mobile devices. Guys, I’m telling you, if you’ve said it to your phone, it’s been recorded…and there’s a damn good chance a 3rd party is going to hear it.
(tags: privacy google siri cortana android voice-recognition outsourcing mobile)
-
Fantastic 1997-era book of interviews with the programmers behind some of the greatest games in retrogaming history:
Halcyon Days: Interviews with Classic Computer and Video Game Programmers was released as a commercial product in March 1997. At the time it was one of the first retrogaming projects to focus on lost history rather than game collecting, and certainly the first entirely devoted to the game authors themselves. Now a good number of the interviewees have their own web sites, but none of them did when I started contacting them in 1995. […] If you have any of the giddy anticipation that I did whenever I picked up a magazine containing an interview with Mark Turmell or Dan [M.U.L.E.] Bunten, then you want to start reading.
(tags: book games history coding interviews via:walter)
Pub Table Quiz – In Aid of Digital Rights Ireland
Jason Roe is organising a Table Quiz in Dublin on March 26th to support fundraising efforts by Digital Rights Ireland. We will supply tables, questions and a ready supply of beer and maybe finger food.
Why are transhumanists such dicks?
Good discussion from a transhumanist forum (via Boing Boing):
“I’ve been around and interviewed quite a lot of self-identified transhumanists in the last couple of years, and I’ve noticed many of them express a fairly stark ideology that is at best libertarian, and at worst Randian. Very much “I want super bionic limbs and screw the rest of the world”. They tend to brush aside the ethical, environmental, social and political ramifications of human augmentation so long as they get to have their toys. There’s also a common expression that if sections of society are harmed by transhumanist progress, then it is unfortunate but necessary for the greater good (the greater good often being bestowed primarily upon those endorsing the transhumanism). That attitude isn’t prevalent on this forum at all – I think the site tends to attract more practical body-modders than theoretical transhumanists – but I wondered if anyone else here had experienced the same attitudes in their own circles? What do you make of it?”
(tags: transhumanism evolution body-modding surgery philosophy via:boingboing libertarianism society politics)
Release Protocol Buffers v3.0.0-alpha-2 · google/protobuf
New major-version track for protobuf, with some interesting new features: Removal of field presence logic for primitive value fields, removal of required fields, and removal of default values. This makes proto3 significantly easier to implement with open struct representations, as in languages like Android Java, Objective C, or Go. Removal of unknown fields. Removal of extensions, which are instead replaced by a new standard type called Any. Fix semantics for unknown enum values. Addition of maps. Addition of a small set of standard types for representation of time, dynamic data, etc. A well-defined encoding in JSON as an alternative to binary proto encoding.
(tags: protobuf binary marshalling serialization google grpc proto3 coding open-source)
RIPQ: Advanced photo caching on flash for Facebook
Interesting priority-queue algorithm optimised for caching data on SSD
(tags: priority-queue algorithms facebook ssd flash caching ripq papers)
-
Performance-diagnosis-as-a-service. Cool.
Users download and install an Illuminate Daemon using a simple installer which starts up a small stand alone Java process. The Daemon sits quietly unless it is asked to start gathering SLA data and/or to trigger a diagnosis. Users can set SLA’s via the dashboard and can opt to collect latency measurements of their transactions manually (using our library) or by asking Illuminate to automatically instrument their code (Servlet and JDBC based transactions are currently supported). SLA latency data for transactions is collected on a short cycle. When the moving average of latency measurements goes above the SLA value (e.g. 150ms), a diagnosis is triggered. The diagnosis is very quick, gathering key data from O/S, JVM(s), virtualisation and other areas of the system. The data is then run through the machine learned algorithm which will quickly narrow down the possible causes and gather a little extra data if needed. Once Illuminate has determined the root cause of the performance problem, the diagnosis report is sent back to the dashboard and an alert is sent to the user. That alert contains a link to the result of the diagnosis which the user can share with colleagues. Illuminate has all sorts of backoff strategies to ensure that users don’t get too many alerts of the same type in rapid succession!
(tags: illuminate jclarity java jvm scala latency gc tuning performance)
-
Binary message marshalling, client/server stubs generated by an IDL compiler, bidirectional binary protocol. CORBA is back from the dead! Intro blog post: http://googledevelopers.blogspot.ie/2015/02/introducing-grpc-new-open-source-http2.html Relevant: Steve Vinoski’s commentary on protobuf-rpc back in 2008: http://steve.vinoski.net/blog/2008/07/13/protocol-buffers-leaky-rpc/
(tags: http rpc http2 netty grpc google corba idl messaging)
Bloom Cookies: web search personalization without user tracking
Interesting paper
(tags: bloom-cookies bloom-filters data-structures cookies privacy personalization user-tracking http)
Why we run an open source program – Walmart Labs
This is a great exposition of why it’s in a company’s interest to engage with open source. Not sure I agree with ‘engineers are the artists of our generation’ but the rest are spot on
(tags: development open-source walmart node coding via:hn hiring)
-
MQTT definitely has a smaller size on the wire. It’s also simpler to parse (let’s face it, Huffman isn’t that easy to implement) and provides guaranteed delivery to cater to shaky wireless networks. On the other hand, it’s also not terribly extensible. There aren’t a whole lot of headers and options available, and there’s no way to make custom ones without touching the payload of the message. It seems that HTTP/2 could definitely serve as a reasonable replacement for MQTT. It’s reasonably small, supports multiple paradigms (pub/sub & request/response) and is extensible. Its also supported by the IETF (whereas MQTT is hosted by OASIS). From conversations I’ve had with industry leaders in the embedded software and chip manufacturing, they only want to support standards from the IETF. Many of them are still planning to support MQTT, but they’re not happy about it. I think MQTT is better at many of the things it was designed for, but I’m interested to see over time if those advantages are enough to outweigh the benefits of HTTP. Regardless, MQTT has been gaining a lot of traction in the past year or two, so you may be forced into using it while HTTP/2 catches up.
(tags: http2 mqtt iot pub-sub protocols ietf embedded push http)
Automatically Deploy from GitHub Using AWS CodeDeploy – Application Management Blog
I like this
(tags: github aws ec2 codedeploy deployment ops)
Programmer IS A Career Path, Thank You
Well said — Amazon had a good story around this btw
(tags: programming coding career work life)
how Curator fixed issues with the Hive ZooKeeper Lock Manager Implementation
Ugh, ZK is a bear to work with.
Apache Curator is open source software which is able to handle all of the above scenarios transparently. Curator is a Netflix ZooKeeper Library and it provides a high-level API, CuratorFramework, that simplifies using ZooKeeper. By using a singleton CuratorFramework instance in the new ZooKeeperHiveLockManager implementation, we not only fixed the ZooKeeper connection issues, but also made the code easy to understand and maintain.
(tags: zookeeper apis curator netflix distributed-locks coding hive)
Advanced cryptographic ratcheting
Forward secrecy and in-session key “ratcheting”
(tags: crypto privacy key-management forward-secrecy pfs key-ratcheting key-rotation)
-
What a mess.
What’s faster: PV, HVM, HVM with PV drivers, PVHVM, or PVH? Cloud computing providers using Xen can offer different virtualization “modes”, based on paravirtualization (PV), hardware virtual machine (HVM), or a hybrid of them. As a customer, you may be required to choose one of these. So, which one?
(tags: ec2 linux performance aws ops pv hvm xen virtualization)
Proving that Android’s, Java’s and Python’s sorting algorithm is broken (and showing how to fix it)
Wow, this is excellent work. A formal verification of Tim Peters’ TimSort failed, resulting in a bugfix:
While attempting to verify TimSort, we failed to establish its instance invariant. Analysing the reason, we discovered a bug in TimSort’s implementation leading to an ArrayOutOfBoundsException for certain inputs. We suggested a proper fix for the culprit method (without losing measurable performance) and we have formally proven that the fix actually is correct and that this bug no longer persists.
(tags: timsort algorithms android java python sorting formal-methods proofs openjdk)
-
“Cheap SSL certs from $4.99/yr” — apparently recommended for cheap, low-end SSL certs
-
Erasure codes, such as Reed-Solomon (RS) codes, are increasingly being deployed as an alternative to data-replication for fault tolerance in distributed storage systems. While RS codes provide significant savings in storage space, they can impose a huge burden on the I/O and network resources when reconstructing failed or otherwise unavailable data. A recent class of erasure codes, called minimum-storage-regeneration (MSR) codes, has emerged as a superior alternative to the popular RS codes, in that it minimizes network transfers during reconstruction while also being optimal with respect to storage and reliability. However, existing practical MSR codes do not address the increasingly important problem of I/O overhead incurred during reconstructions, and are, in general, inferior to RS codes in this regard. In this paper, we design erasure codes that are simultaneously optimal in terms of I/O, storage, and network bandwidth. Our design builds on top of a class of powerful practical codes, called the product-matrix-MSR codes. Evaluations show that our proposed design results in a significant reduction the number of I/Os consumed during reconstructions (a 5 reduction for typical parameters), while retaining optimality with respect to storage, reliability, and network bandwidth.
(tags: erasure-coding reed-solomon compression reliability reconstruction replication fault-tolerance storage bandwidth usenix papers)
Everyday I’m Shuffling – Tips for Writing Better Spark Programs [slides]
Two Spark experts from Databricks provide some good tips
Cowen went golfing and officials dithered as country burned in 2008 – Independent.ie
Lest we forget, the sheer bullshitting ineptitude of Fianna Fail as they managed to shamble into destroying Ireland’s economy in 2008:
Once that nasty bit of business was done, the Cabinet departed en masse for six weeks on their summer holidays, despite the emerging economic and financial tsunami. Cowen and family famously took up residence in a caravan park in Connemara as opposed to his ‘official’ residence at the Mannin Bay Hotel nearby. When pressed by our reporter Niamh Horan as to why he was not at his station, he defensively replied: “I don’t understand it. First the media have a go at me because I’m taking a holiday with my family and then they come down to see if I’m having a good time!” he exclaimed.
(tags: 2008 meltdown ireland brian-cowen connemara politics history fianna-fail)
How I Became A Minor Celebrity In China (After My Stolen Phone Ended Up There)
Phone is stolen, shipped to China, and winds up being bought by “Brother Orange” — then the story becomes China’s biggest viral hit
-
40 minutes of multi-zone network outage for majority of instances. ‘The internal software system which programs GCE’s virtual network for VM egress traffic stopped issuing updated routing information. The cause of this interruption is still under active investigation. Cached route information provided a defense in depth against missing updates, but GCE VM egress traffic started to be dropped as the cached routes expired.’ I wonder if Google Pimms fired the alarms for this ;)
(tags: google outages gce networking routing pimms multi-az cloud)
Listen to a song made from data lost during MP3 conversion
Ryan McGuire, a PhD student in Composition and Computer Technologies at the University of Virginia Center for Computer Music, has created the project The Ghost In The MP3 [….] For his first trick, McGuire took Suzanne Vega’s ‘Tom’s Diner’ and drained it into a vaporous piece titled ‘moDernisT.” McGuire chose the track he explains on his site because it was famously used as one of the main controls in the listening tests used to develop the MP3 algorithm.
(tags: mp3 music suzanne-vega compression)
-
A gateway script, now included in PCP
-
System performance metrics framework, plugged by Netflix, open-source for ages
(tags: open-source pcp performance system metrics ops red-hat netflix)
Superfish: A History Of Malware Complaints And International Surveillance – Forbes
Superfish, founded and led by former Intel employee and ex-surveillance boffin Adi Pinhas, has been criticised by users the world over since its inception in 2006.
(tags: superfish lenovo privacy surveillance ads java windows mac firefox pups ssl tls ad-injection komodia)
The Superfish certificate has been cracked, exposing Lenovo users to attack | The Verge
The cracked certificate exposes Lenovo users to man-in-the-middle attacks, similar to those opened up by Heartbleed. Armed with this password and the right software, a coffee shop owner could potentially spy on any Lenovo user on her network, collecting any passwords that were entered during the session. The evil barista could also insert malware into the data stream at will, disguised as a software update or a trusted site.
Amazingly stupid.Police have asked Dropcam for video from people’s home cameras — Fusion
“Like any responsible father, Hugh Morrison had installed cameras in every room in the flat,” is the opening line of Intrusion, a 2012 novel set in the near future. Originally installed so that Hugh and his wife can keep an eye on their kids, the Internet-connected cameras wind up being used later in the novel by police who tap into the feeds to monitor the couple chatting on their couch when they are suspected of anti-societal behavior. As with so many sci-fi scenarios, the novel’s vision was prophetic. People are increasingly putting small Internet-connected cameras into their homes. And law enforcement officials are using the cameras to collect evidence about them.
(tags: privacy dropcam cameras surveillance law-enforcement)
Extracting the SuperFish certificate
not exactly the most challenging reverse I’ve ever seen ;)
(tags: reverse-engineering security crypto hacking tls ssl superfish lenovo)
The Great SIM Heist: How Spies Stole the Keys to the Encryption Castle
Holy shit. Gemalto totally rooted.
With [Gemalto’s] stolen encryption keys, intelligence agencies can monitor mobile communications without seeking or receiving approval from telecom companies and foreign governments. Possessing the keys also sidesteps the need to get a warrant or a wiretap, while leaving no trace on the wireless provider’s network that the communications were intercepted. Bulk key theft additionally enables the intelligence agencies to unlock any previously encrypted communications they had already intercepted, but did not yet have the ability to decrypt. […] According to one secret GCHQ slide, the British intelligence agency penetrated Gemalto’s internal networks, planting malware on several computers, giving GCHQ secret access. We “believe we have their entire network,” the slide’s author boasted about the operation against Gemalto.
(tags: encryption security crypto nsa gchq gemalto smartcards sim-cards privacy surveillance spying)
One year of InfluxDB and the road to 1.0
half of the [Monitorama] attendees were employees and entrepreneurs at monitoring, metrics, DevOps, and server analytics companies. Most of them had a story about how their metrics API was their key intellectual property that took them years to develop. The other half of the attendees were developers at larger organizations that were rolling their own DevOps stack from a collection of open source tools. Almost all of them were creating a “time series database” with a bunch of web services code on top of some other database or just using Graphite. When everyone is repeating the same work, it’s not key intellectual property or a differentiator, it’s a barrier to entry. Not only that, it’s something that is hindering innovation in this space since everyone has to spend their first year or two getting to the point where they can start building something real. It’s like building a web company in 1998. You have to spend millions of dollars and a year building infrastructure, racking servers, and getting everything ready before you could run the application. Monitoring and analytics applications should not be like this.
(tags: graphite monitoring metrics tsd time-series analytics influxdb open-source)
-
Sysdig Cloud users have the ability to view and analyze Java Management Extensions (JMX) metrics out of the box with no additional configuration or setup required.
Will the madness never end? Komodia SSL certificates are EVERYWHERE
I think that at this point it is safe to assume that any SSL interception product sold by Komodia or based on the Komodia SDK is going to be using the same method. What does this mean? Well, this means that those dodgy certificates aren’t limited to Lenovo laptops sold over a specific date range. It means that anyone who has come into contact with a Komodia product, or who has had some sort of Parental Control software installed on their computer should probably check to see if they are affected.
(tags: komodia via:jgc ssl lenovo parental-control censorware mitm)
Twitter’s Answers architecture
Twitter’s mobile-device analytics service architecture, with Kafka and Storm in full Lambda-Architecture mode
(tags: twitter lambda-architecture storm kafka architecture)
-
The argument for the “monorepo” — ie. lots of projects in a single Git repo. There’s lots more discussion pro/con on twitter, e.g.: https://twitter.com/search?q=monorepo&src=typd , https://twitter.com/hivetheory/timelines/449385567982067713
(tags: monorepo git repository dependencies libraries coding)
-
Poor hardware imaging practices, basically:
It looks like all devices with the fingerprint are Dropbear SSH instances that have been deployed by Telefonica de Espana. It appears that some of their networking equipment comes setup with SSH by default, and the manufacturer decided to re-use the same operating system image across all devices.
FreeBSD breaks its kernel RNG for 4 months
If you are running a current kernel r273872 or later, please upgrade your kernel to r278907 or later immediately and regenerate keys. I discovered an issue where the new framework code was not calling randomdev_init_reader, which means that read_random(9) was not returning good random data. This means most/all keys generated may be predictable and must be regenerated.
What every programmer should know about solid-state drives
Lots of good advice here for dealing with SSDs
Azul Zing on Ubuntu on AWS Marketplace
hmmm, very interesting — the super-low-latency Zing JVM is available as a commercial EC2 instance type, at costs less than the EC2 instance price
Crowdsourcing isn’t broken — Backchannel — Medium
‘A great compendium by @harper of techniques for handling trolls and griefers in online communities’, via kragen
(tags: via:kragen antispam filtering trolls community crowdsourcing threadless harper griefers abuse tips)
South Korea faces $1bn bill after hackers raid national ID database • The Register
Simon McGarr says: ‘80% of S.Korea’s population have had their ID number stolen, crimewave ongoing. >> Turns out a pot of honey is sweet’
(tags: fail south-korea korea security id-cards ssn id-numbers privacy)
Sign up for Privacy International’s anti-surveillance campaign
Have you ever made a phone call, sent an email, or, you know, used the internet? Of course you have! Chances are, at some point over the past decade, your communications were swept up by the U.S. National Security Agency. The NSA then shares information with the UK Government’s intelligence agency GCHQ by default. A recent court ruling found that this sharing was unlawful. But no one could find out if their records were collected and then illegally shared between these two agencies… until now! Because of our recent victory against the UK intelligence agency in court, now anyone in the world — yes, ANYONE, including you — can find out if GCHQ illegally received information about you from the NSA. Join our campaign by entering your details below to find out if GCHQ illegally spied on you, and confirm via the email we send you. We’ll then go to court demanding that they finally come clean on unlawful surveillance.
(tags: gchq nsa spying surveillance internet phone uk law campaign privacy-international)
How “omnipotent” hackers tied to NSA hid for 14 years—and were found at last
‘”Equation Group” ran the most advanced hacking operation ever uncovered.’ Mad stuff. The security industry totally failed here
(tags: nsa privacy security surveillance hacking keyloggers malware)
-
decent set of intro slides
Apache Spark: A Delight for Developers | Cloudera Engineering Blog
Another Spark intro blog post
-
‘JOL (Java Object Layout) is the tiny toolbox to analyze object layout schemes in JVMs. These tools are using Unsafe, JVMTI, and Serviceability Agent (SA) heavily to decoder the actual object layout, footprint, and references. This makes JOL much more accurate than other tools relying on heap dumps, specification assumptions, etc.’ Recommended by Nitsan Wakart, looks pretty useful for JVM devs
(tags: java jvm tools scala memory estimation ram object-layout debugging via:nitsan)
HdrHistogram: A better latency capture method
An excellent intro to HdrHistogram usage
(tags: hdrhistogram hdr histograms statistics latency measurement metrics percentiles quantiles gil-tene nitsan-wakart)
-
Butterfield insists that Slack improves on the basic messaging functionality offered by its predecessors. The company plans to expand from 100 employees to 250 this year, open an office in Dublin, and launch a version that supports large companies with multiple teams.
-
A tool for managing Apache Kafka. It supports the following : Manage multiple clusters; Easy inspection of cluster state (topics, brokers, replica distribution, partition distribution); Run preferred replica election; Generate partition assignments (based on current state of cluster); Run reassignment of partition (based on generated assignments)
Vaurien, the Chaos TCP Proxy — Vaurien 1.8 documentation
Vaurien is basically a Chaos Monkey for your TCP connections. Vaurien acts as a proxy between your application and any backend. You can use it in your functional tests or even on a real deployment through the command-line. Vaurien is a TCP proxy that simply reads data sent to it and pass it to a backend, and vice-versa. It has built-in protocols: TCP, HTTP, Redis & Memcache. The TCP protocol is the default one and just sucks data on both sides and pass it along. Having higher-level protocols is mandatory in some cases, when Vaurien needs to read a specific amount of data in the sockets, or when you need to be aware of the kind of response you’re waiting for, and so on. Vaurien also has behaviors. A behavior is a class that’s going to be invoked everytime Vaurien proxies a request. That’s how you can impact the behavior of the proxy. For instance, adding a delay or degrading the response can be implemented in a behavior. Both protocols and behaviors are plugins, allowing you to extend Vaurien by adding new ones. Last (but not least), Vaurien provides a couple of APIs you can use to change the behavior of the proxy live. That’s handy when you are doing functional tests against your server: you can for instance start to add big delays and see how your web application reacts.
(tags: proxy tcp vaurien chaos-monkey testing functional-testing failures sockets redis memcache http)
Embed-able Computers are a Thing. — February 12, 2015
‘If it works, a copy of Burgertime for DOS is now in your browser, clickable from my entry. If it doesn’t… well, no Burgertime for you. (Unless you visit the page.) There’s a “share this” link in the new archive.org interface for sharing these in-browser emulations in web pages, weblogs and who knows what else.’
(tags: sharing embeds html javascript emulation msdos burgertime games archive.org)
-
According to a report posted Thursday to the website of the state-run China Youth Daily, the Cyberspace Administration of China choral group this week unveiled a new song, “Cyberspace Spirit,” glorifying the cleanliness and clarity of China’s uniquely managed Internet. The song, an orchestral march built around a chorus that proclaims China’s ambition to become an “Internet power,” opens with lyrics describing celestial bodies keeping careful watch over the sky. From there, the lyrics conjure more vivid imagery, comparing the Internet to “a beam of incorruptible sunlight” that unites “the powers of life from all creation.”
(tags: china great-firewall censorship music songs cyberspace-spirit omgwtfbbq)
An Algorithm to Extract Looping GIFs From Videos
/r/loopinggifs, automated
(tags: gif python pymovie video algorithms looping animated-gifs)
Samsung’s smart TVs are inserting unwanted ads into users’ own movies
Amazingly shitty. Never buying a Samsung TV if this is what they think is acceptable
Massive thumbs-down Docker review
extensive.
The Pizza Party Where Everyone Got Fired
The testers at [MAJOR PUBLISHER] had just finished wrapping up testing on a project we’ll call “Biolands.” And to congratulate them, the man in charge arranged a huge bowling/pizza party for the end of the week. Of course everyone is hyped for the event. So the day finally arrives and all the testers show up. They all start bowling and eating pizza. After a few hours of everyone enjoying themselves, the VP asks for everyone’s attention. When he does manage to get the team to listen, he begins to thank them for their hard work and has the leads hand them their termination papers.
And many other horror stories from the worst software industry of all — games.(tags: games software jobs bowling pizza fired horror-stories hr employment)
Automating Tinder with Eigenfaces
While my friends were getting sucked into “swiping” all day on their phones with Tinder, I eventually got fed up and designed a piece of software that automates everything on Tinder.
This is awesome. (via waxy)(tags: via:waxy tinder eigenfaces machine-learning k-nearest-neighbour algorithms automation ai)
-
Our latest open source release from Swrve Labs: an Apache-licensed, SLF4J-compatible, simple, fluent API for rate-limited logging in Java: ‘A RateLimitedLog object tracks the rate of log message emission, imposes an internal rate limit, and will efficiently suppress logging if this is exceeded. When a log is suppressed, at the end of the limit period, another log message is output indicating how many log lines were suppressed. This style of rate limiting is the same as the one used by UNIX syslog; this means it should be comprehensible, easy to predict, and familiar to many users, unlike more complex adaptive rate limits.’ We’ve been using this in production for months — it’s pretty nifty ;) Never fear your logs again!
(tags: logs logging coding java open-source swrve slf4j rate-limiting libraries)
BENCHMARKING THE RASPBERRY PI 2
Retro console emulation! Mario Kart and Ocarina of Time and Conker’s Bad Fur Day! Nobody actually builds stuff with the Raspberry Pi, it’s just an odd form of nostalgic consumerism wrapped up in a faddish ‘making’ trend! The original Raspberry Pi saw a lot of emulator use, but it was limited: the Pi 1 could handle the NES, SNES, Genesis/Mega Drive, and other earlier consoles with ease. Emulator performance for N64 and original Playstation games was just barely unplayable. Now, the Raspi 2 can easily handle N64 and PSX games. [HoZyVN] tried out N64’s Mario Kart and PSX’s Spyro the Dragon. They’re playable, and an entire generation rushed out to Microcenter to relive their glory days of sitting with their faces embedded in a console television drinking Sunny D all day.
(tags: raspberry-pi emulation n64 playstation gaming hardware benchmarks)
“Man vs Machine: Practical Adversarial Detection of Malicious Crowdsourcing Workers” [paper]
“traditional ML techniques are accurate (95%–99%) in detection but can be highly vulnerable to adversarial attacks”. ain’t that the truth
(tags: security adversarial-attacks machine-learning paper crowdsourcing via:kragen)
-
Nice looking static code validation tool for Java, from Google. I recognise a few of these errors ;)
(tags: google static code-validation lint testing java coding)
Totally Mexico! How the Nathan Barley nightmare came true | Television & radio | The Guardian
Nathan Barley was scarcely less prophetic when it came to TV itself. In one episode Nathan’s friend Claire makes a comically po-faced, self-righteous but secretly rather narcissistic documentary about a choir made up of drug addicts. Nine years later, Channel 4 made Addicts’ Symphony for real.
(tags: nathan-barley well-weapon vice shoreditch drugs charlie-brooker chris-morris sitcoms channel-4)
Google Maps Tenth Anniversary | Re/code
the whole story of GMaps
(tags: google history maps technology mapping recode via:anildash)
0x74696d | Falling In And Out Of Love with DynamoDB, Part II
Good DynamoDB real-world experience post, via Mitch Garnaat. We should write up ours, although it’s pretty scary-stuff-free by comparison
South Korean spymaster had a team posting political comments on Twitter and rigging polls
Mad stuff. The South Korean National Intelligence Service directly interfering in a democratic election by posting fake comments and rigging online polls
(tags: web polls twitter social-media psyops korea south-korea nis sock-puppets democracy)
Dept of Education and Primary Online Database
Simon McGarr has a theory — the indefinite data retention of sensitive data on primary schoolchildren actually has a genesis in the Irish state wishing to protect itself against prosecution from future child abuse cases
(tags: ireland child-abuse schools simon-mcgarr pod)
UK-US surveillance regime was unlawful ‘for seven years’ | UK news | The Guardian
The regime that governs the sharing between Britain and the US of electronic communications intercepted in bulk was unlawful until last year, a secretive UK tribunal has ruled. The Investigatory Powers Tribunal (IPT) declared on Friday that regulations covering access by Britain’s GCHQ to emails and phone records intercepted by the US National Security Agency (NSA) breached human rights law.
Digital Rights Ireland announces its first conference!
Digital Rights Europe, Wednesday, April 15th in Dublin. deadly!
(tags: digital-rights ireland dri privacy data-protection europe eu)
Twitter CEO: ‘We suck at dealing with abuse’ | The Verge
‘We suck at dealing with abuse and trolls on the platform and we’ve sucked at it for years. It’s no secret and the rest of the world talks about it every day. We lose core user after core user by not addressing simple trolling issues that they face every day. I’m frankly ashamed of how poorly we’ve dealt with this issue during my tenure as CEO. It’s absurd. There’s no excuse for it. I take full responsibility for not being more aggressive on this front. It’s nobody else’s fault but mine, and it’s embarrassing. We’re going to start kicking these people off right and left and making sure that when they issue their ridiculous attacks, nobody hears them. Everybody on the leadership team knows this is vital.’
More like this!(tags: trolls twitter gamergate dickc abuse leaks social-media)
-
nice deep-dive from Adrian Colyer
The DOs and DON’Ts of Blue/Green Deployment – CloudNative
Excellent post — Delta sounds like a very well-designed product
(tags: blue-green-deployments delta cloudnative ops deploy ec2 elb)
Can we have medical privacy, cloud computing and genomics all at the same time?
Today sees the publication of a report I [Ross Anderson] helped to write for the Nuffield Bioethics Council on what happens to medical ethics in a world of cloud-based medical records and pervasive genomics. As the information we gave to our doctors in private to help them treat us is now collected and treated as an industrial raw material, there has been scandal after scandal. From failures of anonymisation through unethical sales to the care.data catastrophe, things just seem to get worse. Where is it all going, and what must a medical data user do to behave ethically? We put forward four principles. First, respect persons; do not treat their confidential data like were coal or bauxite. Second, respect established human-rights and data-protection law, rather than trying to find ways round it. Third, consult people who’ll be affected or who have morally relevant interests. And fourth, tell them what you’ve done – including errors and security breaches.
(tags: ethics medicine health data care.data privacy healthcare ross-anderson genomics data-protection human-rights)
Comparing Message Queue Architectures on AWS
A good overview — I like the summary table. tl;dr:
If you are light on DevOps and not latency sensitive use SQS for job management and Kinesis for event stream processing. If latency is an issue, use ELB or 2 RabbitMQs (or 2 beanstalkds) for job management and Redis for event stream processing.
(tags: amazon architecture aws messaging queueing elb rabbitmq beanstalk kinesis sqs redis kafka)
TL;DR: Cassandra Java Huge Pages
Al Tobey does some trial runs of -XX:+AlwaysPreTouch and -XX:+UseHugePages
(tags: jvm performance tuning huge-pages vm ops cassandra java)
Enjoy Bintray and use it as pain-free gateway to Maven Central
ahh, interesting! This looks much easier (via JBaruch)
-
Marc Brooker: ‘When it comes to building working software in the long term, the emotional pursuit of craft is not as important as the human pursuit of teamwork, or the intellectual pursuit of correctness. Patterns is one of the most powerful ideas we have. The critics may be right that it devalues the craft, but we would all do well to remember that the craft of software is a means, not an end.’
(tags: marc-brooker design-patterns coding software teamwork)
-
Via Walter, the best description of the appeal of Minecraft I’ve read:
Minecraft is exceptionally good at intrinsic narrative. It recognises, preserves and rewards everything you do. It presses you to play frontiersman. A Minecraft world ends up dotted with torchlit paths, menhirs, landmarks, emergency caches. Here’s the hole where you dug stone for your first house. Here’s the causeway you built from your spawn point to a handy woodland. Here’s the crater in the landscape where the exploding monster took out you and your wheatfield at once. And, of course, here’s your enormous castle above a waterfall. There’s no utility in building anything bigger than a hut, but the temptations of architecture are irresistible. Minecraft isn’t so much a world generator as a screenshot-generator and a war-story generator. This is what will get the game the bulk of its critical attention, and deservedly so. That’s why I want to call attention to the extrinsic narrative. It’s minimal, implicit, accidental and very powerful. It’s this: you wake alone beside an endless sea in a pristine, infinite wilderness. The world is yours. You can literally sculpt mountains, with time and effort. You’ll die and be reborn on the beach where you woke first. You’ll walk across the world forever and never see another face. You can build a whole empire of roads and palaces and beacon towers, and the population of that empire will only ever be you. When you leave, your towers will stand empty forever. I haven’t seen that surfaced in a game before. It’s strong wine.
Backstage Blog – Prometheus: Monitoring at SoundCloud – SoundCloud Developers
whoa, this is pretty excellent. The major improvement over a graphite-based system would be the multi-dimensional tagging of metrics, which we currently have to do by simply expanding the graphite metric’s name to encompass all those dimensions and use searching at query time, inefficiently.
(tags: monitoring soundcloud prometheus metrics service-metrics graphite alerting)
‘Prometheus instrumentation library for JVM applications’
Good example of a clean java OSS release, from Soundcloud. will be copying bits of this myself soon…
(tags: prometheus java libraries oss github sonatype maven releases)
-
A good set of basic, controversy-free guidelines for clean java code style
(tags: style java google coding guidelines formatting coding-standards)
A Brief History of NSA Backdoors
from 1946 to present
(tags: nsa security backdoors sigint actel dual_ec_drbg crypto-ag crypto)
Study: You Can’t Change an Anti-Vaxxer’s Mind
According to a major new study in the journal ‘Pediatrics’, trying to [persuade anti-vaxxers to vaccinate] may actually make the problem worse. The paper tested the effectiveness of four separate pro-vaccine messages, three of which were based very closely on how the Centers for Disease Control and Prevention (CDC) itself talks about vaccines. The results can only be called grim: Not a single one of the messages was successful when it came to increasing parents’ professed intent to vaccinate their children. And in several cases the messages actually backfired, either increasing the ill-founded belief that vaccines cause autism or even, in one case, apparently reducing parents’ intent to vaccinate.
(tags: vaccination health measles mmr autism facts via:mrneutron stupidity cdc papers vaccines)
-
“dysaguria” is the perfect noun, and “dysagurian” is the perfect adjective, to describe the eponymous company in Dave Eggers’ The Circle. It’s not in the same league as Orwell, or Huxley, or Bradbury, or Burgess. But it does raise very important questions about what could possibly go wrong if one company controlled all the world’s information. In the novel, the company operates according to the motto “all that happens must be known”; and one of its bosses, Eamon Bailey, encourages everywoman employee Mae Holland to live an always-on (clear, transparent) life according the maxims “secrets are lies”, “sharing is caring”, and “privacy is theft”. Eggers’s debts to dystopian fiction are apparent. But, whereas writers like Orwell, Huxley, Bradbury, and Burgess were concerned with totalitarian states, Eggers is concerned with a totalitarian company. However, the noun “dystopia” and the adjective “dystopian” – perfect though they are for the terror of military/security authoritarianism in 1984, or Brave new World, or Farenheit 451, or A Clockwork Orange – do not to my mind encapsulate the nightmare of industrial/corporate tyranny in The Circle. On the other hand, “dysaguria” as a noun and “dysagurian” as an adjective, in my view really do capture the essence of that “frightening company”.
(tags: dysaguria dystopia future sf authoritarianism surveillance the-circle google facebook)
-
Via negatendo: ‘I would like to share my excitement about the fact that after almost a year of development, an instance of my NetHack bot has finally managed to ascend a game for the first time without human interventions, wizard mode cheats or bones stuffing, and did so at the public server at acehack.de.’ The bot is written in Clojure. Apparently ‘pudding farming’ did the trick…
(tags: clojure via:negatendo pudding-farming games nethack bots)
-
lol.
(tags: funny data-science statistics machine-learning hadoop bayes memes image-macros)
NA Server Roadmap Update: PoPs, Peering, and the North Bridge
League of Legends has set up private network links to a variety of major US ISPs to avoid internet weather (via Nelson)
(tags: via:nelson peering games networks internet ops networking)
-
Because there exists no method known to man, more terribly suited to expose the cosmic meaningless of existence than pairing the words of H.P. Lovecraft with seemingly delightful and charming pictures of adorable kittens.
(tags: lovecraft cthulhu horror funny kittens cats images gif)
8 gdb tricks you should know (Ksplice Blog)
These are very good — bookmarking for the next time I’m using gdb, probably about 3 years from now
EFF’s Game Plan for Ending Global Mass Surveillance
For years, we’ve been working on a strategy to end mass surveillance of digital communications of innocent people worldwide. Today we’re laying out the plan, so you can understand how all the pieces fit together—that is, how U.S. advocacy and policy efforts connect to the international fight and vice versa. Decide for yourself where you can get involved to make the biggest difference. This plan isn’t for the next two weeks or three months. It’s a multi-year battle that may need to be revised many times as we better understand the tools and authorities of entities engaged in mass surveillance and as more disclosures by whistleblowers help shine light on surveillance abuses.
(tags: eff privacy nsa surveillance gchq law policy us-politics)
-
This group aims to consolidate opposition, give clear information and support letter writing and information awareness against the Dept. of Education’s Primary Online Database.
(tags: pod ireland privacy data-protection children kids schools)
Apple Pay suffering fraud problems
Fraud in Apple Pay will in time, come to be managed – but the fact that easily available PII can waylay best in class protection should give us all pause.
(tags: fraud apple apple-pay pii identity-theft)
Excellent example of failed “anonymisation” of a dataset
Fred Logue notes how this failed Mayo TD Michelle Mulherin:
From recent reports it mow appears that the Department of Education is discussing anonymisation of the Primary Online Database with the Data Protection Commissioner. Well someone should ask Mayo TD Michelle Mulherin how anonymisation is working for her. The Sunday Times reports that Ms Mulherin was the only TD in the Irish parliament on the dates when expensive phone calls were made to a mobile number in Kenya. The details of the calls were released under the Freedom of Information Act in an “anonymised” database. While it must be said the fact that Ms Mulherin was the only TD present on those occasions does not prove she made the calls – the reporting in the press is now raising the possibility that it was her. From a data protection point of view this is a perfect example of the difficulty with anonymisation. Data protection rules apply to personal data which is defined as data relating to a living individual who is or can be identified from the data or from the data in conjunction with other information. Anonymisation is often cited as a means for processing data outside the scope of data protection law but as Ms Mulherin has discovered individuals can be identified using supposedly anonymised data when analysed in conjunction with other data. In the case of the mysterious calls to Kenya even though the released information was “anonymised” to protect the privacy of public representatives, the phone log used in combination with the attendance record of public representatives and information on social media was sufficient to identify individuals and at least raise evidence of association between individuals and certain phone calls. While this may be well and good in terms of accounting for abuses of the phone service it also has worrying implications for the ability of public representatives to conduct their business in private. The bottom line is that anonymisation is very difficult if not impossible as Ms Mulherin has learned to her cost. It certainly is a lot more complex than simply removing names and other identifying features from a single dataset. The more data that there is and the more diverse the sources the greater the risk that individuals can be identified from supposedly anonymised datasets.
(tags: data anonymisation fred-logue ireland michelle-mulherin tds kenya data-protection privacy)
Publishing from GitHub to Maven Central
A good starting point. This looks bloody complex :(
(tags: maven sonatype gradle jar open-source github release gpg)
-
Nice wrapper for ‘tc’ and ‘netem’, for network latency/packet loss emulation
(tags: networking testing linux tc netem latency packet-loss iptables)
Visualizing AWS Storage with Real-Time Latency Spectrogram
ohhhh this is very nice indeed. Great viz!
(tags: dataviz latency io ops sysdig charts graphs commandline linux)
Stop Playing Monopoly With Your Kids (And Play These Games Instead) | FiveThirtyEight
538 apply their numbercrunching skills to the BoardGameGeek ratings index
(tags: boardgames games kids children 538 statistics ratings)
ODROID-C1 – Multicore credit card computer
Pretty amazing specs for a 33 quid SBC.
Amlogic ARM® Cortex®-A5(ARMv7) 1.5Ghz quad core CPUs * Mali™-450 MP2 GPU (OpenGL ES 2.0/1.1 enabled for Linux and Android) * 1Gbyte DDR3 SDRAM * Gigabit Ethernet * 40pin GPIOs * eMMC4.5 HS200 Flash Storage slot / UHS-1 SDR50 MicroSD Card slot * USB 2.0 Host x 4, USB OTG x 1, * Infrared(IR) Receiver * Uses Ubuntu 14.04 or Android KitKat operating systems
Includes HDMI out. (via Conor O’Neill)(tags: via:conoro uk sbc hacking linux hardware odroid gadgets)
How TCP backlog works in Linux
good description of the process