Skip to content

Justin's Linklog Posts

Links for 2013-10-13

Published October 13, 2013

  • Reverse Engineering a D-Link Backdoor

    Using the correct User-Agent: string, all auth is bypassed on several released models of D-Link and Planex routers. Horrific fail by D-Link

    (tags: d-link security backdoors authorization reversing planex networking routers)

  • The US fears back-door routes into the net because it’s building them too | Technology | The Observer

    one of the most obvious inferences from the Snowden revelations published by the Guardian, New York Times and ProPublica recently is that the NSA has indeed been up to the business of inserting covert back doors in networking and other computing kit. The reports say that, in addition to undermining all of the mainstream cryptographic software used to protect online commerce, the NSA has been “collaborating with technology companies in the United States and abroad to build entry points into their products”. These reports have, needless to say, been strenuously denied by the companies, such as Cisco, that make this networking kit. Perhaps the NSA omitted to tell DARPA what it was up to? In the meantime, I hear that some governments have decided that their embassies should no longer use electronic communications at all, and are returning to employing couriers who travel the world handcuffed to locked dispatch cases. We’re back to the future, again.

    (tags: politics backdoors snowden snooping networking cisco nsa gchq)

  • Azerbaijan accidentally publishes the results of its election before the polls open

    The mistake came when an electoral commission accidentally published results showing a victory for Ilham Aliyev, the country’s long-standing President, a day before voting. Meydan TV, an online channel critical of the government, released a screenshot from a mobile app for the Azerbaijan Central Election Commission which showed that Mr Aliyev had received 72.76 per cent of the vote compared with 7.4 per cent for the opposition candidate, Jamil Hasanli. The screenshot also indicates that the app displayed information about how many people voted at various times during the day. Polls opened at 8am.

    (tags: azerbaijan corruption fix elections voting voter-fraud)

  • DNS registrars that complied with “shakedown” anti-piracy requests may now be in violation of ICANN Transfers Policy

    According to EasyDNS:

    Any registrar that has taken one of these sites offline that now impedes the registrants of those domains from simply getting their domain names out of there and back online somewhere else will then be subject to the TDRP – Transfer Dispute Resolution Policy and if they lose (which they will) they will be subject to TDRP fees assesed by the registry operator, and to quote the TDRP itself “Transfer dispute resolution fees can be substantial”. This is why it is never a good idea to just react to pressure in the face of obnoxious bluster – in the very act of trying to diffuse any perceived culpability you end up opening yourself to real liability.

    (tags: tdrp easydns dns registrars domains piracy law due-process)

  • Schneier on Security: Air Gaps

    interesting discussion in the comments. “Patricia”‘s process is particularly hair-raisingly complex, involving 3 separate machines and a multitude of VMs

    (tags: air-gaps security networking bruce-schneier via:adulau)

Links for 2013-10-11

Published October 11, 2013

Links for 2013-10-10

Published October 10, 2013

Links for 2013-10-09

Published October 9, 2013

  • pt-summary

    from the Percona toolkit. ‘Conveniently summarizes the status and configuration of a server. It is not a tuning tool or diagnosis tool. It produces a report that is easy to diff and can be pasted into emails without losing the formatting. This tool works well on many types of Unix systems.’ — summarises OOM history, top, netstat connection table, interface stats, network config, RAID, LVM, disks, inodes, disk scheduling, mounts, memory, processors, and CPU.

    (tags: percona tools cli unix ops linux diagnosis raid netstat oom)

  • How much can an extra hour’s sleep change you?

    What they discovered is that when the volunteers cut back from seven-and-a-half to six-and-a-half hours’ sleep a night, genes that are associated with processes like inflammation, immune response and response to stress became more active. The team also saw increases in the activity of genes associated with diabetes and risk of cancer. The reverse happened when the volunteers added an hour of sleep.

    (tags: sleep health rest cancer bbc science)

  • Kovet

    some great phone cases from an Irish company, with nifty art by Irish illustrators and artists including Fatti Burke and Chris Judge

    (tags: chris-judge fatti-burke illustrators art ireland iphone cases)

Links for 2013-10-08

Published October 8, 2013

  • What drives JVM full GC duration

    Interesting empirical results using JDK 7u21:

    Full GC duration depends on the number of objects allocated and the locality of their references. It does not depend that much on actual heap size.
    Reference locality has a surprisingly high effect.

    (tags: java jvm data gc tuning performance cms g1)

  • Rhizome | Occupy.here: A tiny, self-contained darknet

    Occupy.here began two years ago as an experiment for the encampment at Zuccotti Park. It was a wifi router hacked to run OpenWrt Linux (an operating system mostly used for computer networking) and a small “captive portal” website. When users joined the wifi network and attempted to load any URL, they were redirected to http://occupy.here. The web software offered up a simple BBS-style message board providing its users with a space to share messages and files.
    Nifty project from Dan Phiffer.

    (tags: occupy.here openwrt hacking wifi network community)

  • Whatever Happened to “Due Process” ?

    Mark Jeftovic is on fire after receiving yet another “take down this domain or else” mail from the City of London police:

    We have an obligation to our customers and we are bound by our Registrar Accreditation Agreements not to make arbitrary changes to our customers settings without a valid FOA (Form of Authorization). To supersede that we need a legal basis. To get a legal basis something has to happen in court. […] What gets me about all of this is that the largest, most egregious perpetrators of online criminal activity right now are our own governments, spying on their own citizens, illegally wiretapping our own private communications and nobody cares, nobody will answer for it, it’s just an out-of-scope conversation that is expected to blend into the overall background malaise of our ever increasing serfdom. If I can’t make various governments and law enforcement agencies get warrants or court orders before they crack my private communications then I can at least require a court order before I takedown my own customer.

    (tags: city-of-london police takedowns politics mark-jeftovic easydns registrars dns via:tjmcintyre)

  • Intellectual Ventures’ Evil Knows No Bounds: Buys Patent AmEx Donated For Public Good… And Starts Suing

    The problem with software patents, part XVII.

    So you have a situation where even when the original patent holder donated the patent for “the public good,” sooner or later, an obnoxious patent troll like IV comes along and turns it into a weapon. Again: AmEx patented those little numbers on your credit card, and then for the good of the industry and consumer protection donated the patent to a non-profit, who promised not to enforce the patent against banks… and then proceeded to sell the patent to Intellectual Ventures who is now suing banks over it.

    (tags: intellectual-ventures scams patents swpats shakedown banking cvv american-express banks amex cmaf)

Links for 2013-10-07

Published October 7, 2013

the coming world of automated mass anti-terror false positives

Published October 6, 2013

Man sues RMV after driver’s license mistakenly revoked by automated anti-terror false positive:

John H. Gass hadn’t had a traffic ticket in years, so the Natick resident was surprised this spring when he received a letter from the Massachusetts Registry of Motor Vehicles informing him to cease driving because his license had been revoked. […] After frantic calls and a hearing with Registry officials, Gass learned the problem: An antiterrorism computerized facial recognition system that scans a database of millions of state driver’s license images had picked his as a possible fraud. “We send out 1,500 suspension letters every day,” said Registrar Rachel Kaprielian. […] “There are mistakes that can be made.”

See also this New Scientist story. This story notes that the system’s pretty widespread:

Massachusetts bought the system with a $1.5 million grant from the Department of Homeland Security. At least 34 states use such systems, which law enforcement officials say help prevent identity theft and ID fraud.

In my opinion, this kind of thing — trial by inaccurate, false-positive-prone algorithm, is one of the most worrying things about the post-PRISM world.

When we created SpamAssassin, we were well aware of the risk of automated misclassification. Any machine-learning classifier will always make mistakes. The key is to carefully calibrate the expected false-positive/false-negative ratio so that the negative side-effects of a misclassification corresponds to the expected rate.

These anti-terrorism machine learning systems are calibrated to catch as many potential cases as possible, but by aiming to reduce false negatives to this degree, they become wildly prone to false positives. And when they’re applied as a dragnet across all citizens’ interactions with the state — or even in the case of PRISM, all citizens’ interactions that can be surveilled en masse — it’s going to create buckets of bureaucratic false-positive horror stories, as random innocent citizens are incorrectly tagged as criminals due to software bugs and poor calibration.

Links for 2013-10-04

Published October 4, 2013

Links for 2013-10-03

Published October 3, 2013

Links for 2013-10-02

Published October 2, 2013

Links for 2013-10-01

Published October 1, 2013

  • How an Engineer Earned 1.25 Million Air Miles By Buying Pudding

    An amazing hack. ‘Air Miles are awesome, they can be used to score free flights, hotel stays and if you’re really lucky, the scorn and hatred of everyone you come in contact with who has to pay full price when they travel. The king of all virtually free travelers is one David Phillips, a civil engineer who teaches at the University of California, Davis. David came to the attention of the wider media when he managed to convert about 12,150 cups of Healthy Choice chocolate pudding [costing $3000] into over a million Air Miles. Ever since, David and his entire family have been travelling the world for next to nothing.’ (via al3xandru)

    (tags: via:al3xandru hacks cool pudding small-print air-miles free)

  • Down the Rabbit Hole

    An adventure that takes you through several popular Java language features and shows how they compile to bytecode and eventually JIT to assembly code.

    (tags: charles-nutter java jvm compilation reversing talks slides)

Links for 2013-09-30

Published September 30, 2013

  • Model checking for highly concurrent code

    Applied formal methods in order to test distributed systems — specifically GlusterFS:

    I’ll use an example from my own recent experience. I’m developing a new kind of replication for GlusterFS. To make sure the protocol behaves correctly even across multiple failures, I developed a Murphi model for it. […] I added a third failure [to the simulated model]. I didn’t expect a three-node system to continue working if more than one of those were concurrent (the model allows the failures to be any mix of sequential and concurrent), but I expected it to fail cleanly without reaching an invalid state. Surprise! It managed to produce a case where a reader can observe values that go back in time. This might not make much sense without knowing the protocol involved, but it might give some idea of the crazy conditions a model checker will find that you couldn’t possibly have considered. […] So now I have a bug to fix, and that’s a good thing. Clearly, it involves a very specific set of ill-timed reads, writes, and failures. Could I have found it by inspection or ad-hoc analysis? Hell, no. Could I have found it by testing on live systems? Maybe, eventually, but it probably would have taken months for this particular combination to occur on its own. Forcing it to occur would require a lot of extra code, plus an exerciser that would amount to a model checker running 100x slower across machines than Murphi does. With enough real deployments over enough time it would have happened, but the only feasible way to prevent that was with model checking. These are exactly the kinds of bugs that are hardest to fix in the field, and that make users distrust distributed systems, so those of us who build such systems should use every tool at our disposal to avoid them.

    (tags: model-checking formal-methods modelling murphi distcomp distributed-systems glusterfs testing protocols)

  • Is Trypophobia a Real Phobia? | Popular Science

    ie. “fear of small, clustered holes”. Sounds like it’s not so much a “phobia” as some kind of innate, visceral disgust response; I get it. ‘As for who actually made the word up, that distinction probably belongs to a blogger in Ireland named Louise, Andrews says. According to an archived Geocities page, Louise settled on “trypophobia” (Greek for “boring holes” + “fear”) after corresponding with a representative at the Oxford English Dictionary. Louise, Andrews and trypophobia Facebook group members have petitioned the dictionary to include the word. The term will need to be used for years and have multiple petitions and scholarly references before the dictionary accepts it, Andrews says. I, for one, would prefer to forget about it forever.’

    (tags: disgusting revulsion fear phobias trypophobia holes ugh innate)

  • Common phobia you have never heard of: Fear of holes may stem from evolutionary survival response

    “We think that everyone has trypophobic tendencies even though they may not be aware of it,” said Dr Cole. “We found that people who don’t have the phobia still rate trypophobic images as less comfortable to look at than other images. It backs up the theory that we are set-up to be fearful of things which hurt us in our evolutionary past. We have an innate predisposition to be wary of things that can harm us.”

    (tags: trypophobia holes fear aversion disgust ugh evolution innate)

Links for 2013-09-26

Published September 26, 2013

Links for 2013-09-25

Published September 25, 2013

Links for 2013-09-24

Published September 24, 2013

Links for 2013-09-23

Published September 23, 2013

Links for 2013-09-22

Published September 22, 2013

  • RSA warns developers not to use RSA products

    In case you’re missing the story here, Dual_EC_DRBG (which I wrote about yesterday) is the random number generator voted most likely to be backdoored by the NSA. The story here is that — despite many valid concerns about this generator — RSA went ahead and made it the default generator used for all cryptography in its flagship cryptography library. The implications for RSA and RSA-based products are staggering. In a modestly bad but by no means worst case, the NSA may be able to intercept SSL/TLS connections made by products implemented with BSafe.

    (tags: bsafe rsa crypto backdoors nsa security dual_ec_drbg rngs randomness)

  • A Case Against Cucumber

    This is exactly my problem with Cucumber and similar BDD test frameworks.

    When I write a Cucumber feature, I have to write the Gherkin that describes the acceptance criteria, and the Ruby code that implements the step definitions. Since the code to implement the step definitions is just normal RSpec (or whichever testing library you use), if someone else is writing the Gherkin, the amount of setup to create a working test should be about the same. So you’re only breaking even! However, I don’t believe that it would really be breaking even. Cucumber adds another layer of indirection on top of your tests. When I’m trying to see why a specific scenario is failing, first I need to find the step that is failing. Since these steps are defined with regular expressions, I have to grep for the step definition.

    (tags: ruby testing bdd cucumber rspec coding)

  • Gamasutra – Opinion: The tragedy of Grand Theft Auto V

    This is watching your sharp, witty father start telling old fart jokes as his mind slows down. And as much as the internet is habituated to defending GTA as “satire,” what is it satirizing, if everything is either sad or awful? Where is the “satire” when the awful parts no longer seem edgy or provocative, just attempts at catch-all “offense” that aren’t honed enough to even connect? Here’s a series that has been creating real, meaningful friction with conventional entertainment for as long as I can remember, and rather than push the envelope by creating new kinds of monsters, it’s reciting the same old gangland fantasies, like a college boy who can’t stop staring at the Godfather II poster on his wall, talking about how he’s gonna be a big Hollywood director in between bong rips. You call the trading index BAWSAQ? Oh, bro, you’re so funny, you’re gonna be huge.

    (tags: gamasutra games gaming gta gta-v via:skamille)

  • CCC | Chaos Computer Club breaks Apple TouchID

    “We hope that this finally puts to rest the illusions people have about fingerprint biometrics. It is plain stupid to use something that you can´t change and that you leave everywhere every day as a security token”, said Frank Rieger, spokesperson of the CCC. “The public should no longer be fooled by the biometrics industry with false security claims. Biometrics is fundamentally a technology designed for oppression and control, not for securing everyday device access.” iPhone users should avoid protecting sensitive data with their precious biometric fingerprint not only because it can be easily faked, as demonstrated by the CCC team. Also, you can easily be forced to unlock your phone against your will when being arrested. Forcing you to give up your (hopefully long) passcode is much harder under most jurisdictions than just casually swiping your phone over your handcuffed hands.

Links for 2013-09-18

Published September 18, 2013

Links for 2013-09-17

Published September 17, 2013

  • To solve hard problems, you need to use bricolage

    In a talk about a neat software component he designed, Bruce Haddon observed that there is no way that the final structure and algorithmic behavior of this component could have been predicted, designed, or otherwise anticipated. Haddon observed that computer science serves as a source of core ideas: it provides the data structures and algorithms that are the building blocks. Meanwhile, he views software engineering as a useful set of methods to help design reliable software without losing your mind. Yet he points out that neither captures the whole experience. That’s because much of the work is what Haddon calls hacking, but what others would call bricolage. Simply put, there is much trial and error: we put ideas to together and see where it goes.
    This is a great post, and I agree (broadly). IMO, most software engineering requires little CS, but there are occasional moments where a single significant aspect of a project requires a particular algorithm, and would be kludgy, hacky, or over-complex to solve without it.

    (tags: bricolage hacking cs computer-science work algorithms)

  • Getting Real About Distributed System Reliability

    I have come around to the view that the real core difficulty of [distributed] systems is operations, not architecture or design. Both are important but good operations can often work around the limitations of bad (or incomplete) software, but good software cannot run reliably with bad operations. This is quite different from the view of unbreakable, self-healing, self-operating systems that I see being pitched by the more enthusiastic NoSQL hypesters. Worse yet, you can’t easily buy good operations in the same way you can buy good software—you might be able to hire good people (if you can find them) but this is more than just people; it is practices, monitoring systems, configuration management, etc.

    (tags: reliability nosql distributed-systems jay-kreps ops)

  • Don’t use Hadoop – your data isn’t that big

    see also HN comments: https://news.ycombinator.com/item?id=6398650 , particularly davidmr’s great one:

    I suppose all of this is to say that the amount of required parallelization of a problem isn’t necessarily related to the size of the problem set as is mentioned most in the article, but also the inherent CPU and IO characteristics of the problem. Some small problems are great for large-scale map-reduce clusters, some huge problems are horrible for even bigger-scale map-reduce clusters (think fluid dynamics or something that requires each subdivision of the problem space to communicate with its neighbors). I’ve had a quote printed on my door for years: Supercomputers are an expensive tool for turning CPU-bound problems into IO-bound problems.
    I love that quote!

    (tags: hadoop big-data scaling map-reduce)

  • Gilt Tech

    Gilt ran a stress-test of Riak to replace Voldemort (I think) in a shadow stack, with good results:

    Riak’s strong performance suggests that, should we pursue implementation, it will withstand our unique traffic needs and prove reliable. As for the Gilt-Basho team’s strong performance: It was amazing that we were able to accomplish so much in just a week’s time! Thanks again to Seth and Steve for making this possible.

    (tags: riak testing shadow-stack voldemort storage gilt)

  • THE LONG DARK, a first-person post-disaster survival sim by Hinterland — Kickstarter

    wow this looks great.

    The Long Dark is a thoughtful, first-person survival simulation that emphasizes quiet exploration in a stark, yet hauntingly beautiful, post-disaster setting. The breathtakingly picturesque Pacific Northwest frames the backdrop for the drama of The Long Dark.

    (tags: games survival via:fp eclaire the-long-dark kickstarter)

  • The Rational Choices of Crack Addicts – NYTimes.com

    “The key factor is the environment, whether you’re talking about humans or rats,” Dr. Hart said. “The rats that keep pressing the lever for cocaine are the ones who are stressed out because they’ve been raised in solitary conditions and have no other options. But when you enrich their environment, and give them access to sweets and let them play with other rats, they stop pressing the lever.”

    (tags: crack drugs policy science addiction society)

Links for 2013-09-16

Published September 16, 2013

Links for 2013-09-15

Published September 15, 2013

  • Being poor changes your thinking about everything

    Very interesting research into poverty and scarcity, in the Washington Post:

    The scarcity trap captures this notion we see again and again in many domains. When people have very little, they undertake behaviors that maintain or reinforce their future disadvantage. If you have very little, you often behave in such a way so that you’ll have little in the future. In economics, people talk about the poverty trap. We’re generalizing that, saying this happens a lot, and we’ve experienced it.

    (tags: poor poverty society economics scarcity washington-post)

  • Good SSL for your website is absurdly difficult in practice

    Yet again, security software fails on packaging and UI. via Tony Finch

    (tags: security ssl tls packaging via:fanf)

  • Former NSA and CIA director says terrorists love using Gmail

    At one point, Hayden expressed a distaste for online anonymity, saying “The problem I have with the Internet is that it’s anonymous.” But he noted, there is a struggle over that issue even inside government. The issue came to a head during the Arab Spring movement when the State Department was funding technology [presumably Tor?] to protect the anonymity of activists so governments could not track down or repress their voices. “We have a very difficult time with this,” Hayden said. He then asked, “is our vision of the World Wide Web the global digital commons — at this point you should see butterflies flying here and soft background meadow-like music — or a global free fire zone?” Given that Hayden also compared the Internet to the wild west and Somalia, Hayden clearly leans toward the “global free fire zone” vision of the Internet.
    well, that’s a good analogy for where we’re going — a global free-fire zone.

    (tags: gmail cia nsa surveillance michael-hayden security snooping law tor arab-spring)

Links for 2013-09-14

Published September 14, 2013

  • Google swaps out MySQL, moves to MariaDB

    When we asked Sallner to quantify the scale of the migration he said, “They’re moving it all. Everything they have. All of the MySQL servers are moving to MariaDB, as far as I understand.” By moving to MariaDB, Google can free itself of any dependence on technology dictated by Oracle – a company whose motivations are unclear, and whose track record for working with the wider technology community is dicey, to say the least. Oracle has controlled MySQL since its acquisition of Sun in 2010, and the key InnoDB storage engine since it got ahold of Innobase in 2005. […] We asked Cole why Google would shift from MySQL to MariaDB, and what the key technical differences between the systems were. “From my perspective, they’re more or less equivalent other than if you look at specific features and how they implement them,” Cole said, speaking in a personal capacity and not on behalf of Google. “Ideologically there are lots of differences.”
    So — AWS, when will RDS offer MariaDB as an option?

    (tags: google mysql mariadb sql open-source licensing databases storage innodb oracle)

  • FBI Admits It Controlled Tor Servers Behind Mass Malware Attack

    The code’s behavior, and the command-and-control server’s Virginia placement, is also consistent with what’s known about the FBI’s “computer and internet protocol address verifier,” or CIPAV, the law enforcement spyware first reported by WIRED in 2007. Court documents and FBI files released under the FOIA have described the CIPAV as software the FBI can deliver through a browser exploit to gather information from the target’s machine and send it to an FBI server in Virginia. The FBI has been using the CIPAV since 2002 against hackers, online sexual predators, extortionists, and others, primarily to identify suspects who are disguising their location using proxy servers or anonymity services, like Tor. Prior to the Freedom Hosting attack, the code had been used sparingly, which kept it from leaking out and being analyzed.

    (tags: cipav fbi tor malware spyware security wired)

  • Creating Flight Recordings

    lots more detail on the new “Java Mission Control” feature in Hotspot 7u40 JVMs, and how to use it to start and stop profiling in a live, production JVM from a separate “jcmd” command-line client. If the overhead is small, this could be really neat — turn on profiling for 1 minute every hour on a single instance, and collect realtime production profile data on an automated basis for post-facto analysis if required

    (tags: instrumentation logging profiling java jvm ops)

Links for 2013-09-12

Published September 12, 2013

  • Necessary and Proportionate — In Which Civil Society is Caught Between a Cop and a Spy

    Modern telecommunications technology implied the development of modern telecommunications surveillance, because it moved the scope of action from the physical world (where intelligence, generally seen as part of the military mission, had acted) to the virtual world—including the scope of those actions that could threaten state power. While the public line may have been, as US Secretary of State Henry Stimson said in 1929, “gentlemen do not open each other’s mail”, you can bet that they always did keep a keen eye on the comings and goings of each other’s shipping traffic. The real reason that surveillance in the context of state intelligence was limited until recently was because it was too expensive, and it was too expensive for everyone. The Westphalian compromise demands equality of agency as tied to territory. As soon as one side gains a significant advantage, the structure of sovereignty itself is threatened at a conceptual level?—?hence Oppenheimer as the death of any hope of international rule of law. Once surveillance became cheap enough, all states were (and will increasingly be) forced to attempt it at scale, as a reaction to this pernicious efficiency. The US may be ahead of the game now, but Moore’s law and productization will work their magic here.

    (tags: government telecoms snooping gchq nsa surveillance law politics intelligence spying internet)

Links for 2013-09-11

Published September 11, 2013

  • Observability at Twitter

    Bit of detail into Twitter’s TSD metric store.

    There are separate online clusters for different data sets: application and operating system metrics, performance critical write-time aggregates, long term archives, and temporal indexes. A typical production instance of the time series database is based on four distinct Cassandra clusters, each responsible for a different dimension (real-time, historical, aggregate, index) due to different performance constraints. These clusters are amongst the largest Cassandra clusters deployed in production today and account for over 500 million individual metric writes per minute. Archival data is stored at a lower resolution for trending and long term analysis, whereas higher resolution data is periodically expired. Aggregation is generally performed at write-time to avoid extra storage operations for metrics that are expected to be immediately consumed. Indexing occurs along several dimensions–service, source, and metric names–to give users some flexibility in finding relevant data.

    (tags: twitter monitoring metrics service-metrics tsd time-series storage architecture cassandra)

  • NSA: Possibly breaking US laws, but still bound by laws of computational complexity

    I didn’t clearly explain that there’s an enormous continuum between, on the one hand, a full break of RSA or Diffie-Hellman (which still seems extremely unlikely to me), and on the other, “pure side-channel attacks” involving no new cryptanalytic ideas.  Along that continuum, there are many plausible places where the NSA might be.  For example, imagine that they had a combination of side-channel attacks, novel algorithmic advances, and sheer computing power that enabled them to factor, let’s say, ten 2048-bit RSA keys every year.  In such a case, it would still make perfect sense that they’d want to insert backdoors into software, sneak vulnerabilities into the standards, and do whatever else it took to minimize their need to resort to such expensive attacks.  But the possibility of number-theoretic advances well beyond what the open world knows certainly wouldn’t be ruled out.  Also, as Schneier has emphasized, the fact that NSA has been aggressively pushing elliptic-curve cryptography in recent years invites the obvious speculation that they know something about ECC that the rest of us don’t.

    (tags: ecc rsa crypto security nsa gchq snooping sniffing diffie-hellman pki key-length)

  • Low Overhead Method Profiling with Java Mission Control now enabled in the most recent HotSpot JVM release

    Built into the HotSpot JVM [in JDK version 7u40] is something called the Java Flight Recorder. It records a lot of information about/from the JVM runtime, and can be thought of as similar to the Data Flight Recorders you find in modern airplanes. You normally use the Flight Recorder to find out what was happening in your JVM when something went wrong, but it is also a pretty awesome tool for production time profiling. Since Mission Control (using the default templates) normally don’t cause more than a per cent overhead, you can use it on your production server.
    I’m intrigued by the idea of always-on profiling in production. This could be cool.

    (tags: performance java measurement profiling jvm jdk hotspot mission-control instrumentation telemetry metrics)

Links for 2013-09-09

Published September 9, 2013

  • How the NSA Spies on Smartphones

    One of the US agents’ tools is the use of backup files established by smartphones. According to one NSA document, these files contain the kind of information that is of particular interest to analysts, such as lists of contacts, call logs and drafts of text messages. To sort out such data, the analysts don’t even require access to the iPhone itself, the document indicates. The department merely needs to infiltrate the target’s computer, with which the smartphone is synchronized, in advance. Under the heading “iPhone capability,” the NSA specialists list the kinds of data they can analyze in these cases. The document notes that there are small NSA programs, known as “scripts,” that can perform surveillance on 38 different features of the iPhone 3 and 4 operating systems. They include the mapping feature, voicemail and photos, as well as the Google Earth, Facebook and Yahoo Messenger applications.
    and, of course, the alternative means of backup is iCloud…. wonder how secure those backups are.

    (tags: nsa surveillance gchq iphone smartphones backups icloud security)

  • Behind the Screens at Loggly

    Boost ASIO at the front end (!), Kafka 0.8, Storm, and ElasticSearch

    (tags: boost scalability loggly logging ingestion cep stream-processing kafka storm architecture elasticsearch)

  • Schneier on Security: Excess Automobile Deaths as a Result of 9/11

    The inconvenience of extra passenger screening and added costs at airports after 9/11 cause many short-haul passengers to drive to their destination instead, and, since airline travel is far safer than car travel, this has led to an increase of 500 U.S. traffic fatalities per year. Using DHS-mandated value of statistical life at $6.5 million, this equates to a loss of $3.2 billion per year, or $32 billion over the period 2002 to 2011 (Blalock et al. 2007).

    (tags: risk security death 9-11 politics screening dhs air-travel driving road-safety)

Links for 2013-09-08

Published September 8, 2013

  • Perhaps I’m out of step and Britons just don’t think privacy is important | Henry Porter | Comment is free | The Observer

    The debate has been stifled in Britain more successfully than anywhere else in the free world and, astonishingly, this has been with the compliance of a media and public that regard their attachment to liberty to be a matter of genetic inheritance. So maybe it is best for me to accept that the BBC, together with most of the newspapers, has moved with society, leaving me behind with a few old privacy-loving codgers, wondering about the cause of this shift in attitudes. Is it simply the fear of terror and paedophiles? Are we so overwhelmed by the power of the surveillance agencies that we feel we can’t do anything? Or is it that we have forgotten how precious and rare truly free societies are in history?

    (tags: privacy uk politics snooping spies gchq society nsa henry-porter)

  • Big data is watching you

    Some great street art from Brighton, via Darach Ennis

    (tags: via:darachennis street-art graffiti big-data snooping spies gchq nsa art)

  • Blocking The Pirate Bay appears to have ‘no lasting net impact’ on illegal downloading

    In the fight against the unauthorised sharing of copyright protected material, aka piracy, Dutch Internet Service Providers have been summoned by courts to block their subscribers’ access to The Pirate Bay (TPB) and related sites. This paper studies the effectiveness of this approach towards online copyright enforcement, using both a consumer survey and a newly developed non-infringing technology for BitTorrent monitoring. While a small group of respondents download less from illegal sources or claim to have stopped, and a small but significant effect is found on the distribution of Dutch peers, no lasting net impact is found on the percentage of the Dutch population downloading from illegal sources.

    (tags: fail blocking holland pirate-bay tpb papers via:tjmcintyre internet isps)

  • How Advanced Is the NSA’s Cryptanalysis — And Can We Resist It?

    Bruce Schneier’s suggestions:

    Assuming the hypothetical NSA breakthroughs don’t totally break public-cryptography — and that’s a very reasonable assumption — it’s pretty easy to stay a few steps ahead of the NSA by using ever-longer keys. We’re already trying to phase out 1024-bit RSA keys in favor of 2048-bit keys. Perhaps we need to jump even further ahead and consider 3072-bit keys. And maybe we should be even more paranoid about elliptic curves and use key lengths above 500 bits. One last blue-sky possibility: a quantum computer. Quantum computers are still toys in the academic world, but have the theoretical ability to quickly break common public-key algorithms — regardless of key length — and to effectively halve the key length of any symmetric algorithm. I think it extraordinarily unlikely that the NSA has built a quantum computer capable of performing the magnitude of calculation necessary to do this, but it’s possible. The defense is easy, if annoying: stick with symmetric cryptography based on shared secrets, and use 256-bit keys.

    (tags: bruce-schneier cryptography wired nsa surveillance snooping gchq cryptanalysis crypto future key-lengths)

  • DevOps Eye for the Coding Guy: Metrics

    a pretty good description of the process of adding service metrics to a Django webapp using graphite and statsd. Bookmarking mainly for the great real-time graphing hack at the end…

    (tags: statsd django monitoring metrics python graphite)

  • Probabalistic Scraping of Plain Text Tables

    a nifty hack.

    Recently I have been banging my head trying to import a ton of OCR acquired data expressed in tabular form. I think I have come up with a neat approach using probabilistic reasoning combined with mixed integer programming. The method is pretty robust to all sorts of real world issues. In particular, the method leverages topological understanding of tables, encodes it declaratively into a mixed integer/linear program, and integrates weak probabilistic signals to classify the whole table in one go (at sub second speeds). This method can be used for any kind of classification where you have strong logical constraints but noisy data.
    (via proggit)

    (tags: scraping tables ocr probabilistic linear-programming optimization machine-learning via:proggit)

  • vimeo/timeserieswidget

    ‘Plugin to make highly interactive graphite graph objects ((i.e. graphs where you can interactively toggle on/off individual series, inspect datapoints, zoom in realtime, etc) Supports Flot (canvas), Rickshaw (svg) and standard graphite png images (in case you’re nostalgic and don’t like interactivity).’

    (tags: graphs graphing graphite dataviz flot rickshaw svg canvas javascript)

Links for 2013-09-05

Published September 5, 2013

Links for 2013-09-04

Published September 4, 2013

  • How To Buffer Full YouTube Videos Before Playing

    summary – turn off DASH (Dynamic adaptive streaming) using a userscript.

    (tags: chrome youtube google video dash mpeg streaming)

  • Voldemort on Solid State Drives [paper]

    ‘This paper and talk was given by the LinkedIn Voldemort Team at the Workshop on Big Data Benchmarking (WBDB May 2012).’

    With SSD, we find that garbage collection will become a very significant bottleneck, especially for systems which have little control over the storage layer and rely on Java memory management. Big heapsizes make the cost of garbage collection expensive, especially the single threaded CMS Initial mark. We believe that data systems must revisit their caching strategies with SSDs. In this regard, SSD has provided an efficient solution for handling fragmentation and moving towards predictable multitenancy.

    (tags: voldemort storage ssd disk linkedin big-data jvm tuning ops gc)

Links for 2013-09-03

Published September 3, 2013

  • Streaming MapReduce with Summingbird

    Before Summingbird at Twitter, users that wanted to write production streaming aggregations would typically write their logic using a Hadoop DSL like Pig or Scalding. These tools offered nice distributed system abstractions: Pig resembled familiar SQL, while Scalding, like Summingbird, mimics the Scala collections API. By running these jobs on some regular schedule (typically hourly or daily), users could build time series dashboards with very reliable error bounds at the unfortunate cost of high latency. While using Hadoop for these types of loads is effective, Twitter is about real-time and we needed a general system to deliver data in seconds, not hours. Twitter’s release of Storm made it easy to process data with very low latencies by sacrificing Hadoop’s fault tolerant guarantees. However, we soon realized that running a fully real-time system on Storm was quite difficult for two main reasons: Recomputation over months of historical logs must be coordinated with Hadoop or streamed through Storm with a custom log loading mechanism; Storm is focused on message passing and random-write databases are harder to maintain. The types of aggregations one can perform in Storm are very similar to what’s possible in Hadoop, but the system issues are very different. Summingbird began as an investigation into a hybrid system that could run a streaming aggregation in both Hadoop and Storm, as well as merge automatically without special consideration of the job author. The hybrid model allows most data to be processed by Hadoop and served out of a read-only store. Only data that Hadoop hasn’t yet been able to process (data that falls within the latency window) would be served out of a datastore populated in real-time by Storm. But the error of the real-time layer is bounded, as Hadoop will eventually get around to processing the same data and will smooth out any error introduced. This hybrid model is appealing because you get well understood, transactional behavior from Hadoop, and up to the second additions from Storm. Despite the appeal, the hybrid approach has the following practical problems: Two sets of aggregation logic have to be kept in sync in two different systems; Keys and values must be serialized consistently between each system and the client. The client is responsible for reading from both datastores, performing a final aggregation and serving the combined results Summingbird was developed to provide a general solution to these problems.
    Very interesting stuff. I’m particularly interested in the design constraints they’ve chosen to impose to achieve this — data formats which require associative merging in particular.

    (tags: mapreduce streaming big-data twitter storm summingbird scala pig hadoop aggregation merging)

  • Thoughts on Granby Park, the recent pop-up park off Parnell St

    We mentioned above that pop-up spaces have become popular across Europe because they allow developers and city councils to harness urban creativity in order to drive up real estate prices without ceding control of a given site. Those who produce the space through hard work, collaboration and passion move on, making way for property development and speculation. The international research in this area is very clear on this point and it has been documented in places from Lower-East Side Manhattan to Berlin’s Kreuzberg. Most perversely, increased property prices make it even more difficult for creativity to flourish in a given area and end up driving out long-term working class communities, migrants and young people. But what can we do? If every attempt we make to make our city a better place simply ends up being captured in the calculations of real estate players, surely the situation is hopeless? Is it better, then, to do nothing? We don’t think it is better to do nothing and, like Upstart, we still believe we can find a way together through experimentation and collaboration. However, this means questioning, reflecting on and publicly discussing the relationship between our efforts to make a city more after our hearts desire and the process of gentrification. As noted above, this is especially the case with pop-up spaces given their temporary nature. It is really necessary that we think about how to make sure our activities don’t contribute to gentrification in the long term, but instead benefit the city as a whole. We certainly don’t have the solutions, but if we sweep these awkward questions under the carpet we risk contributing to the very forces we want to challenge and alienating those who will perceive us as the ‘front-line’ of gentrification.

    (tags: gentrification pop-up parks dublin ireland cities upstart spaces urban-planning)

  • [#CASSANDRA-5582] Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor

    Disruptor: decimating P99s since 2011

    (tags: disruptor cassandra java p99 latency speed performance concurrency via:kellabyte)

  • Time is a Dimension

    I love these.

    Photographic prints are great because they don’t need power to be displayed. They are more or less permanent. Videos are great because they record a sequence of time which shows reality almost like how we experience. Is it possible to combine the two? And not via long exposure photography where often details are lost from motion. So I played around with the tools of digital photography and post processing to give you this series: Time is a dimension. This series of images are mostly landscapes, seascapes and cityscapes, and they are a single composite made from sequences that span 2-4 hours, mostly of sunrises and sunsets. The basic structure of a landscape is present in every piece. But each panel or concentric layer shows a different slice of time, which is related to the adjacent panel/layer. The transition from daytime to night is gradual and noticeable in every piece, but would not be something you expect to see in a still image.

    (tags: photography beautiful photos art time dimensions prints via:matthaughey)

Links for 2013-09-02

Published September 2, 2013

Links for 2013-08-31

Published August 31, 2013

  • Reversing Sinclair’s amazing 1974 calculator hack – half the ROM of the HP-35

    Amazing reverse engineering.

    In a hotel room in Texas, Clive Sinclair had a big problem. He wanted to sell a cheap scientific calculator that would grab the market from expensive calculators such as the popular HP-35. Hewlett-Packard had taken two years, 20 engineers, and a million dollars to design the HP-35, which used 5 complex chips and sold for $395. Sinclair’s partnership with calculator manufacturer Bowmar had gone nowhere. Now Texas Instruments offered him an inexpensive calculator chip that could barely do four-function math. Could he use this chip to build a $100 scientific calculator? Texas Instruments’ engineers said this was impossible – their chip only had 3 storage registers, no subroutine calls, and no storage for constants such as ?. The ROM storage in the calculator held only 320 instructions, just enough for basic arithmetic. How could they possibly squeeze any scientific functions into this chip? Fortunately Clive Sinclair, head of Sinclair Radionics, had a secret weapon – programming whiz and math PhD Nigel Searle. In a few days in Texas, they came up with new algorithms and wrote the code for the world’s first single-chip scientific calculator, somehow programming sine, cosine, tangent, arcsine, arccos, arctan, log, and exponentiation into the chip. The engineers at Texas Instruments were amazed. How did they do it? Up until now it’s been a mystery. But through reverse engineering, I’ve determined the exact algorithms and implemented a simulator that runs the calculator’s actual code. The reverse-engineered code along with my detailed comments is in the window below.

    (tags: reversing reverse-engineering history calculators sinclair ti hp chips silicon hacks)

Links for 2013-08-30

Published August 30, 2013

  • Microsoft CEO Steve Ballmer retires: A firsthand account of the company’s employee-ranking system

    LOL MS. Sadly, this talk of “core competencies” and “visibility” is pretty reminiscent of Amazon’s review season, too:

    This illustrated another problem with [stack ranking]: It destroyed trust between individual contributors and management, because the stack rank required that all lower-level managers systematically lie to their reports. Why? Because for years Microsoft did not admit the existence of the stack rank to nonmanagers. Knowledge of the process gradually leaked out, becoming a recurrent complaint on the much-loathed (by Microsoft) Mini-Microsoft blog, where a high-up Microsoft manager bitterly complained about organizational dysfunction and was joined in by a chorus of hundreds of employees. The stack rank finally made it into a Vanity Fair article in 2012, but for many years it was not common knowledge, inside or outside Microsoft. It was presented to the individual contributors as a system of objective assessment of “core competencies,” with each person being judged in isolation. When review time came, and programmers would fill out a short self-assessment talking about their achievements, strengths, and weaknesses, only some of them knew that their ratings had been more or less already foreordained at the stack rank. […] If you did know about the stack rank, you weren’t supposed to admit it. So you went through the pageantry of the performance review anyway, arguing with your manager in the rhetoric of “core competencies.” The managers would respond in kind. Since the managers had little control over the actual score and attendant bonus and raise (if any), their job was to write a review to justify the stack rank in the language of absolute merit. (“Higher visibility” was always a good catch-all: Sure, you may be a great coder and work 80 hours a week, but not enough people have heard of you!)

    (tags: amazon stack-ranking employees ranking work microsoft core-competencies)

Links for 2013-08-29

Published August 29, 2013

  • BBC News – How one man turns annoying cold calls into cash

    This is hilarious. Quid pro quo!

    Once he had set up the 0871 line, every time a bank, gas or electricity supplier asked him for his details online, he submitted it as his contact number. He added he was “very honest” and the companies did ask why he had a premium number. He told the programme he replied: “Because I’m getting annoyed with PPI phone calls when I’m trying to watch Coronation Street so I’d rather make 10p a minute.” He said almost all of the companies he dealt with were happy to use it and if they refused he asked them to email.

    (tags: spam cold-calls phone ads uk funny 0871 premium-rate ppi)

  • The Edge Minecraft cover

    This is brilliant. Half of the office now wants prints.

    Massive congratulations to Edge magazine. The stellar publication has been around for 20 years! To celebrate, their 258th issue comes in 20 different flavours, and one of those flavours includes the earthly overtones of both Minecraft and Dungeons & Dragons. Junkboy drew it, and I [Owen] worded it a few weeks ago.

    (tags: covers images edge minecraft gaming funny dungeons-and-dragons retro dnd)

  • Forecast Blog

    Forecast.io are doing such a great job of applying modern machine-learning to traditional weather data. “Quicksilver” is their neural-net-adjusted global temperature geodata, and here’s how it’s built

    (tags: quicksilver forecast forecast.io neural-networks ai machine-learning algorithms weather geodata earth temperature)

  • _MillWheel: Fault-Tolerant Stream Processing at Internet Scale_ [paper, pdf]

    from VLDB 2013:

    MillWheel is a framework for building low-latency data-processing applications that is widely used at Google. Users specify a directed computation graph and application code for individual nodes, and the system manages persistent state and the continuous flow of records, all within the envelope of the framework’s fault-tolerance guarantees. This paper describes MillWheel’s programming model as well as its implementation. The case study of a continuous anomaly detector in use at Google serves to motivate how many of MillWheel’s features are used. MillWheel’s programming model provides a notion of logical time, making it simple to write time-based aggregations. MillWheel was designed from the outset with fault tolerance and scalability in mind. In practice, we find that MillWheel’s unique combination of scalability, fault tolerance, and a versatile programming model lends itself to a wide variety of problems at Google.

    (tags: millwheel google data-processing cep low-latency fault-tolerance scalability papers event-processing stream-processing)

Links for 2013-08-28

Published August 28, 2013

  • GCHQ tapping at least 14 EU fiber-optic cables

    Süddeutsche Zeitung (SZ) had already revealed in late June that the British had access to the cable TAT-14, which connects Germany with the USA, UK, Denmark, France and the Netherlands. In addition to TAT-14, the other cables that GCHQ has access to include Atlantic Crossing 1, Circe North, Circe South, Flag Atlantic-1, Flag Europa-Asia, SeaMeWe-3 and SeaMeWe-4, Solas, UK France 3, UK Netherlands-14, Ulysses, Yellow and the Pan European Crossing.

    (tags: sz germany cables fiber-optic tapping snooping tat-14 eu politics gchq)

  • In historic vote, New Zealand bans software patents | Ars Technica

    This is amazing news. Paying attention, Sean Sherlock?

    A major new patent bill, passed in a 117-4 vote by New Zealand’s Parliament after five years of debate, has banned software patents. The relevant clause of the patent bill actually states that a computer program is “not an invention.” Some have suggested that was a way to get around the wording of the TRIPS intellectual property treaty, which requires patents to be “available for any inventions, whether products or processes, in all fields of technology.” […] One Member of Parliament who was deeply involved in the debate, Clare Curran, quoted several heads of software firms complaining about how the patenting process allowed “obvious things” to get patented and that “in general software patents are counter-productive.” Curran quoted one developer as saying, “It’s near impossible for software to be developed without breaching some of the hundreds of thousands of patents granted around the world for obvious work.” “These are the heavyweights of the new economy in software development,” said Curran. “These are the people that needed to be listened to, and thankfully, they were.”

    (tags: new-zealand nz patents swpats law trips ip software-patents yay)

  • Docker: Git for deployment

    Docker is to deployment as Git is to development. Developers are able to leverage Git’s performance and flexibility when building applications. Git encourages experiments and doesn’t punish you when things go wrong: start your experiments in a branch, if things fall down, just git rebase or git reset. It’s easy to start a branch and fast to push it. Docker encourages experimentation for operations. Containers start quickly. Building images is a snap. Using another images as a base image is easy. Deploying whole images is fast, and last but not least, it’s not painful to rollback. Fast + flexible = deployments are about to become a lot more enjoyable.

    (tags: docker deployment sysadmin ops devops vms vagrant virtualization containers linux git)

Links for 2013-08-27

Published August 27, 2013

  • Using set cover algorithm to optimize query latency for a large scale distributed graph | LinkedIn Engineering

    how LI solved a tricky graph-database-query latency problem with a set-cover algorithm

    (tags: linkedin algorithms coding distributed-systems graph databases querying set-cover set replication)

  • How might the feds have snooped on Lavabit?

    “I have been told that they cannot change your fundamental business practices,” said Callas, who unlike Levison was able to say SilentCircle has received no NSLs or court orders of any kind. “I presume that would mean things like getting SSL keys because that would mean they could impersonate your servers. That would be like setting up a store front that says your business name and putting [government agents] in your company uniforms.” Similarly, he added: “They cannot make changes to existing operating systems. They can’t make you change source code.” To which [Lavabit’s] Levison replied: “That was always my understanding, too. That’s why this is so important. Like [Callas] at SilentCircle said, the assumption has been that the government can’t force us to change our business practices like that and compromise that information. Like I said, I don’t hold those beliefs anymore.”

    (tags: ars-technica security privacy nsls ssl silentcircle jon-callas crypto)

  • Lock-Based vs Lock-Free Concurrent Algorithms

    An excellent post from Martin Thompson showing a new JSR166 concurrency primitive, StampedLock, compared against a number of alternatives in a simple microbenchmark. The most interesting thing for me is how much the lock-free, AtomicReference.compareAndSet()-based approach blows away all the lock-based approaches — even in the 1-reader-1-writer case. Its code is extremely simple, too: https://github.com/mjpt777/rw-concurrency/blob/master/src/LockFreeSpaceship.java

    (tags: concurrency java threads lock-free locking compare-and-set cas atomic jsr166 microbenchmarks performance)

  • Juniper Adds Puppet support

    This is super-cool. ‘Network engineering no longer should be mundane tasks like conf, set interfaces fe-0/0/0 unit o family inet address 10.1.1.1/24. How does mindless CLI work translate to efficiently spent time ? What if you need to change 300 devices? What if you are writing it by hand? An error-prone waste of time. Juniper today announced Puppet support for their 12.2R3,5 JUNOS code. This is compatible with EX4200, EX4550, and QFX3500 switches. These are top end switches, but this start is directly aimed at their DC and enterprise devices. Initially, the manifest interactions offered are interface, layer 2 interface, vlan, port aggregation groups, and device names.’ Based on what I saw in the Network Automation team in Amazon, this is an amazing leap forward; it’d instantly render obsolete a bunch of horrific SSH-CLI automation cruft.

    (tags: ssh cli automation networking networks puppet ops juniper cisco)

  • awscli

    The future of the AWS command line tools is awscli, a single, unified, consistent command line tool that works with almost all of the AWS services. Here is a quick list of the services that awscli currently supports: Auto Scaling, CloudFormation, CloudSearch, CloudWatch, Data Pipeline, Direct Connect, DynamoDB, EC2, ElastiCache, Elastic Beanstalk, Elastic Transcoder, ELB, EMR, Identity and Access Management, Import/Export, OpsWorks, RDS, Redshift, Route 53, S3, SES, SNS, SQS, Storage Gateway, Security Token Service, Support API, SWF, VPC. Support for the following appears to be planned: CloudFront, Glacier, SimpleDB. The awscli software is being actively developed as an open source project on Github, with a lot of support from Amazon. You’ll note that the biggest contributors to awscli are Amazon employees with Mitch Garnaat leading. Mitch is also the author of boto, the amazing Python library for AWS.

    (tags: aws awscli cli tools command-line ec2 s3 amazon api)

Links for 2013-08-26

Published August 26, 2013

  • Let Me Explain Why Miley Cyrus’ VMA Performance Was Our Top Story This Morning | The Onion – America’s Finest News Source

    Absolute genius from The Onion.

    Those of us watching on Google Analytics saw the number of homepage visits skyrocket the second we put up that salacious image of Miley Cyrus dancing half nude on the VMA stage. But here’s where it gets great: We don’t just do a top story on the VMA performance and call it a day. No, no. We also throw in a slideshow called “Evolution of Miley,” which, for those of you who don’t know, is just a way for you to mindlessly click through 13 more photos of Miley Cyrus. And if we get 500,000 of you to do that, well, 500,000 multiplied by 13 means we can get 6.5 million page views on that slideshow alone. Throw in another slideshow titled “6 ‘don’t miss’ VMA moments,” and it’s starting to look like a pretty goddamned good Monday, numbers-wise. Also, there are two videos — one of the event and then some bullshit two-minute clip featuring our “entertainment experts” talking about the performance. Side note: Advertisers, along with you idiots, love videos. Another side note: The Miley Cyrus story was in the same top spot we used for our 9/11 coverage.

    (tags: humor journalism cnn miley-cyrus vma news funny advertising ads)

  • Why wireless mesh networks won’t save us from censorship

    I’m not saying mesh networks don’t work ever; the people in the wireless mesh community I’ve met are all great people doing fantastic work. What I am saying is that unplanned wireless mesh networks never work at scale. I think it’s a great problem to think about, but in terms of actual allocation of time and resources I think there are other, more fruitful avenues of action to fight Internet censorship.
    (via Kragen)

    (tags: wireless censorship internet networking mesh mesh-networks organisation scaling wifi)

Links for 2013-08-24

Published August 24, 2013

  • Newest YouTube user to fight a takedown is copyright guru Lawrence Lessig

    This is lovely. Here’s hoping it provides a solid precedent.

    Illegitimate or simply unnecessary copyright claims are, unfortunately, commonplace in the Internet era. But if there’s one person who’s probably not going to back down from a claim of copyright infringement, it’s Larry Lessig, one of the foremost writers and thinkers on digital-age copyright. [..] If Liberation Music was thinking they’d have an easy go of it when they demanded that YouTube take down a 2010 lecture of Lessig’s entitled “Open,” they were mistaken. Lessig has teamed up with the Electronic Frontier Foundation to sue Liberation, claiming that its overly aggressive takedown violates the DMCA and that it should be made to pay damages.

    (tags: liberation-music eff copyright law larry-lessig fair-use)

  • TCP is UNreliable

    Great account from Cliff Click describing an interest edge-case risk of using TCP without application-level acking, and how it caused a messy intermittent bug in production.

    In all these failures the common theme is that the receiver is very heavily loaded, with many hundreds of short-lived TCP connections being opened/read/closed every second from many other machines.  The sender sends a ‘SYN’ packet, requesting a connection. The sender (optimistically) sends 1 data packet; optimistic because the receiver has yet to acknowledge the SYN packet.  The receiver, being much overloaded, is very slow.  Eventually the receiver returns a ‘SYN-ACK’ packet, acknowledging both the open and the data packet.  At this point the receiver’s JVM has not been told about the open connection; this work is all opening at the OS layer alone.  The sender, being done, sends a ‘FIN’ which it does NOT wait for acknowledgement (all data has already been acknowledged).  The receiver, being heavily overloaded, eventually times-out internally (probably waiting for the JVM to accept the open-call, and the JVM being overloaded is too slow to get around to it) – and sends a RST (reset) packet back…. wiping out the connection and the data.  The sender, however, has moved on – it already sent a FIN & closed the socket, so the RST is for a closed connection.  Net result: sender sent, but the receiver reset the connection without informing either the JVM process or the sender.

    (tags: tcp protocols SO_LINGER FIN RST connections cliff-click ip)

  • The ultimate SO_LINGER page, or: why is my tcp not reliable

    If we look at the HTTP protocol, there data is usually sent with length information included, either at the beginning of an HTTP response, or in the course of transmitting information (so called ‘chunked’ mode). And they do this for a reason. Only in this way can the receiving end be sure it received all information that it was sent. Using the shutdown() technique above really only tells us that the remote closed the connection. It does not actually guarantee that all data was received correctly by program B. The best advice is to send length information, and to have the remote program actively acknowledge that all data was received.

    (tags: SO_LINGER sockets tcp ip networking linux protocols shutdown FIN RST)