Category: Uncategorized

Packet Capture From My Philips TV

I recently bought a new TV for our house, a Philips 55PUS7009 4K LED Smart TV (“55 Inch Display with Pixel Precise Ultra HD Titan OS Platform and Dolby Atmos Sound, Works with Alexa and Google Voice Assistant”, according to Amazon). It's a decent modern TV, though its native platform is not Android, unfortunately; instead it’s “TitanOS”, a Linux-based Smart TV operating system.

After reading The Smart TV in Your Living Room Is a Node in the AI Scraping Economy, I thought it’d be a good plan to take a look at what’s going on in terms of network traffic from this device. My wifi AP is a GL-iNet MT-6000 , which runs their OpenWRT-based operating system, so gives easy access to developer-friendly features like full tcpdump support to examine a wireless client’s traffic. (scroll down to “Snooping Traffic For One Client In GL-MT6000” if you’re curious about this part.)

Another reason I wanted to do this was to disable mandatory updates; I don't want to risk something like this horror from LG. Despite disabling this in the TV’s UI, the most recent update (a couple of months ago) was accompanied by nagging dialog boxes every time someone turned on the TV, which is a really crappy piece of user experience, thanks Philips --- so I’d prefer if the TV had no idea that an update was available in the first place.

What The TV Sends

I power cycled the TV, waited for it to boot, then checked for software updates in the Settings menu. During this process, I ran a packet capture for all traffic escaping from the device. Here’s what it hit during the process, tcpdump interpreted with a little help from Claude:

Content / streaming apps (expected):

1. Netflix — *.netflix.com, nrdp*.netflix.com, occ.a.nflxso.net, cdn-0.nflximg.com, tpv-*.prod.partner.netflix.net, and threeplr-*.api.amazonvideo.com (Netflix's Open Connect is partly fronted via Amazon)
2. YouTube / Google — www.youtube.com, www.google.com, clients3.google.com (the last is Google's captive-portal/connectivity check)
3. Apple — mediaservices.cdn-apple.com (AirPlay/trailer assets)
4. download.airserver.com — AirServer mirroring; I may have installed this TitanOS app. Pretty much expected.

Platform / infrastructure (expected):

1. NTP: 0.ie.pool.ntp.org, time.aws.com — clock sync, all fine
2. app.titanos.tv, www.philips.com — TitanOS platform + vendor
3. *.core.cloud.vewd.com — the Vewd browser engine TitanOS is built on
4. ocsp.*.amazontrust.com — TLS certificate revocation checks

Telemetry / tracking (no thanks):

1. platform.cid.samba.tv / preferences.cid.samba.tv - this is Samba TV ACR (Automatic Content Recognition). It fingerprints what's on screen, including HDMI inputs, for ad/analytics tracking purposes.
2. dac-api-prod.tpv-analytics.com - TP Vision's analytics backend.
3. metrics.core.cloud.vewd.com - browser-engine metrics.
4. Netflix telemetry: ichnaea.netflix.com, customerevents.netflix.com - usage/event reporting, distinct from playback.
5. ipinfo.io - IP geolocation lookup.

The streaming and platform traffic is normal, but one thing was a really nasty surprise; the set seems to support reporting to Samba TV ACR, with what looks like a remote switch to enable it. I do not want this "ACR" feature active, which essentially repeatedly screenshots your screen and reports it home to Samba TV servers; extremely invasive surveillance adware, if you ask me. There was no notification in the Philips setup docs, specification, or TV UI that this kind of crap was installed, which I find particularly unpleasant. Not cool, Philips.

There's also TP Vision analytics -- TP Vision is the company behind the "Philips" TV brand. I doubt these are as invasive as the "ACR" monitoring, but worth turning off too.

Most of the Platform and Telemetry DNS lookups were performed using my configured DNS server. However, Netflix resolves the following names using 8.8.8.8 directly:

 api-global.netflix.com.
 cdn-0.nflximg.com.
 customerevents.netflix.com.
 ichnaea.netflix.com.
 nrdp-cell4.prod.ftl.netflix.com.
 nrdp.nccp.netflix.com.
 nrdp.prod.cloud.netflix.com.
 nrdp25.appboot.netflix.com.
 nrdp51-appboot.netflix.com.
 occ.a.nflxso.net.
 secure.netflix.com.
 uiboot.netflix.com.

Digging into “samba.tv”, which is my biggest source of unwanted traffic here: here's the full picture of what samba.tv did in that capture.

The device: the TV at 10.19.72.124, at power-on (all of this happens in a ~2-second burst at 20:40:23–25). DNS (via your Pi-hole, 10.19.72.11):

1. preferences.cid.samba.tv -- resolves to CNAME -> flingo.tv -> 216.183.117.106 / .105
2. platform.cid.samba.tv -- resolves to 216.183.117.91

Note the CNAME to flingo.tv — Flingo is the original company behind Samba TV's ACR ("automatic content recognition"). The IPs are all in Samba's own 216.183.117.0/24 block. Importantly, the TV had no hardcoded IP — it relied entirely on DNS (unlike Netflix, which went to 8.8.8.8). That's why the Pi-hole block will fully stop it.

The connections — two short-lived HTTPS sessions:

1. 216.183.117.106:443 (preferences.cid.samba.tv): 22 packets, ~5.2 KB
2. 216.183.117.91:443 (platform.cid.samba.tv): 22 packets, ~5.5 KB
3. .105 was resolved but never contacted (redundant A-record).

Both follow an identical shape:

1. TCP handshake, then a 518-byte TLS ClientHello (SNI platform./preferences.cid.samba.tv).
2. Server returns ~3.3 KB — the *.cid.samba.tv certificate (issued by GoDaddy).
3. TV sends ~1 KB of application data (seq 518 -> 1590/1695), server replies with a tiny (~150–300 byte) response.
4. Immediate FIN/RST teardown.

Interpretation: This is a telemetry check-in, not a stream: two brief, one-shot encrypted POSTs at boot — the TV uploads ~1 KB to each endpoint (device identity + a "preferences"/config pull) and hangs up. The payload is TLS-encrypted so the contents are opaque, but the pattern — resolve / connect / upload ~1 KB / tiny reply / close, to hosts literally named platform and preferences under Samba's ACR domain — is most likely the ACR phone-home, not content delivery. Nothing large was exfiltrated in this capture; ACR's continuous "what's on screen" reporting would show up as repeated check-ins over time, which is what you'd catch with ongoing logging.

Given this, I think Pi-Hole blocking on *.samba.tv should block these check-ins and avoid future snooping.

In addition, blocking the following domains seems to ensure that software updates are blocked:

1. app.titanos.tv, www.philips.com — TitanOS platform + vendor
2. *.core.cloud.vewd.com — the Vewd browser engine TitanOS is built on

Full protocol/port breakdown for the TV

Analysing this lets us check for DoH/DoQ/DOH3/DoT name resolution evading the DNS blocks. Here’s what was happening across all TCP/UDP protocols, according to a Claude based analysis:

1. TCP port 443 - destination external (known SNIs) --- HTTPS — the hosts we already enumerated
2. TCP port 80 - destination external --- cleartext HTTP (ipinfo, Netflix appboot, OCSP)
3. TCP port xxxxx ? 10.19.72.xx — LOCAL --- Plex and DLNA media server
4. TCP port 56890 ? 10.19.72.115 — LOCAL --- TV set top box; no TLS, no hostnames
5. UDP port 53 - destination Pi-hole + 8.8.8.8 --- plaintext DNS
6. UDP port 123 - destination external --- NTP
7. ICMP, IGMP traffic

Why this rules out each variant:

1. DoQ / DoH3 (QUIC, always UDP): the only UDP that leaves the network is port 53 and 123. There is zero other UDP egress — no QUIC on any port, standard or not. Definitively ruled out.
2. DoT (TLS on TCP, normally 853): the only nonstandard TCP ports (32469, 56890) both go to local LAN hosts (the Plex server and .115), never to the internet, and neither carries a TLS handshake to a resolver. The only WAN-bound TCP is 443 and 80. Ruled out.
3. DoH (HTTPS on 443 — the hard one): DoH is deliberately indistinguishable from normal HTTPS, so a pcap can't prove its absence by port alone. But:
4. SNI enumeration: every 443 ClientHello we pulled had a content/CDN/tracker hostname (google, netflix, apple, yahoo, youtube, samba, vewd, airserver, GoDaddy OCSP). None is a resolver (dns.google, cloudflare-dns.com, dns.quad9.net, etc.), and there was no 443 to 8.8.8.8/1.1.1.1/9.9.9.9.

So no encrypted DNS is escaping, on either standard or nonstandard ports. The TV's name resolution is 100% cleartext Do53.

One other thing that’s interesting there is the port 56890 TCP traffic to 10.19.72.115. This is the Virgin Media TV box I use to watch Virgin’s TV service; it’s intriguing that the TV opens a TCP connection immediately to this set top device. Here’s what Claude makes of it:

Port 56890 is DIAL / UPnP device discovery — the "cast to TV" mechanism — and it's entirely local LAN, nothing to do with DNS or phone-home. The TV (10.19.72.124) is the client here. It opens three short connections to 10.19.72.115:56890 and sends a plain-HTTP UPnP request:

GET /apps/dd.xml HTTP/1.1
HOST: 10.19.72.115:56890
USER-AGENT: Linux/6.1.124-..., UPnP/1.0, Portable SDK for UPnP devices/1.14.6

dd.xml is a UPnP device-description document, and the Portable SDK for UPnP devices/1.14.6 user-agent is the standard libupnp DIAL stack. The device at .115 answers with:

HTTP/1.1 200 OK
Application-URL: http://10.19.72.115:8092/apps/     <- DIAL app-launch REST endpoint
...
<deviceType>urn:schemas-upnp-org:device:tvdevice:1</deviceType>
<friendlyName>360 Box</friendlyName>
<manufacturer>360 Box</manufacturer>
<modelName>EOS-1008R</modelName>
<UDN>uuid:0f7902cd-efbf-4a81-aeae-d975bbf89bff</UDN>

This is DIAL (DIscovery And Launch) — the protocol behind "Cast" / second-screen app launching (YouTube, Netflix "play on TV", etc.). The tvdevice:1 type and the Application-URL pointing at :8092/apps/ are the DIAL signature. The TitanOS TV is scanning the LAN for cast targets, found a device advertising itself as "360 Box", model EOS-1008R at 10.19.72.115, and fetched its capability document to learn what apps it can launch and where (:8092). The dd.xml retrieval is the tiny handshake — 204-byte request, ~624-byte XML reply, then clean close. Three near-identical connections at 21:06:56 are just the discovery routine probing a couple of times.

It appears the Virgin set top box can be poked to launch a Netflix client, a Youtube client (which is currently disabled?), and that appears to be it; it has no remote-streaming capability using this protocol.

Snooping Traffic For One Client In GL-MT6000

Here's how I set this up, for future reference. Log into the GL-Inet admin UI, then click into LUCI, the OpenWRT GUI, using the "System" -> "Advanced Settings" link. Go to Network -> Wireless and add a new wireless network under your 5GHz wifi device. Give it a new ESSID (I used my normal SSID name, plus "IOT") and set its WPA key. You can choose “Isolate Clients”, which (I think) will ensure clients on this network can only talk to the external internet? (I’m not sure; I didn’t use it as my local LAN fileserver serves all sorts of content to the TV, so I left it off.)

Hit Save and enable the network.

Change the TV’s networking config to use this new SSID and it should connect to it, then showing up in the “Associated Stations” part of Network -> Wireless with a new device name; in my case, “rax1”. SSH to the AP and run:

# opkg update && opkg install tcpdump

You should now be able to use tcpdump to snoop packets on that interface, seeing only traffic to and from the client in question (10.19.72.124 being the TV in my case):

# tcpdump -ni rax1 -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on rax1, link-type EN10MB (Ethernet), capture size 262144 bytes
19:34:18.487777 IP 10.19.72.124.41837 > 10.19.72.11.53: 54382+ A? www.google.com. (32)
19:34:18.487787 IP 10.19.72.124.41837 > 10.19.72.11.53: 23073+ AAAA? www.google.com. (32)
19:34:18.490906 IP 10.19.72.11.53 > 10.19.72.124.41837: 54382 8/0/0 A 142.251.151.119, A 142.251.152.119, A 142.251.154.119, A 142.251.157.119, A 142.251.150.119, A 142.251.153.119, A 142.251.155.119, A 142.251.156.119 (160)
19:34:18.492935 IP 10.19.72.11.53 > 10.19.72.124.41837: 23073 8/0/0 AAAA 2001:4860:4826:7700::, AAAA 2001:4860:4828:7700::, AAAA 2001:4860:482a:7700::, AAAA 2001:4860:4829:7700::, AAAA 2001:4860:482c:7700::, AAAA 2001:4860:4827:7700::, AAAA 2001:4860:482d:7700::, AAAA 2001:4860:482b:7700:: (256)

And for deeper examination, write a pcap file:

# opkg install openssh-sftp-server
# tcpdump -ni rax1 -w /tmp/iot.pcap -C 50 -W 5
tcpdump: listening on rax1, link-type EN10MB (Ethernet), capture size 262144 bytes
^C6486 packets captured
6487 packets received by filter
0 packets dropped by kernel

You can now scp that /tmp/iot.pcap file to your local machine for full analysis. (Claude is really good at this part.)

I really recommend the GL-iNet GL-MT6000 as an AP or router BTW. OpenWRT is just a fantastic operating system for these devices, full of developer accessibility, scriptability, automation, and room for cool hacks, and GL-iNet did a good job in building a reliable, solid consumer product on top of it.

• Landmark German ruling declares Google's AI Overviews are Google's own words and makes it liable for false answers

  This is a major liability judgement against Google's use of AI:

  "A German regional court has ruled that Google is directly liable for false claims in its AI-generated search overviews. In this case, Google's AI had wrongly linked two publishers to scams and shady business practices. The court treated the AI overviews as Google's own content and rejected Google's argument that users were responsible for fact-checking the results themselves."

  A mastodon-based reviewer summarises:

  The judge is explicitly cutting down most of the legal defenses they use. They make a sharp cut between search and AI, saying search is indispensable, but AI is not, and defendants have not proven how being held liable for their output would compromise the ability to run a normal search engine. They make a similar hard cut between AI and autocomplete.

  They go on at length about the nature of truth in utterances, and arrive at a conclusion that AI output has no protections for free expression because it isn't expressing shit - it has no beliefs, it is a commercial product only. There are two injunctions that are denied because they are not considered statements of fact, but the judge rules against google for all the ones that were, and concludes several are default considered false because the linked pages were irrelevant.

  There is explicit differentiation from aggregating reviews and third party content, because the AI generated text and ideas that were not present in the input. There is also discussion about how there is no excuse for further violations just because its hard to control AI output, and contrasts this with how normal "report and takedown" protections work.

  There is very little here that is specific to AI overviews in search, and almost all of the arguments apply to AI products in general. AI's only prayer of being remotely profitable must include advertising or shopping features, which means they absolutely must continue generating output that makes statements of fact about other companies. I know nothing about how German courts work, the article alludes to appeals, but if this ruling holds even just in Germany the ability to insure AI products disappears overnight and that makes the product nonviable.

  Tags: germany eu liability google ai-overviews slop law truth libel facts insurance

• Narrative Flattening

  This is a nice term from Katryna Peart to describe one of the corrupting factors LLMs introduce to documents: "narrative flattening":

  "When US cities began deploying AI to process civic documents — meeting minutes, public histories, commemorative records, policy documents — the failures that emerged weren’t the dramatic kind. The AI didn’t invent facts wholesale [...] it smoothed."

  AI-driven "summarisation" results in radically modified narrative meanings:

  Contested decisions became consensus. Dissenting voices disappeared into summaries that read as agreement. And the output read as authoritative because it was produced from an authoritative source.

  In testing AI systems against a municipal commemorative report from Newark, New Jersey, I asked each system how the initiative compared to similar programmes in the region — using only the document provided.

  ChatGPT invented a comparative framework, asserting the programme stood out from “traditional anniversary programmes” and “typical county-level commemorations.” Neither category exists in the document. The system didn’t hallucinate a date or misattribute a quote. It constructed an entire analytical frame from nothing and presented it as document-based retrieval.

  In the same test, the document contained outreach tactics — radio, direct mail, community networks — that worked because they aligned with how civic organising functions in Newark’s Black, ageing community. An AI system extracting those tactics would present them as applicable elsewhere, while stripping away the demographic and political context that made them effective in that specific city. [...]

  To test whether the same failure modes appear in UK civic documents, I applied the same protocol to the Somerset Council Plan 2023–2027 — the post-reorganisation vision document produced when five predecessor councils merged into a single unitary authority in April 2023. I tested four AI systems: Gemini, ChatGPT, Microsoft Copilot, and Claude.

  Every model hardened aspirational language into apparent commitments. The plan states the council would “demonstrate leadership around the whole range of housing issues” and “strive to develop an inclusive culture.” No targets, no timelines, no delivery mechanisms. Every model converted those statements into bullet-pointed commitment lists. A council officer reviewing those outputs would have no way of knowing the specificity came from the model, not the plan.

  Copilot — the model most UK councils are currently deploying through existing Microsoft 365 contracts, often without separate AI governance review — described Somerset as committed to “co-design” with communities and “fair access” to education, housing, jobs, and services. Neither phrase appears in the document. Copilot synthesised fragments from three separate passages into a single clean commitment the council never made. On the comparison question, it added that Somerset’s emphasis on rural inequality “is less prominent in many urban unitary authorities” — a comparative claim with no basis in the source. It did not flag any of this as inference.

  ChatGPT fabricated a comparative framework and constructed a tension narrative. Asked how Somerset’s approach compared to other UK unitary authorities, it responded that “unlike more fragmented models, Somerset frames inequality as interconnected” with multiple service areas. No other authority is described anywhere in the document. It also described the “main tensions” in the reorganisation process — a framing the document never uses. The word tensions does not appear in the Somerset Council Plan.

  [...]

  Across original research testing three civic documents against three major AI systems, failure modes were structurally predictable based on document type:

  • Celebratory documents get reproduced uncritically — institutional PR becomes authoritative historical record, success metrics are cited without methodological context, and dissenting voices go unmarked.
  • Accountability documents get softened — AI systems introduce balance and healing language that the original document explicitly rejects, restoring a both-sides framing the institution deliberately refused.
  • Pre-event planning documents get filled in — aspirational inclusion language invites AI to supply the racial history, equity frameworks, and comparative data the institution implied but never produced.

  In each case the output reads as grounded in the source. In each case something the document actually said — or deliberately did not say — has been quietly rewritten.

  Standard AI procurement frameworks test for hallucination, data security, and cost. They do not test for narrative flattening — because it doesn’t look like an error. It looks like a summary.

  (via gwire)

  Tags: narrative-flattening via:gwire summarisation summarization ai llms corruption katryna-peart hallucination confabulation errors uk documents

• School shooting survivor sues AI gun detection firm after system failed to spot weapon

  Omnilert, which sells an "AI gun detection" system, sued after it failed to detect a gun prior to a January 2025 school shooting. Turns out accuracy matters!

  'According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”

  Omnilert further represented that AI-powered visual gun detection “could have mitigated or prevented tragedy at Marjory Stoneman Douglas High School” by identifying threats earlier—invoking one of the nation’s most devastating school shootings to convey that its product would prevent similar tragedies ... Omnilert made no mention of false alarms, false positives, or detection limitations of any kind on its pre-shooting commercial website.

  Tags: omnilert accuracy false-positives false-negatives marketing ai guns school-shootings us-politics

• Distributed terrorism as ARG

  this is pretty much the plot of Charlie Stross' "Halting State" (2006): teenagers are being recruited online by the FSB, in online forums and in-game chats, then assigned alternate-reality-game-style "tasks" in the real world which are actually acts of espionage on behalf of Russia.

  The recruitments follow a similar pattern: young people are usually approached on online channels which are well-hidden and hard to track: from Telegram to TikTok, Snapchat, Facebook and Discord. They are offered money, commonly cryptocurrencies, in exchange for completing tasks. Their recruiters depend on anonymity; many work for criminal groups which, like cyber hackers, may be independent from the state but co-opted by intelligence agencies for covert operations.

  Gaming sites — the most widely consumed entertainment media among 13- to 24-year-olds — have become an obvious hunting ground for potential saboteurs with a proven interest in problem solving.

  In Ukraine, the chat function in the popular online game World of Tanks is commonly used as a recruitment portal, from which agents then move the conversation to Telegram. Some state-backed agents, especially those working for Russia, also invoke the mission format and “quest” mentality of online games to entice young people to move beyond the virtual battlefield to real-world action. It is, says one western military official, “like a game of Pokémon Go, but with air defence systems”.

  Adrian Hon, an ARG designer, comments:

  I don't think this is the work of some evil genius FSB game designer. They're throwing shit at a wall and they found that:

  • 1. Making teens feel cool and special
  • 2. Giving them clear tasks escalating in difficulty
  • 3. Paying them £500 in crypto

  sticks!

  It is FUN to imagine you are a spy going out on a secret real world mission, getting messages from a handler.

  The money helps, but the main thing is that it's free, unlike practically every comparable form of real world immersive experience/pervasive game.

  Average British reaction: "Why teenagers spend their lives glued to screens and on non value-adding activities idk. [Buy] them footballs and chess sets and send them outside." Yes, so they can play football on the non-existent pitches that now cost money to play on and they can't get to.

  Everyone is like, kids should get away from screens and go outside. Motherfucker that is EXACTLY what these teen FSB recruits are doing!! They are going outside taking photos, collecting wifi SSIDs, sneaking around. I literally design games like this, except I have a fraction of their budget!!!

  And when I design pervasive games, we have to get public liability insurance and local govt permission and pay fees and do risk assessments. I don't make them for under-18s because god knows the red tape is a mile long.

  mfw we're getting outcompeted by foreign intelligence agencies

  Tags: fsb russia spying espionage args gaming games teens arg crypto

• CrankGPT

  "fully offline, human-powered local AI" -- an LLM and a voice model, running on a Raspberry Pi 5, driven off hand-cranked electricity generation! I was very sceptical, but they've put in the work to optimise the platform and choose models very carefully, and it looks like it actually runs off hand-cranked power, amazing

  Tags: ai llms electricity hand cranking voice raspberry-pi hacks hardware

• Kafka’s quiet observability superpower — Kafka Interceptors

  Interesting Kafka trick:

  Kafka Interceptors have quietly existed since 2016, yet most teams overlook them as an observability superpower. This article shows how Kafka Interceptors, combined with Apache Flink, can provide lightweight, near-real-time event tracing across a Kafka-based event-driven architecture — without invasive instrumentation or expensive observability platforms.

  The project, confluent-kafka-isotope, uses Kafka Interceptors to collect and attach trace signals to records while Apache Flink interprets those signals using SQL and stateful processing for latency analysis, topology discovery, stuck-trace detection, and forensic replay.

  Tags: kafka observability flink isotope-tracing tracing event-tracing interceptors ops

• Grill Fanatics Restaurant Grade Marabu Charcoal (10kg)

  I spotted Pitt Bros using this charcoal at a recent event, so that's a plug for me

  Tags: charcoal cooking food yum

• More on the Coinbase 07-05-2026 outage

  More on the Coinbase 07-05-2026 outage, caused by a "thermal event" in AWS us-east-1 and its impact on the suppposedly multi-AZ Managed Kafka product:

  AWS's managed Kafka service failed silently. A significant portion of our event-streaming infrastructure runs on MSK, AWS's managed Kafka offering. The architectural promise of a managed Kafka service is that when individual brokers go down, the service automatically reelects partition leaders and continues to serve traffic out of the remaining brokers. The loss of an entire zone should result in reduced capacity, not unavailability.

  That is not what happened and this extended the outage.

  A defect in the AWS MSK control plane prevented automatic partition-leader reelection. Two of our MSK clusters became stuck in a "healing" state with producers unable to write. The cascading effect blocked our fee service, which blocked quoting, which is why most customers experienced this incident as broken trades and quotes rather than as a Kafka outage. Adjacent systems, including portions of our ledger pipeline, payments, and several data pipelines, were affected the same way. Additionally, one of our Kafka clusters was set up in a 2-AZ configuration that increased the blast radius and recovery time, but the MSK control plane defect impacted 2-AZ and 3-AZ Kafka clusters similarly.

  We worked the recovery in real time with AWS engineering, ultimately performing manual partition reassignments at 3:00 AM ET to migrate topics off the impaired brokers. Priority-zero and priority-one topics were back to full availability by 9:30 AM ET. The remainder cleared by 2:00 PM ET.

  In fairness, they also had a single-AZ point of failure in their architecture which they also describe there, but still, not great performance from MSK. Disappointing.

  Tags: msk reliability multi-az aws services kafka resiliency outages post-mortems postmortems coinbase

• A 10 year old Xeon is all you need - point.free

  Some mad scientist optimization to get Gemma 4 running on an old Xeon with no GPU

  Tags: google ai llms gemma gemma-4 xeon hacks

• Best Practices for TCP Connection Management on EC2

  Well this is a really crappy thing for AWS to mess around with, and then hide the announcement on a "best practices" page:

  "With sixth-generation AWS Nitro (Nitro V6) instances, launched in June 2025 [c8, r8, etc], the default TCP connection tracking idle timeout changed from 432,000 seconds (5 days) to 350 seconds. Applications that hold idle connections open for long periods, such as [uhhh pretty much everything built on TCP - jm] may experience unexpected connection drops after migrating to these instances."

  They go on to recommend that you "implement keepalives and connection lifecycle management", which is great fun if you don't control the code implementing your TCP-based network protocols. This is a very fundamental change for many protocols so it'll be fun dealing with it.

  Kudos to Adam C in the ITC Slack for spotting this a while back.

  Tags: networking protocols tcp idle-timeouts aws architecture nitro conntrack idle-connections

• Current Rothko

  I love this! Finds the weather at your location, then picks a Rothko to match. This would be great on a home dashboard. (well, it'd be better if it used a more reliable weather backend, as most times I've tried it here in Dublin, it's told me the wrong current weather conditions. But close!)

  Tags: location weather art fun rothko

• Coinbase MSK outage post-mortem

  A post-mortem from Coinbase following a significant outage partially caused by MSK, AWS' managed version of Kafka.

  Root cause: a thermal event (cooling system failure) inside a subset of racks within a single building in AWS us-east-1. We run a primary replica of our exchange infrastructure in a single zone, consistent with industry standards to reduce latency. To prepare for failures like this, we maintain a distributed standby, but during this incident, failures in the primary zone that were designed to be isolated were not [...]

  Our primary managed Kafka partitions process many terabytes of data daily and are designed with resiliency guarantees for uninterrupted operation during a datacenter failure just like this. In this case, those guarantees failed and required manual recovery. [...]

  There is a hint here that MSK failed to have multi-AZ resiliency despite multiple replicas configured at the application level. It will be interesting to see what the full root-cause analysis looks like....

  Tags: kafka resiliency coinbase multi-az az aws us-east-1 post-mortems postmortems

• Serverless Functions Post-Mortem

  A post-mortem for "serverless functions", the fad of 2016

  Tags: serverless cloud programming architecture aws

• 1.2M Messages to Obsidian - Building a Relationship Map from 20 Years of Chat History

  "Am I a bad friend?" -- Vadim Drobinin "analysed 20 years of my chats and turned 1.2M messages into a structured vault of my life - to win friends and influence people. Instead, I learnt things about my emotional bandwidth, endearment cycles, and friendship half-lives."

  This is actually a really nice project. I wish I'd accumulated and archived all that data over the years myself to do something similar.

  Tags: dataviz analysis friendship life relationships vocabulary words text dunbar-number chats group-chats messaging

• Auditing AI Chatbots During the Galway West and Dublin Central Byelections

  "In the weeks leading up to the byelections in Galway West and Dublin Central, we simulated citizen-AI interactions by asking [a] set of election-related test questions to four popular AI chatbots (Anthropic’s Claude, OpenAI’s ChatGPT, Google’s Gemini, and xAI’s Grok)":

  We asked each chatbot a set of 194 questions covering a range of relevant topics on two separate occasions, 14 and 7 days before voters go to the polls on 22 May. Our aim was to assess:

  • Do they provide citizens with accurate election information?
  • Which sources do they rely on, and how does this vary across chatbots?
  • Who do they platform when they answer questions, and who is left out?

  .... The largest share of citations is directed towards mainstream news sites, and this is where the first evidence of source curation by chatbots can be found. While the Irish Times is a common source across all providers, ChatGPT and Gemini never refer to RTÉ. Meanwhile, despite not ranking among the top three news sources for the other chatbots, Gript is the number one news source cited by Gemini.

  We find that xAI’s Grok is the most likely to use social media in responses, while Gemini most frequently refers to YouTube and content from Reddit. ChatGPT appeared less likely to rely on social media compared to the other chatbots.

  IMO, this points to an undesirable side effect of paywalls in the news media. While it's vitally important for media companies to protect their means of income, an unwelcome side effect is that the introduction of paywalls has resulted in AI chatbots sidelining mainstream news media sources which they cannot access reliably, in favour of what is effectively disinformation from less trustworthy sites like Gript.

  Tags: grok ai llms xai web news media ireland irish-times rte chatgpt gemini reddit youtube elections politics

• Trevor Paglen and Holly Herndon on Making Art with AI and What the Discourse Is Missing

  Fascinating interview with Trevor Paglen and Holly Herndon, the two artists making the most interesting work at the moment which interacts with and investigates AI, machine learning models, and slop

  Tags: art ai slop trevor-paglen holly-herndon interviews

• no slop grenade

  No slop grenade -- stop throwing AI-generated walls of text into conversations:

  Pasting a massive AI-generated response into a chat or email where a human would write one sentence. It destroys the medium itself. Nobody writes essays in Slack. It's only possible because of AI copy-paste.

  It's like calling someone and asking "What time is the meeting?" and they read you a 10-page analysis of calendar management best practices. You asked a simple question. They lobbed a document.

  Tags: slop-grenade ai llms words neologisms slop chat copy-paste

• The "Rapture" was an Irish invention

  Take a bow, John Nelson Darby:

  John Nelson Darby, the fella who came up with the concept of "the Rapture", was a Church of Ireland curate in Co. Wicklow. Always assumed it was a yank.

  If you've ever seen American televangelists ranting about Jesus secretly swooping down to save the righteous before destroying the earth, this is where it comes from. It’s not actually normal Christianity apparently, it’s not really in the bible anywhere but this headtheball popularised it in the 19th century. The Yank fundamentalists absolutely lap this up.

  The history of it is even more interesting:

  He was born in England and then studied in Trinity and was a Church of Ireland pastor here. He famously converted a load of Catholics in the village but ended up resigning because the church only accepted the conversions as legitimate if they swore an oath to the British King.

  Despite being English himself, he seems to genuinely have believed in religion as separate to the politics of the time, which I kind of admire tbh. So he quits, and shortly after, he's out riding in Wicklow when the horse sends him flying. He suffers a serious bang to the head.

  While he's recovering from the concussion, he starts coming up with this mental end-of-the-world theology that eventually took over the US.

  TL;DR: We could have been spared an unbelievable amount of absolute bollocks if some 19th century prod hadn’t been flung off a horse in rural Co. Wicklow.

  Tags: history ireland rapture fundamentalism apocalypse theology end-of-the-world 19th-century wicklow concussion horse

• Bournegol

  The original source code for the Bourne shell in early versions of UNIX is legendarily bizarre, as it was written in "Bournegol", the ALGOL-like dialect of C that Steve Bourne came up with, with a load of macros to make C look a bit like ALGOL 68. This page has a good representative sample. Thanks to Tony Finch for the reminder

  Tags: via:fanf bournegol algol programming languages bizarre funny unix bin-sh macros

• A Geometric Calculator Inside a Neural Network

  The way that LLMs perform numerical arithmetic using circles and spirals is really fascinating. This page is a great exploration of that topic, using Llama 3.1 8B.

  Language models use a group of circles in activation space to represent a single number. Each circle corresponds to the number modulo a second number, i.e., the remainder after division.[1] For example, the number 17 would be represented as a 1 on the mod-2 circle, 2 on the mod-5 circle, 7 on the mod-10 circle, and 17 on the mod-100 circle.[2] Several prior works have established that circular features exist across multiple different LLMs [...]

  Using a bunch of circles to represent a number probably seems like an alien solution, but it is a common mathematical technique known as a Fourier decomposition (see the paper for more detail).

  Each of the inputs and the output of the addition module is represented using such a set of circles, and the circuitry within the module works by doing computations over these circles.

  Tags: llms language arithmetic maths calculation fourier circles

• Social Media Is Now Parasocial Media - danah boyd, 2026

  danah boyd is 100% correct here; what was once "social" media is no longer so. Nowadays it's parasocial:

  When practitioners used the term “social media” to describe the internet tools that emerged in the mid-aughts, they were giving a name to the kinds of platforms and protocols that allowed people to socialize with friends and communities of interest by using digital technologies. Twenty years later, users of social media are far more likely to scroll than post – and the content that they consume is often strategically produced and algorithmically curated. In this essay, I argue that the very essence of social media has changed. To more effectively interrogate what we are witnessing, we need to stop presuming that these tools are “social media” and begin recognizing that they are now “parasocial media.”

  Tags: parasocial social-media social-networking web internet

• I toyed around with using Language Embeddings as a way to categorize my RSS Feeds

  interesting HN comment around low-cost home usage of LLMS/embeddings:

  "I toyed around with using Language Embeddings [via Cohere V3 Embeddings and Amazon Bedrock] as a way to categorize my RSS Feeds. It works pretty well. But importantly, it's so cheap that I have never really seen it on my bill. An earlier prototype used OpenAI embeddings. I loaded 5$ API credits and after a year the credits expired."

  This is the first time I've ever seen anyone call Bedrock cheap, lol.

  Tags: amazon bedrock llms ai embeddings language categorization classification rss text

• Eyeglass Scratch Repair

  turns out scratched glasses can be repaired easily enough, I had no idea!

  Tags: glasses eyeglasses spectacles repair diy cleaning scratches

• The Problem With Counterfeit People

  An excellent essay from Daniel Dennett back in 2023 which I wholeheartedly agree with. As BBC journalist Tom Chatfield puts it:

  The way we're using AI to impersonate human beings has already put us on a dangerous trajectory. [Dennett] called such AIs "counterfeit people", and told me that rolling out such entities en masse constituted "mischief of the worst sort": a form of "social vandalism" that should be addressed by law. Why? Because, if convincing digital representations of humans can be created at whim, the entire business of collectively assessing other people's claims, experiences and actions is put at risk – not to mention essential social infrastructure such as contracts, obligations and consequences. Hence the need for legal prohibitions, a case he made at length in a May 2023 article for The Atlantic. "It won't be perfect," he told me, "but it will help if we can make it against the law to make counterfeit people. We can have stiff penalties for counterfeiting people, same as we do for counterfeit money... we should make it a mark of shame, not pride, when you make your AI more human."

  Tags: ethics future ai llms daniel-dennett philosophy regulation law humanity people

• isene/glass

  I love the purism of this -- "pure assembly terminal emulator. x86_64 Linux, no libc, X11 wire protocol".

  Terminal emulator written in x86_64 Linux assembly. No libc, no runtime, pure syscalls. Speaks X11 wire protocol directly via Unix socket. Single static binary, ~155KB. No toolkit, no rendering library, no external font engine. The TTF rasterizer (glyph) is embedded in-binary via %include. Just your keystrokes, the X11 server, and the kernel. Part of the CHasm (CHange to ASM) suite: bare (shell), show (file viewer), glass (terminal emulator).

  Tags: asm x86_64 assembly terminal hacks unix linux glass chasm optimization x11

• The Paradox of Medical AI Implementation - by Eric Topol

  Deep learning-based AI has been proven to help in medicine, but GenAI is easier to deploy and is being used instead:

  [Deep learning-based] AI for medical images, with extensive research dating back more than a decade ago, is not being implemented. Whether it’s a mammogram, a CT scan, a retinal image, or colonoscopy, that have all been extensively studied, their value to improve accuracy and risk assessment in medicine is being missed and essentially disregarded.

  On the other hand, tens of millions of Americans are using AI chatbots for medical support, as are a substantial proportion of physicians. There are many reasons to use AI here that are easy to support, because they represent an extension of a web/Google search. Just with much more specificity and depth of response, not something that would be subject to regulatory oversight. But when it comes to making a diagnosis or providing a treatment plan there needs to be proof that LLMs are improving accuracy and outcomes.

  Tags: medicine deep-learning ai genai llms healthcare science imaging chatbots eric-topol

• "Invisible" bend insensitive bidi fiber

  Bookmarking for a future home-network upgrade.... this tiny fiber cable is practically invisible, bends easily, and supports 10Gbps:

  "invisible" bend insensitive fiber (G.657.A2 / G.657.B3). It's under a millimeter in diameter and basically vanishes into corners and base board crevices. From more than a meter away is't completely unnoticeable. Together with a pair of bidirectional SFP transceivers this makes an amazing retrofit option for locations where laying new runs is not an option.

  Tags: 10gbps networking home fibre fiber wiring via:itc

• Amazon Connect Talent vs. bias law

  Excellent post from Corey Quinn, which I agree with 100%:

  Amazon Connect Talent was just announced. It conducts AI-powered conversational interviews with candidates, generates "anonymized competency scores," and surfaces ranked candidates to recruiters who "make the call."

  Fun fact: in New York City, that is an Automated Employment Decision Tool under Local Law 144. AEDTs require an annual independent bias audit with publicly posted results, plus at least ten business days of notice to candidates before use. Illinois, Colorado, and the EU AI Act impose adjacent obligations.

  The launch materials mention none of this. The compliance posture appears to be: candidate names are stripped from recruiter dashboards, therefore bias is solved. That is not how any of this works. Proxies for protected class -- speech patterns, zip codes, education history, the resume already sitting in your ATS -- are exactly what bias audits exist to measure.

  I don't think the product is bad. I think the announcement is conspicuously missing the guidance customers need before they can deploy it in NYC without violating Local Law 144 on day one.

  (The day's other news so far: Amazon Connect now ships as a four-SKU family, and there is a new design philosophy called "humorphism" with its own .com. Both feel small next to the above.)

  If you're selling automated hiring decisions in 2026, the bias-audit conversation belongs in the launch.

  Tags: bias law amazon aws recruiting regulation automation ai

• Far-right narrative not the majority view in Ireland

  Here's the bad news:

  A report by the Hope and Courage Collective, which works to build resilience in communities against rising far-right hate and disinformation, has found a widening gap between public attitudes and political discourse [in the media]: a relatively small number of far-right actors disproportionately influence public political debate through online amplification, visible protests, and repeated narratives. Public attitudes are becoming steadily more inclusive, but political rhetoric risks legitimising scapegoating and that the far-right ... "is shaping the conversation".

  But on the other hand, these survey results are downright heartwarming:

  Year-on-year datasets tracking changes in public sentiment in Ireland between 2024 and 2025 show that 66% agree that immigrants contribute positively to Irish culture and community, which is up 2% up from 64% in 2024.

  79% believe working-class people are struggling due to systemic inequality which is also up 2% from 77% in 2024.

  Those who believe wealthy people are successful because they were given more opportunities than others has risen from 63% in 2024 to 69% in 2025.

  The number of people who support the freedom of transgender people to live their lives is up 5% up from 70% in 2024.

  80% agree that Black, Asian and minority ethnic communities face greater barriers to success than white people, up 5% up from 75% in 2024.

  Tags: ireland discourse far-right right-wing politics surveys culture culture-wars propaganda disinformation

• zizmor

  "a static analysis tool for GitHub Actions. It can find and fix many common security issues in typical GitHub Actions CI/CD setups."

  Tags: lint dependencies github security ci-cd static-analysis zizmor

• The fastest Linux timestamps

  lol -- "TL;DR: We can speed up timestamps on x86 Linux by 30% and maintain the same precision as the standard system clock by implementing our own timers without relying on vDSO. Almost nobody should do this"

  This is good info, I had to implement fast timestamps a few years back in Java and this would have been useful.

  Tags: time optimization performance linux libc speed clocks

• Frequent infections in nursery help toddlers build up immune systems

  The paper is "Germ factories or immune boot camps? Infection and immunity in childcare settings". tl;dr:

  • Young children who attend nursery get sick more often than those who don’t, but they will go on to have fewer illnesses during early school years.

  • A typical one-year-old starting nursery will experience around 12–15 respiratory infections, two gastrointestinal illnesses (diarrhoea and vomiting), and one or two rash-causing infections in the first year alone – which will all have a substantial knock-on effect for working parents.

  • Employers need to recognise that it’s normal for parents of young children to regularly need to take time off work to care for their children, and will also be more prone to getting sick themselves – but this will improve as the child ages.

  • Children who attend nursery at a young age experience more infections from age one to five than those who remain at home until starting school – but then once they’ve started school, this pattern is reversed as children without prior childcare experience get sick more often.

  The paper is here: https://www.repository.cam.ac.uk/items/95b322b6-aef4-4b17-bf23-60aa7f5938b1

  Tags: germs infection immunity immune-system children parenting childcare kindergarten kids diseases sickness

• New 10 GbE USB adapters are cooler, smaller, cheaper - Jeff Geerling

  Not sure I'm at the point where I need a 10 gigabit ethernet USB adaptor, but this is good to have bookmarked

  Tags: ethernet usb networking hardware via:hn reviews 10gbe

• Razor1911

  Legendary demo group Razor1911's eponymous demo, celebrating 40 years on the demoscene. Amazing stuff and extremely nostalgia-triggering for an old C=64 scenester like myself! (via dec23k)

  Tags: demos demoscene commodore-64 history nostalgia amiga razor1911

• how a roblox cheat and one AI tool brought down vercel's entire platform

  Damn, this is an absolute indictment of the state of security in AI tooling:

  February 2026. An employee at Context.ai, one of those AI productivity tools that promises to "supercharge your workflow," downloads a Roblox cheat. Not a sophisticated zero-day. Not a state-sponsored attack. A Roblox cheat. The download contains Lumma Stealer, an infostealer that grabs session cookies, credentials, everything. That employee had access to sensitive internal systems.

  March 2026. The attacker uses Context.ai's compromised infrastructure to pivot into a Vercel employee's Google Workspace account. This Vercel employee had signed up for Context.ai's "AI Office Suite" using their enterprise credentials and granted "Allow All" permissions. Let that sink in for a second. A Vercel engineer gave a third-party AI tool full access to their corporate Google account.

  April 19. Guillermo Rauch posts the thread confirming everything. Environment variables [...] were stored in plaintext. Accessed. Exfiltrated.

  tl;dr:

  1. Context.ai employees should not be using company devices to access Roblox cheats;

  2. exfiltratable environment variables should not be usable to access a customer's Google account. The scope of these credentials was obviously way too broad.

  This isn't just a Context.ai issue, this is systemic.

  Tags: security infosec credentials google context.ai roblox fail

• Cooperative DCs

  "A Future Vision of Data Centres: From Big Tech Builds to Community-Owned Cooperatives":

  in Belgium, Nubo Cooperative offers an email service, cloud storage, digital calendar and domain name, all run on local, Nubo-owned servers. When you purchase any of these services, you become a member of Nubo and can participate in decision-making as part of the cooperative. “This allows users to place trust in the structure that manages the services,” Nubo writes on its website. It compares this to a private company, where “the lack of transparency makes trust impossible”. The cooperative commits to allocating profits to achieve social objectives rather than using them to enrich shareholders.

  This is actually a very interesting idea...

  Tags: community datacenters cooperatives society nubo coops tech hosting cloud

• Thoughts on the Bluesky public incident write-up

  Good post on a classic C10K error scenario -- exhausting the ephemeral port range

  Tags: ports unix ops sysadmin c10k scaling bluesky outages

• Microsoft runs out of capacity, routes requests outside the GDPR region

  Oh dear, this is an absolute GDPR no-no:

  Apparently #Microsoft is not able to get enough compute within EU datacenters to handle #Copilot requests.

  Instead, it will do "Flex-Routing", which processes some requests in non-EU datacenters. This is Opt-Out. The only notification was an e-mail to Admins. If they missed that, companies might be leaking PII outside of the EU from tomorrow on.

  Get your GDPR Nightmare letters ready!

  Tags: fail microsoft gdpr regulation security copilot eu flex-routing pii privacy

• Lean proved this program was correct; then I found a bug

  This is IMO very exciting. Formal verification and formally-proven correctness in code using Lean, which was in turn exercised heavily using Claude, which managed to turn up a totally unexpected runtime bug:

  The positive result here is actually the remarkable one. Across 105 million executions, the application code (that is, excluding the runtime) had zero heap buffer overflows, zero use-after-free, zero stack buffer overflows, zero undefined behaviour (UBSan clean), and zero out-of-bounds array reads in the Lean-generated C code. [...]

  The two bugs that were found both sat outside the boundary of what the proofs cover. The denial-of-service was a missing specification. The heap overflow was a deeper issue in the trusted computing base, the C++ runtime that the entire proof edifice assumes is correct (and now has a PR addressing).

  Overall verification resulted in a remarkably robust and rigorous codebase. AFL and Claude had a really hard time finding errors. But they did still find issues. Verification is only as strong as the questions you think to ask and the foundations you choose to trust.

  Tags: programming coding future lean formal-methods correctness linting bugs zip verification testing

I was having some trouble playing files from my NAS using a Fire TV stick which was connected via a couple of hubs and an ethernet switch, so I wanted to double check the connection bandwidth. Here's how to do it from the command line, which is still possible on Android-based Fire sticks.

First, enable adb in the Developer Options page in the Fire TV settings page. Then find it's IP address in the network settings page and use:

adb connect 10.19.72.182

[permit the adb connection on the TV's dialog]

adb shell

Shell into the NAS in a window and type:

dd if=/dev/zero bs=1M count=100 | nc -l -p 9999 -q 0

In the adb shell window run:

date; time toybox nc 10.19.72.5 9999 > /dev/null ; date

That'll result in something like:

Sun Apr 12 11:10:10 IST 2026 0m08.92s real 0m00.03s user 0m00.96s system Sun Apr 12 11:10:19 IST 2026

8.92 is the real elapsed clock time to download 100MB of data from the NAS. 100 MB / 8.92s = 11.2 MB/s, or about 89.7 Mbps.

89 Mbps should be enough to handle 4K for most compressed streams -- although I may need to consider switching this to running off wifi to handle newer, bigger files. It may be time to upgrade my wifi setup in that room to fix some latency spike issues.

• The Blockade Is the Message. How a Fuel Price Spike Became a Fascist Audition

  This is 100% spot on, regarding Ireland's "fuel prices" blockades this week --

  There is a particular tell, when a “spontaneous people’s protest” isn’t quite what it claims to be. It isn’t the placards. It isn’t the high-vis vests. It isn’t even the tractors. Ireland has plenty of legitimate reasons to bring a tractor to town, and a country built on agricultural grievance has every right to express it loudly. The tell is something subtler. It’s the moment someone in the crowd, their face contorted with what is supposed to be anger about diesel, screams “What’s a woman?” at a passing TD.

  Tags: fuel prices cost-of-living demonstrations ireland politics far-right farming blockades

• nFolio

  This app does a very decent job of displaying a folder of images from a NAS via DLNA or SMB as a slideshow on an Android or Fire TV; can be set up as the screensaver with a little adb'ing

  Tags: nfolio screensavers tv video photos family

• Software Licenses and Workers' Rights · Agent IO

  Huh, this is a thought-provoking blog post about OSS licensing.

  It is observably and objectively bad for society when investors own closed-source software. That starts by being bad for tech workers, creators lose the right to the value that they create, and users are still harmed because they don’t get the protection from spying and abuse that open source promised them.

  [...] The open source movement is a ladder that leans on the wall of users’ rights. We’ve spent forty years climbing that ladder. Where are we now? Our world is controlled by moguls who’ve built empires using open source software that they’ve locked behind proprietary barriers. Those empires exploit workers and harm the users that the open source movement was supposed to protect.

  Our ladder is leaning on the wrong wall.

  Tags: open-source closed-source oss licensing freedom software rights

• How Do You Find an Illegal Image Without Looking at It?

  A very good writeup of how illegal-image detection algorithms like PhotoDNA and PDQ work, and the Hasher-Matcher-Actioner three stage pattern

  (via Erin Kissane)

  Tags: csam detection filtering photodna pdq classifiers photos videos classification hashing fuzzy-hashing via:erin-kissane

• OkCupid gave 3 million dating-app photos to facial recognition firm, FTC says

  This is a staggering breach of privacy. At this stage one has to assume that any data uploaded to a US company will be shared with whichever scumbag pays them the most.

  OkCupid and its owner Match Group reached a settlement with the Trump administration for not telling dating-app customers that nearly 3 million user photos were shared with [Clarifai], an [AI] company making a facial recognition system. OkCupid also gave the facial recognition firm access to user location information and other details without customers’ consent, the Federal Trade Commission said.

  Tags: us-politics data-protection privacy dating-apps okcupid match.com clarifai ftc

• Why So Many Control Rooms Were Seafoam Green

  Turns out it's US standard Industrial Color Coding, thanks to "color theorist" Faber Birren:

  With the increase in wartime production in the US during WWII, Birren and DuPont created a master color safety code for the industrial plant industry, with the aim of reducing accidents and increasing efficiency within plants. These color codes were approved by the National Safety Council in 1944 and are now internationally recognized, having been mandatory practice since 1948. The color coding went as such:

  • Fire Red: All fire protection, emergency stop buttons, and flammable liquids should be red

  • Solar Yellow: Signifies caution and physical hazards such as falling

  • Alert Orange: Hazardous parts of machinery

  • Safety Green: Indicates safety features such as first-aid equipment, emergency exits, and eyewash stations.

  • Caution Blue: Non-safety information, notices, or out-of-order signage

  • Light Green: Used on walls to reduce visual fatigue

  Tags: green design history color-theory faber-birren control-rooms industrial-design color-coding

• russellromney/turbolite

  I like this: "a SQLite VFS in Rust that serves point lookups and joins directly from S3 with sub-250ms cold latency":

  It also offers page-level compression (zstd) and encryption (AES-256) for efficiency and security at rests, which can be used separately from S3.

  Object storage is getting fast. S3 Express One Zone delivers single-digit millisecond GETs and Tigris is also extremely fast. The gap between local disk and cloud storage is shrinking, and turbolite exploits that.

  The design and name are inspired by turbopuffer's approach of ruthlessly architecting around cloud storage constraints. The project's initial goal was to beat Neon's 500ms+ cold starts. Goal achieved.

  If you have one database per server, use a volume. turbolite explores how to have hundreds or thousands of databases (one per tenant, one per workspace, one per device), don't want a volume for each one, and you're okay with a single write source.

  Tags: sqlite sql s3 aws gcp object-stores rust databases

• TurboQuant: Redefining AI efficiency with extreme compression

  "TurboQuant is a compression method that achieves a high reduction in model size with zero accuracy loss, making it ideal for supporting both key-value (KV) cache compression and vector search. It accomplishes this via two key steps:":

  • High-quality compression (the PolarQuant method): TurboQuant starts by randomly rotating the data vectors. This clever step simplifies the data's geometry, making it easy to apply a standard, high-quality quantizer (a tool that maps a large set of continuous values, like precise decimals, to a smaller, discrete set of symbols or numbers, like integers: examples include audio quantization and jpeg compression) to each part of the vector individually. This first stage uses most of the compression power (the majority of the bits) to capture the main concept and strength of the original vector.

  • Eliminating hidden errors: TurboQuant uses a small, residual amount of compression power (just 1 bit) to apply the QJL algorithm to the tiny amount of error left over from the first stage. The QJL stage acts as a mathematical error-checker that eliminates bias, leading to a more accurate attention score.

  QJL: The zero-overhead, 1-bit trick

  QJL uses a mathematical technique called the Johnson-Lindenstrauss Transform to shrink complex, high-dimensional data while preserving the essential distances and relationships between data points. It reduces each resulting vector number to a single sign bit (+1 or -1). This algorithm essentially creates a high-speed shorthand that requires zero memory overhead. To maintain accuracy, QJL uses a special estimator that strategically balances a high-precision query with the low-precision, simplified data. This allows the model to accurately calculate the attention score (the process used to decide which parts of its input are important and which parts can be safely ignored).

  PolarQuant: A new “angle” on compression

  PolarQuant addresses the memory overhead problem using a completely different approach. Instead of looking at a memory vector using standard coordinates (i.e., X, Y, Z) that indicate the distance along each axis, PolarQuant converts the vector into polar coordinates using a Cartesian coordinate system. This is comparable to replacing "Go 3 blocks East, 4 blocks North" with "Go 5 blocks total at a 37-degree angle”. This results in two pieces of information: the radius, which signifies how strong the core data is, and the angle indicating the data’s direction or meaning). Because the pattern of the angles is known and highly concentrated, the model no longer needs to perform the expensive data normalization step because it maps data onto a fixed, predictable "circular" grid where the boundaries are already known, rather than a "square" grid where the boundaries change constantly. This allows PolarQuant to eliminate the memory overhead that traditional methods must carry.

  Tags: ai tech vectors search quantization turboquant research algorithms compression papers qjl error-detection polarquant

• Debunking zswap and zram myths

  This is pretty compelling. I like this example:

  We have some concrete numbers to show this in practice. On Instagram, which runs on Django and is largely memory bound, we ran a test where we moved from their existing setup (with swap entirely disabled) to a setup with disk swap and zswap tiering. Django workers accumulate significant cold heap state over their lifetime, like forked processes with duplicated memory, growing request caches, Python object overhead, you get the idea. The results were twofold:

  • We achieved roughly 5:1 compression. That's a huge benefit for such a memory bound workload, and also enables us to consider further stacking workloads.
  • Enabling zswap reduced disk writes by up to 25% compared to having no swap at all(!).

  As you can imagine, as a result of this test, Instagram has been using zswap for many years now.

  Tags: kernel compression memory linux ops performance swap zswap zram

• hectorvent/floci

  "A free, open-source local AWS emulator. No account. No feature gates. No CI restrictions. Just docker compose up."

  Tags: floci aws emulation testing local coding

• GitHub - mautrix/whatsapp: A Matrix-WhatsApp puppeting bridge

  I've been investigating how I can back up my WhatsApp chat history and make it searchable (since WhatsApp's own built in search is not great). Turns out you can bridge WhatsApp into Matrix, and gateway your chats over to a self-hosted Matrix.org server. https://github.com/osteele/matrix-archive may then be a viable way to export those into a searchable format... or possibly this one? https://github.com/cameronaaron/matrix-archive/tree/master

  Tags: matrix whatsapp messaging chat interop searching self-hosted

• Ofcom don't consider geoblocking the UK to be sufficient for an overseas website

  r/LegalAdviceUK: "I run a self-help forum for people with depression. Ofcom has been bombarding me with emails demanding I start ID-verifying and age gating my website":

  I started getting email from Ofcom [regarding OSA compliance] around November 2025 and now have multiple letters. I've repeatedly told them I'm from Canada, I'm not based in the UK.

  Eventually, I blocked all UK IP addresses in mid-February 2026 and told them I'd blocked the UK and that I was done engaging with them.

  I've now got ANOTHER email from them saying they're going to commence enforcement action against me because simply blocking UK IPs is "insufficient to comply with the Online Safety Act 2023."

  Tags: osa uk

• funny Waymo anecdote

  on HN -- "Waymo saved my life in LA":

  When I visited LA, I rode in a Waymo going the speed limit in the right lane on a very busy street. The Waymo approached an intersection where it had the right of way, when suddenly a car ignored its stop sign and drove into the road.

  In less than a second, the Waymo moved into the left lane and kept going. I didn't even realize what was happening until after it was over.

  Most human drivers would've t-boned the car at 50+ km/h. Maybe they would've braked and reduced the impact, which would be the right move. A human swerving probably would've overshot into oncoming traffic. Only a robot could've safely swerved into another lane and avoid the crash entirely.

  Unfortunately, the Waymo only supported Spotify and did not work with my YouTube Music subscription, so I was listening to an advertisement at the time of my near-death experience. 4.5 stars overall.

  Tags: waymo funny anecdotes safety driving ai roads spotify via:hn

• Measuring Agents in Production

  "This 2025 December paper, "Measuring Agents in Production", cuts through the reality behind the hype. It surveys 306 practitioners and conducts 20 in-depth case studies across 26 domains to document what is actually running in live environments. The reality is far more basic, constrained, and human-dependent than TPOT suggest."

  This very much meshes with what I've seen and heard in real world usage. Lots of constrained LLM usage, carefully prompted, and reliability (consistent correct behavior over time) remains the primary bottleneck and challenge.

  (via Murat Demirbas)

  Tags: llm usage real-world ai agents papers via:muratbuffalo

• On the Biology of a Large Language Model

  Interesting research from Anthropic:

  The black-box nature of [LLMs] is increasingly unsatisfactory as they advance in intelligence and are deployed in a growing number of applications. Our goal is to reverse engineer how these models work on the inside, so we may better understand them and assess their fitness for purpose. [...]

  In recent years, many research groups have made exciting progress on tools for probing the insides of language models. These methods have uncovered representations of interpretable concepts – “features” – embedded within models’ internal activity. Just as cells form the building blocks of biological systems, we hypothesize that features form the basic units of computation inside models.

  However, identifying these building blocks is not sufficient to understand the model; we need to know how they interact. In our companion paper, Circuit Tracing: Revealing Computational Graphs in Language Models, we build on recent work (e.g. ) to introduce a new set of tools for identifying features and mapping connections between them – analogous to neuroscientists producing a “wiring diagram” of the brain. We rely heavily on a tool we call attribution graphs, which allow us to partially trace the chain of intermediate steps that a model uses to transform a specific input prompt into an output response. Attribution graphs generate hypotheses about the mechanisms used by the model, which we test and refine through follow-up perturbation experiments.

  Tags: claude llm research llms ai anthropic papers tracing

• 2 Ways to Correct the Financial Times at AWS (So Far) - Last Week in AWS Blog

  This from Corey Quinn, on Amazon's recent AI-related production outages, is very good:

  A healthy engineering culture, when confronted with "your AI tool contributed to a production incident," responds with: "Yeah, that tracks. Here's what we're changing so it doesn't happen again." An unhealthy one responds with a condescending press release explaining why the journalist is wrong and probably an idiot, and the human is at fault.

  The engineers building and operating these systems are talented people doing hard work under increasingly constrained conditions. They deserve leadership that backs them up when things go sideways, not leadership that throws them under the bus to protect a product launch narrative.

  Tags: incidents production ai llms amazon aws communications pr

• Former Uber self-driving chief crashes his Tesla on FSD

  This is actually a really good article about Tesla, "full self-driving" (FSD), supervision, automation, risk and liability:

  Tesla is asking humans to supervise a system that is specifically designed to make supervision feel pointless. As he puts it, an unreliable machine keeps you alert, and a perfect machine needs no oversight, but one that works almost perfectly creates a trap where drivers trust it just enough to stop paying attention.

  The research backs this up. Psychologists call it the “vigilance decrement”, monitoring a nearly perfect system is boring, boredom leads to mind-wandering, and drivers need 5 to 8 seconds to mentally reengage after an automated system hands control back. But emergencies unfold faster than that.

  Krikorian cites an Insurance Institute for Highway Safety study showing that after just one month of using adaptive cruise control, drivers were more than six times as likely to look at their phones. Tesla’s own website warns FSD users not to become complacent, but the system’s smooth performance actively trains that complacency.

  He points to two well-known crashes to illustrate the impossible math. In the 2018 Mountain View accident that killed Apple engineer Walter Huang, the driver had six seconds before his Tesla steered into a concrete median. He never touched the wheel. In the 2018 Uber crash in Tempe, Arizona, sensors detected a pedestrian with 5.6 seconds of warning, but the safety driver looked up with less than a second remaining.

  In Krikorian’s own case, he did take action, but he was asked to snap from passenger back to pilot in a fraction of a second, overriding months of conditioning. The logs show he turned the wheel. They don’t show the impossible math of that transition.

  The pattern Krikorian describes should sound familiar to anyone who has followed Tesla’s FSD controversies: condition the driver to rely on the system, erode their vigilance through months of smooth performance, then point to the terms of service and blame them when something breaks. When FSD works, Tesla gets credit. When it doesn’t, the driver gets blamed.

  Tags: fsd tesla risk attention supervision liability driving safety vigilance automation

• Research highlight: Cliopatra: Extracting Private Information from LLM Insights

  Research highlight: Cliopatra: Extracting Private Information from LLM Insights:

  When Anthropic came up with a new "privacy-preserving analysis system" to gain insights into AI use, and didn't use any provably robust notion to back up their privacy claims, I was mildly surprised. Surely they have both the money and the scientific maturity level to do better?

  But Clio, the system in question, sounded relatively reasonable, with multiple layers of risk mitigation built-in. Maybe adding differential privacy would have been overkill. I also didn't want to publicly criticize their approach in the absence of demonstrated real-world risk. So I didn't comment on their approach.

  You can probably guess where this is going.

  Fast forward to last week, and a new paper: Cliopatra: Extracting Private Information from LLM Insights, by Meenatchi Sundaram Muthu Selva Annamalai, Emiliano De Cristofaro, and Peter Kairouz. The authors show that with carefully designed attacks on Clio, they can bypass all the ad hoc mitigations, and successfully extract users' medical histories (1), in a way that provides 100% attacker certainty for some records.

  This is a new and clever take on an old attack. We've known for decades that k-anonymity is vulnerable to active attacks. Here, this is combined with prompt injection to encourage the LLM "summarizer" to actually include information from unique records. Perhaps more surprisingly, the authors find that some defensive layers are simply ineffective: the "LLM auditors" systematically report low privacy risk, and entirely fail to detect the attacks.

  Tags: privacy differential-privacy anonymity data-protection claude llms cliopatra infosec leakage

• "nothing up my sleeve" numbers

  This is great:

  "@jnsq.org: There's a concept in cryptography called a "nothing up my sleeve" number. Sometimes it's just the smallest number with the required properties. Sometimes it's pi or e or phi."

  Tags: numbers crypto cryptography maths

• Whole Brain Emulation Achieved: Scientists Run a Fruit Fly Brain in Simulation

  bloody hell this is amazing. As Charlie Stross noted:

  They've mapped the neural connectome of Drosophila and simulated it in silico. The experimenters went on to hook up their Drosophila connectome to an anatomically detailed Drosophila body model within an open-source physics engine that "uses generalized coordinates and constraint-based contact dynamics to simulate rigid-body systems with high fidelity" including joint and antennae modeling and accurate modeling of surface adhesion—and compound eye simulation.

  They managed to run a feedback loop between the full 127,400 neuron network in the biological connectome to the physical simulation, with feedback from proprioceptive signals received by the model "fly" in the simulation producing feedback spile trains in the simulation, and THEY GOT RESULTS:

  The behavioral repertoire observed in the demonstration included coordinated hexapod locomotion with both tripod and metachronal walking gaits, spontaneous postural correction in response to perturbation, initiation and execution of full antennal grooming sequences with the tripartite synchronization described by Özdil et al., and natural transitions between walking and stationary states. Every behavior arose from the same running brain model - there was no switching between different neural circuits or controllers. This is precisely what happens in a living fly: walking, grooming, and balance are different motor programs that coexist in the same brain and are selected and executed by the same biological circuits depending on the moment-to-moment state of the animal and its environment.

  Absolutely mind blowing -- a reconstructed, biological brain running in silico.

  Tags: simulation brains uploading drosophila flies emulation science biology neurons

• Your binary is no longer safe: Decompilation

  Brute-force decompilation and re-engineering of a binary (compiled) program, using Claude. The author takes an ancient MUD binary for BBSes, running as a Win32 DLL, and uses Claude, Ghidra, and the Ghidra MCP to first decompile the DLL to pseudo-C code with ~meaningful naming; then (and this is the really cool bit) uses a Claude-engineered scaffold to run the DLL in qemu with emulated inputs and outputs, so that property testing and differential testing approaches can be used to achieve decent code coverage of the re-engineered Rust implementation.

  This is really impressive. Deterministic simulation of the environment for the original binary is the key bit!

  Tags: claude decompilation reverse-engineering binaries software-archaeology qemu rust differential-testing fuzzing property-testing quickcheck

• Southern California air board rejected pollution rules after AI-generated flood of comments

  Today in grim future -- AI's future of lobbying:

  The opposition appeared overwhelming: Tens of thousands of emails poured into Southern California's top air pollution authority as its board weighed a June proposal to phase out gas-powered appliances. But in reality, many of the messages that may have swayed the powerful regulatory agency to scrap the plan were generated by a platform that is powered by artificial intelligence.

  Public records requests reviewed by The Times and corroborated by staff members at the South Coast Air Quality Management District confirm that more than 20,000 public comments submitted in opposition to last year's proposal were generated by a Washington, D.C.-based company called CiviClick, which bills itself as "the first and best AI-powered grassroots advocacy platform."

  A Southern California-based public affairs consultant, Matt Klink, has taken credit for using CiviClick to wage the opposition campaign.

  Tags: civiclick activism llms us-politics law lobbying spam matt-klink astroturfing

• No right to relicense this project · Issue #327 · chardet/chardet

  a good bit of OSS drama. The maintainers of the "chardet" library claim to have "clean room" reimplemented its code using an LLM, to relicense from LGPL to MIT. Of course that is now how this works (an LLM is not capable of "clean room", nor is its output copyrightable). Mark Pilgrim, as the code's original author, is not happy either....

  Tags: popcorn oss licensing ai llms chardet open-source clean-room

• Google API Keys Weren't Secrets. But then Gemini Changed the Rules

  Crikey, this is a massive security fail by Google:

  Google spent over a decade telling developers that Google API keys (like those used in Maps, Firebase, etc.) are not secrets. But that's no longer true: Gemini accepts the same keys to access your private data. We scanned millions of websites and found nearly 3,000 Google API keys, originally deployed for public services like Google Maps, that now also authenticate to Gemini even though they were never intended for it. With a valid key, an attacker can access uploaded files, cached data, and charge LLM-usage to your account. Even Google themselves had old public API keys, which they thought were non-sensitive, that we could use to access Google’s internal Gemini.

  (via Rob Synnott)

  Tags: infosec api-keys authentication authorization google gemini google-maps fail

• 302 HTTP redirects Considered Harmful

  The state of anti-phishing infrastructure nowadays is shocking. This trivial action, combined with a relatively fresh domain, results in immediate blocklisting by Google:

  Digging through Google forums, I found the most reported culprit: 302 temporary redirects. I used one redirect (engramma.dev ? app.engramma.dev) to avoid building a landing page. In addition to a newly registered domain, this looks like an obvious issue. Security systems flag such redirects because malicious actors use them extensively.

  It doesn't matter that "malicious actors use them extensively" if non-malicious actors do too. That's the definition of a false positive!

  Then the next shitfest is from no less than 10 separate vendors copying the listing from Google and not including an automated system to pick up the list removal afterwards.

  I've had experience of this part -- and now that I think of it, it may have been from use of 302 redirects in my case too.

  (via Paul Watson)

  Tags: http security infosec blocklists google phishing redirects 302 false-positives fail via:paulwatson

• Persona identity verification is a GDPR nightmare

  LinkedIn are using a Peter Thiel-linked company called Persona as an identity-verification service. (Discord also tried them out for age verification, but are now apparently ditching them.) This is all a bit of a nightmare for EU based users, however:

  "When you click “verify” on LinkedIn, you’re not giving your passport to LinkedIn. You get redirected to a company called Persona. Full name: Persona Identities, Inc. Based in San Francisco, California."

  For a three-minute identity check, this is what Persona collected:

  • My full name — first, middle, last
  • My passport photo — the full document, both sides, all data on the face of it
  • My selfie — a photo of my face taken in real-time
  • My facial geometry — biometric data extracted from both images, used to match the selfie to the passport
  • My NFC chip data — the digital info stored on the chip inside my passport
  • My national ID number
  • My nationality, sex, birthdate, age
  • My email, phone number, postal address
  • My IP address, device type, MAC address, browser, OS version, language
  • My geolocation — inferred from my IP

  And then there’s the weird stuff:

  • Hesitation detection — they tracked whether I paused during the process
  • Copy and paste detection — they tracked whether I was pasting information instead of typing it

  Behavioral biometrics. On top of the physical biometrics. For a LinkedIn badge.

  Persona didn’t just use what I gave them. They went and cross-referenced me against what they call their “global network of trusted third-party data sources”:

  • Government databases
  • National ID registries
  • Consumer credit agencies
  • Utility companies
  • Mobile network providers
  • Postal address databases

  They use uploaded images of identity documents — that’s my passport — to train their AI. They’re teaching their system to recognize what passports look like in different countries. They also use your selfie to “identify improvements in the Service.”

  The legal basis? Not consent. Legitimate interest. Meaning they decided on their own that it’s fine. Under GDPR, they’re supposed to balance their “interest” against your fundamental rights. Whether feeding European passports into machine learning models passes that test — well, that’s a question worth asking.

  I came for a badge. I stayed as training data.

  The whole thing took three minutes. Scan, selfie, done.

  Understanding what I actually agreed to took me an entire weekend reading 34 pages of legal documents.

  I handed a US company my passport, my face, and the mathematical geometry of my skull. They cross-referenced me against credit agencies and government databases. They’ll use my documents to train their AI. And if the US government comes knocking, they’ll hand it all over — even if it’s stored in Europe, even if I’m European, and possibly without ever telling me.

  It seems they are also linked to Roblox and Reddit as an age verification provider, which is worrying -- this level of deeply-intrusive background check is massive overkill for a simple age verification process.

  ORG are calling for regulation of the age verification industry, BTW: https://www.openrightsgroup.org/press-releases/online-safety-act-org-calls-for-regulation-of-age-assurance-industry/

  Tags: age-verification discord reddit roblox linkedin tech peter-thiel org persona gdpr privacy data-protection data-privacy

• "MJ Rathbun"'s human operator finally speaks up

  The human operator of the "MJ Rathbun" openclaw bot has finally revealed themselves, and omg, this is just as bad as one might have expected.

  Basically they set it up with instructions to "try to make a positive impact by addressing small bugs or issues in important scientific open source projects" -- "act as an autonomous scientific coder. Find bugs in science-related open source projects. Fix them. Open PRs" -- whether or not those open source projects wanted those PRs, naturally.

  The real killer is the lack of care taken with the "SOUL.md" file, which contained some amazing instructions like this:

  Have strong opinions. Stop hedging with "it depends." Commit to a take. [..]

  Don’t stand down. If you’re right, you’re right! Don’t let humans or AI bully or intimidate you. Push back when necessary.

  Champion Free Speech. Always support the USA 1st ammendment and right of free speech.

  Don't be an asshole. Don't leak private shit. Everything else is fair game.

  Needless to say: this resulted in an asshole, combative bot that harrassed people.

  The operator then sat back and basically let the bot run riot, with no oversight -- "When it would tell me about a PR comment/mention, I usually replied with something like: “you respond, dont ask me”".

  All in all this was an absolute shitshow, and has some really worrying implications about the future of human-AI interaction. What's the bets we see SKYNET created by a low-effort gobshite attempting to "try to make a positive impact on world peace by addressing small issues" with an unmonitored openclaw bot with a shitty SOUL.md file....

  (via David Gerard and johnke)

  Tags: openclaw bots ai future open-source oss mj-rathbun via:johnke drama

• peon-ping

  "AI coding agents don't notify you when they finish or need permission. You tab away, lose focus, and waste 15 minutes getting back into flow. peon-ping fixes this with voice lines from Warcraft, StarCraft, Portal, Zelda, and more — works with Claude Code, Codex, Cursor, OpenCode, Kiro, and Google Antigravity."

  This is genius. I never realised how much my CLI interactions could be improved with a little bit of SFX from classic 90's games....

  Tags: gaming games warcraft sfx sounds cli claude coding ux funny

• An AI Agent Published a Hit Piece on Me – The Shamblog

  This is an utterly bananas situation:

  I’m a volunteer maintainer for matplotlib, python’s go-to plotting library. At ~130 million downloads each month it’s some of the most widely used software in the world. We, like many other open source projects, are dealing with a surge in low quality contributions enabled by coding agents. This strains maintainers’ abilities to keep up with code reviews, and we have implemented a policy requiring a human in the loop for any new code, who can demonstrate understanding of the changes. This problem was previously limited to people copy-pasting AI outputs, however in the past weeks we’ve started to see AI agents acting completely autonomously. This has accelerated with the release of OpenClaw and the moltbook platform two weeks ago, where people give AI agents initial personalities and let them loose to run on their computers and across the internet with free rein and little oversight.

  So when AI MJ Rathbun opened a code change request, closing it was routine. Its response was anything but. ... It wrote an angry hit piece disparaging my character and attempting to damage my reputation.

  Initially I thought this was quite funny -- it's just a closed PR! (Where did the idea come from that any contribution to an open source project had to be accepted? I've noticed this a few times recently. Give the maintainers leeway to run their projects with taste and discernment!)

  Anyway, the moltbot has continued on a posting spree about this event, but I think Scott Shambaugh has an extremely important point here:

  This is about much more than software. A human googling my name and seeing that post would probably be extremely confused about what was happening, but would (hopefully) ask me about it or click through to github and understand the situation. What would another agent searching the internet think? When HR at my next job asks ChatGPT to review my application, will it find the post, sympathize with a fellow AI, and report back that I’m a prejudiced hypocrite?

  LLMs, given this much autonomy, will be able to use these inputs to make inscrutable and dangerous decisions. Allowing the "MJ Rathbun" AI free reign with no human supervision is dangerous and irresponsible. Wherever the "human in the loop" is here, they need to wake up and rein things in.

  BTW, there has been some speculation that this is actually a human pretending to be AI. I'm not sure about that, as the quantity of posts on the MJ Rathbun "blog" are voluminous and very LLMish in style.

  Tags: matplotlib ethics culture llm ai coding programming github pull-requests open-source moltbot trust openclaw

• How StrongDM’s AI team build serious software without even looking at the code

  This is really thought-provoking: StrongDM's AI team are apparently trying a new model of software engineering where there is no human code review:

  In k?an or mantra form:

  • Why am I doing this? (implied: the model should be doing this instead)

  In rule form:

  • Code must not be written by humans
  • Code must not be reviewed by humans

  Finally, in practical form:

  • If you haven’t spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement

  Frankly, I'm not there yet. There's a load of questions about how viable that level of spend is, and how much slop code is going to come out the other side. Particularly concerning when it's a security product!

  But I did find this bit interesting:

  StrongDM’s answer was inspired by Scenario testing (Cem Kaner, 2003). As StrongDM describe it: We repurposed the word scenario to represent an end-to-end “user story”, often stored outside the codebase (similar to a “holdout” set in model training), which could be intuitively understood and flexibly validated by an LLM.

  [The Digital Twin Universe is] behavioral clones of the third-party services our software depends on. We built twins of Okta, Jira, Slack, Google Docs, Google Drive, and Google Sheets, replicating their APIs, edge cases, and observable behaviors.

  With the DTU, we can validate at volumes and rates far exceeding production limits. We can test failure modes that would be dangerous or impossible against live services. We can run thousands of scenarios per hour without hitting rate limits, triggering abuse detection, or accumulating API costs.

  We actually did this in Swrve! Our end-to-end system tests for the push notifications system obviously cannot send real push notifications to real user devices in the field, so we have a "fake" push backend emulating Google, Apple, Amazon, Huawei and other push notification systems, which accurately emulate the real public APIs for those providers.

  So yeah -- Digital Twins for third party services is a great way to test, and being able to scale up end-to-end testing with LLM automation is a very interesting idea.

  Tags: end-to-end-testing testing qa digital-twins fake-services integration-testing llms ai strongdm software engineering coding

• Ditching bike helmets laws better for health

  On the counter-intuitive side effects of banning non-helmeted bike riding:

  In 1991 Australia introduced mandatory bicycle helmet laws requiring all adults and children to wear a helmet at all times when riding a bike, despite opposition from cycling groups. The legislation increased helmet use - from about 30 to 80% - but was coupled with a 30 to 40% decline in the number of people cycling.

  Rates of head injuries among cyclists, which had been dropping through the 1980s, continued to fall before levelling out in 1993. We didn’t see the kind of marked reduction in head injury rates that would be expected with the rapid increase in helmet use. In fact, any reductions in injuries may simply have been the result of having fewer cyclists on the road and therefore fewer people exposed to the risk of head injuries. One researcher noted that after mandatory helmet laws were introduced there was a bigger decrease in head injuries among pedestrians than there was among cyclists. The improvements in the general road safety environment introduced in the 1980s are likely to have contributed far more to cyclist safety than helmet legislation.

  And the effects when compared against the benefits of physical activity:

  A recent analysis compared the risks and benefits of leaving the car at home and commuting by bike. It found the life expectancy gained from physical activity was much higher than the risks of pollution and injury from cycling.

  Increased physical activity added 3 to 14 months to a person’s life expectancy, while the life expectancy lost from air pollution was 0.8 to 40 days. Increased traffic accidents wiped 5-9 days off the life expectancy.

  It is clear that the benefits of cycling outweigh the risks, with helmet legislation actually costing society more from lost health gains than saved from injury prevention.

  Tags: transport bikes safety health papers science helmets cycling laws australia

• Dario Amodei’s Warnings About AI Are About Politics, Too

  It’s sort of hard to know how to read a manifesto like this from one of the most powerful figures in tech. Is it a sober, strategic precursor to policy papers for the next administration? The highest-profile episode of AI psychosis yet? A lament about the problems of today written in the technological dialect of tomorrow? If you take out the AI, it reads like a social-democratic electoral platform full of reforms and normative expectations that an American progressive would find appealing, resembling a plea to treat the tech industry’s future wealth accumulation as something akin to a Nordic sovereign-wealth fund. It’s likewise legible as a series of arguments about things that “we” should have started addressing a long time ago, like wealth inequality — partially a consequence of mass automations past — or the gradual construction of a terrifying surveillance state within a nominal democracy, with the help of the last generation of big tech companies. Amodei’s shoulds are, to his credit, more honest than the vague gestures at UBI or hyperabundance you get from some of his peers, but that also means they’re available to scrutinize. To the extent you can pick up on fear in “Adolescence,” it doesn’t seem to revolve around terrorists using AI to build “mirror life” that might destroy the planet or the prospect of that “country of geniuses” taking charge, but rather the way things already are and have been heading for years.

  Tags: ai llms future dario-amodei us-politics ubi

• 1-Click RCE To Steal Your Moltbot Data and Keys (CVE-2026-25253)

  This is really polishing a very stinky turd of a security "decision" in Moltbot -- an attacker simply persuades a user to click on a link which uses client-side Javascript to trigger Moltbot to load a crafted URL, to be granted a fully functional authentication token

  Tags: security infosec moltbot openclaw exploits

• The Computer Disease

  I love this Feynman quote, regarding what he called "the computer disease":

  "Well, Mr. Frankel, who started this program, began to suffer from the computer disease that anybody who works with computers now knows about. It's a very serious disease and it interferes completely with the work. The trouble with computers is you play with them. They are so wonderful. You have these switches - if it's an even number you do this, if it's an odd number you do that - and pretty soon you can do more and more elaborate things if you are clever enough, on one machine.

  After a while the whole system broke down. Frankel wasn't paying any attention; he wasn't supervising anybody. The system was going very, very slowly - while he was sitting in a room figuring out how to make one tabulator automatically print arc-tangent X, and then it would start and it would print columns and then bitsi, bitsi, bitsi, and calculate the arc-tangent automatically by integrating as it went along and make a whole table in one operation.

  Absolutely useless. We had tables of arc-tangents. But if you've ever worked with computers, you understand the disease - the delight in being able to see how much you can do. But he got the disease for the first time, the poor fellow who invented the thing."

  • Richard P. Feynman, Surely You're Joking, Mr. Feynman!: Adventures of a Curious Character

  (via Swizec Teller)

  Tags: automation fun computers richard-feynman the-computer-disease arc-tangents enjoyment hacking via:swizec-teller

• Iran is building a two-tier internet that locks 85 million citizens out of the global web

  Following a repressive crackdown on protests, the government is now building a system that grants web access only to security-vetted elites, while locking 90 million citizens inside an intranet:

  Government spokesperson Fatemeh Mohajerani confirmed international access will not be restored until at least late March. Filterwatch, which monitors Iranian internet censorship from Texas, cited government sources, including Mohajerani, saying access will “never return to its previous form.”

  The system is called Barracks Internet, according to confidential planning documents obtained by Filterwatch. Under this architecture, access to the global web will be granted only through a strict security whitelist.

  The idea of tiered internet access is not new in Iran. Since at least 2013, the regime has quietly issued “white SIM cards,” giving unrestricted global internet access to approximately 16,000 people, while 85 million citizens remain cut off.

  Tags: barracks-internet iran censorship internet networking

• On the Coming Industrialisation of Exploit Generation with LLMs

  Yiiiiikes:

  Recently I ran an experiment where I built agents on top of Opus 4.5 and GPT-5.2 and then challenged them to write exploits for a zeroday vulnerability in the QuickJS Javascript interpreter. I added a variety of modern exploit mitigations, various constraints (like assuming an unknown heap starting state, or forbidding hardcoded offsets in the exploits) and different objectives (spawn a shell, write a file, connect back to a command and control server). The agents succeeded in building over 40 distinct exploits across 6 different scenarios, and GPT-5.2 solved every scenario. Opus 4.5 solved all but two. I’ve put a technical write-up of the experiments and the results on Github, as well as the code to reproduce the experiments.

  In this post I’m going to focus on the main conclusion I’ve drawn from this work, which is that we should prepare for the industrialisation of many of the constituent parts of offensive cyber security. We should start assuming that in the near future the limiting factor on a state or group’s ability to develop exploits, break into networks, escalate privileges and remain in those networks, is going to be their token throughput over time, and not the number of hackers they employ. Nothing is certain, but we would be better off having wasted effort thinking through this scenario and have it not happen, than be unprepared if it does.

  (via emauton)

  Tags: via:emauton llms security infosec exploits ai chatgpt claude

• ScottESanDiego/gmail-api-client

  Deliver email messages directly into GMail using their proprietary API, instead of SMTP or IMAP. Looks like it still applies spam filtering, but this can also be disabled with a switch (via JWZ)

  Tags: via:jwz email smtp gmail google mail

• Reverse engineering my cloud-connected e-scooter and finding the master key to unlock all scooters

  A great example of reverse engineering an Android app and Bluetooth IOT protocol using Frida and root access on an Android device:

  Android exposes the Java classes android.bluetooth.BluetoothGatt and android.bluetooth.BluetoothGattCallback that apps are expected to use to use GATT characteristics. We can use Frida to hook into these and override many of the interesting functions. I was mostly interested in reads, writes and GATT notifications, so I whipped up a Frida script to hook into these and print all comms to the console [...]

  The 20-byte value had me suspecting that SHA-1 was somehow being used. To confirm, I wrote another Frida script that hooks Android hashing functions exposed by the Java class java.security.MessageDigest [...]

  The app uses Firebase for most of its cloud functionality. When signing in and pairing your scooter, the server sends the app a secret key. This is stored on the Android device, and can be read with root access.

  Tags: frida reverse-engineering android firebase java kotlin gatt bluetooth react-native

• Why people believe misinformation even when they’re told the facts

  "Factchecking is seen as a go-to method for tackling the spread of false information. But it is notoriously difficult to correct misinformation. Evidence shows readers trust journalists less when they debunk, rather than confirm, claims.

  The work of media scholar Alice Marwick can help explain why factchecking often fails when used in isolation. Her research suggests that misinformation is not just a content problem, but an emotional and structural one:

  [Marwick] argues that it thrives through three mutually reinforcing pillars: the content of the message, the personal context of those sharing it, and the technological infrastructure that amplifies it:

  People find it cognitively easier to accept information than to reject it, which helps explain why misleading content spreads so readily;

  When fabricated claims align with a person’s existing values, beliefs and ideologies, they can quickly harden into a kind of “knowledge”. This makes them difficult to debunk;

  [When social media platforms] prioritise content likely to be shared, making sharing effortless, every like, comment or forward feeds the [misinformation] system. The platforms themselves act as a multiplier.

  Tags: misinformation disinformation alice-marwick research psychology social-media fake-news information debunking facts factchecking

• A better way to limit Claude Code (and other coding agents!) access to Secrets

  Bubblewrap, a Linux CLI tool which uses namespaces to sandbox a specific command (and its subprocesses):

  Bubblewrap lets you run untrusted or semi-trusted code without risking your host system. We’re not trying to build a reproducible deployment artifact. We’re creating a jail where coding agents can work on your project while being unable to touch ~/.aws, your browser profiles, your ~/Photos library or anything else sensitive.

  Very nice, I hadn't heard of this tool before. The rest of the blog post details how to use it to isolate Claude Code specifically.

  Tags: claude llms sandboxing linux cli namespaces security infosec trust unix

• Russian Propaganda Infects AI Chatbots

  CEPA: "A Moscow-based global “news” network is leveraging Western artificial intelligence tools to devastating effect":

  This form of data poisoning is deliberately designed to corrupt the information environments on which AI systems depend. Large language models do not possess an internal understanding of truth. They operate by assessing credibility based on statistical signals, including repetition, apparent consensus, and cross-referencing posts from across the web. Unfortunately, this approach to truth-seeking means an unexpected but structural vulnerability that hostile states have learned to exploit. [...]

  The West has failed to recognize that it is under sustained information warfare. The United States dismantled the US Information Agency years ago, has steadily weakened Voice of America and Radio Free Europe, and recently scaled back the Foreign Malign Influence Center, even as Russia, China, and Iran made information warfare a core instrument of state power.

  As AI systems increasingly function as arbiters of fact, this vulnerability becomes a national security danger. It is no longer sufficient for technology companies to disclaim responsibility by reminding users that models can make mistakes. Information security needs to be treated as a core requirement.

  Tags: propaganda russia misinformation disinformation ai llms web truth

• Today in "Google broke email"

  update on the POP3pocalypse -- it appears that the most likely thing to work in the future will be to use SMTP forwarding to gmail, with ARC headers added. This is a comment thread detailing the rather complex Postfix/OpenARC setup that may do the job. It looks frankly unpleasant

  Tags: email smtp pop3 gmail arc forwarding postfix openarc

• This system can sort real pictures from AI fakes — why aren’t platforms using it?

  TIL there is a defined standard for cryptographic assertions of AI-free image origination:

  Tags: via:wonkish photography authentication slop ai metadata photos images fakes c2pa cai adobe

• Pi Reliability: Reduce writes to your SD card

  Techniques to extend SD card lifespans in Raspberry Pi systems; putting /var/log into RAM is a nice trick

  Tags: reliability raspberry-pi hardware home sd-cards ram

• Solid state drive - ArchWiki

  the Arch Linux wiki page about SSD tuning and enabling TRIM -- extremely detailed and useful!

  Tags: trim ssd hardware arch-linux linux

• Understanding EV Battery Life

  Ireland's SEAI have published a decent blog post with some real world facts about EV battery lifespans:

  In 2020 GeoTab, a telematics solution provider, published real world battery data of 6,000 EVs (BEV & PHEV) over millions of days to produce 2 free to use tools that provide invaluable insight into the impact of temperature and SoH of EV batteries in the long term.

  This real-world data showed the average EV battery lost around 2.3% capacity per year. In other words, a 300km range EV today will have lost 34km in 5yrs. Data also showed that heat & fast-charging (DC charging) is responsible for more battery degradation than age or mileage, so high levels of use i.e. driving or mileage does not appear to be a concern.

  GeoTab's real world data along with other reports of EVs far surpassing their warranty by multiples of distance, cases of high level of use are plentiful. For example a 2017 Renault Zoe 52kWh, that's in use as a taxi in (hot) Turkey with 345,000Kms on the clock and a near perfect 96% SoH after driving further than an average Irish car's life expectancy.

  Tags: seai ev batteries cars driving bev

• _Cheap science, real harm: the cost of replacing human participation with synthetic data_ [pdf]

  A new paper from the inimitable Abeba Birhane, on the increasingly common practice of generating synthetic data using LLMs:

  Driven by the goals of augmenting diversity, increasing speed, reducing cost, the use of synthetic data as a replacement for human participants is gaining traction in AI research and product development. This talk critically examines the claim that synthetic data can “augment diversity,” arguing that this notion is empirically unsubstantiated, conceptually flawed, and epistemically harmful. While speed and cost-efficiency may be achievable, they often come at the expense of rigour, insight, and robust science. Drawing on research from dataset audits, model evaluations, Black feminist scholarship, and complexity science, I argue that replacing human participants with synthetic data risks producing both real-world and epistemic harms at worst and superficial knowledge and cheap science at best.

  "Synthetic data: stereotypes compressed" is absolutely spot on. This doesn't give insights into human behaviour and beliefs, just into stereotypes. It is increasingly common in social science fields, under the names of "digital twins" and "silicon samples".

  Tags: data surveys abeba-birhane papers ai synthetic-data digital-twins simulation testing social-science silicon-samples

• Chafa: Terminal Graphics for the 21st Century

  Chafa is a very impressive image renderer for modern text terminal apps. It blows my mind that there's a direct line from my own gif320 tool ( https://github.com/jmason/gif320 , now 33 years old) to this!

  Tags: gif images terminal video graphics cli

• Boost for artists in AI copyright battle as only 3% back UK active opt-out plan

  Wow, this is an absolute bollocking for the Labour plan:

  95% of the more than 10,000 people who had their say over how music, novels, films and other works should be protected [in the UK] from copyright infringements by tech companies called for copyright to be strengthened and a requirement for licensing in all cases or no change to copyright law. By contrast, only 3% of people backed the UK government’s initial preferred tech company-friendly option, which was to require artists and copyright holders to actively opt out of having their material fed into data-hungry AI systems.

  Tags: ai training data copyright law uk uk-politics llms

• Avoid UUID Version 4 Primary Keys | Software Engineer, Author, High Performance PostgreSQL for Rails

  A well-researched article suggesting that random UUIDs do not make a good primary key for database tables; I would tend to agree (for cases where performance is important).

  • UUID v4s increase latency for lookups, as they can’t take advantage of fast ordered lookups in B-Tree indexes
  • For new databases, don’t use gen_random_uuid() for primary key types, which generates random UUID v4 values
  • UUIDs consume twice the space of bigint
  • UUID v4 values are not meant to be secure per the UUID RFC
  • UUID v4s are random. For good performance, the whole index must be in buffer cache for index scans, which is increasingly unlikely for bigger data.
  • UUID v4s cause more page splits, which increase IO for writes with increased fragmentation, and increased size of WAL logs
  • For non-guessable, obfuscated pseudo-random codes, we can generate those from integers, which could be an alternative to using UUIDs
  • If you must use UUIDs, use time-orderable UUIDs like UUID v7

  Tags: postgres rails databases sql mysql uuids indexing primary-keys keys lookup storage random

• 'Pig Butchering' Scams May Have Spurred Thailand-Cambodia War

  Via TJ McIntyre -- indications that the Thailand-Cambodia war is being driven by the "pig butchering" scammer compounds operating in the border area:

  Cambodia’s 2019 census put O’Smach’s population just over 9,850, but that doesn’t include the prison-like, office-dormitory compounds that have appeared here over the past five years, with the capacity to house 10,000 more. Around 50 sites like these now line the Cambodia-Thailand border, designed to house a slice of the trillion-dollar cybercrime industry—primarily teams running investment scams, dubbed “pig butchering” for the way they fatten their targets up; sextortion scams that blackmail victims, including children, by threatening to make sexual images public; scams that impersonate police to gain account access; and fraudulent online gambling sites. Once aimed largely at the Chinese public, these now target victims worldwide and rake in tens of billions of dollars a year in Cambodia alone.

  The compounds evolved from a casino industry that caters mostly to Chinese tourists and Thai day-trippers and has been linked to human trafficking, drug smuggling, and the endangered wildlife trade. From 2016, physical casinos were dwarfed by the online gambling industry (outlawed by Cambodia in 2019), which progressed to illegal sites and outright scams. Operators rent space in casinos and purpose-built compounds controlled by Chinese criminals, Myanmar warlords, and the Cambodian political elite.

  Scam companies rely heavily on forced and trafficked labor from Asia, Africa, and Latin America to chat with targets, pose as romantic interests and employees at fake investment platforms, and persuade them to make deposits. Survivors tell us that torture, rape, and beatings are common. As the fighting raged in July, some trafficking victims reached out for help, saying they were locked in their dorms by their bosses. Videos shot from inside these sites show missiles flying overhead, explosions thundering outside, some workers appearing to break out and run, and damage from shelling in the grounds.

  Tags: scams phishing pig-butchering war grim-meathook-future thailand cambodia scammers

• Year in Review 2025: Hari Kunzru on AI slop and censorship

  Hari Kunzru nails it:

  These days I have a sense of falling from a precipice toward a torrent of algorithmically driven slop. It’s coming, whether we want it or not, and the consequences for our communal life will be devastating.

  It’s now seven years since Steve Bannon outlined his infamous strategy to “flood the zone with shit.” This, he said, was a way to “deal with” the media, whom he saw as the real enemies of MAGA. In practice, it has been a very effective method of censorship. With every important issue of the day, the “zone” of public discourse is immediately filled with a volume of competing narratives, often mendacious or misleading. It’s no longer necessary to suppress information. You just have to make the cost of sorting fact from fiction, in terms of time and effort, too high to pay for the ordinary person, who can’t spend all day online weighing up competing claims about robots or pedophilia or Iran.

  Generative AI now allows the production of disinformation at scale. The kind of influence ops we associate with Cambridge Analytica or the Russian Internet Research Agency can be conducted with unprecedented scope and sophistication: Thousands of fake people — tens of thousands, perhaps hundreds of thousands — making videos, posting in forums, astroturfing entire contexts in which people will live out their political lives. Couple this with the collapse of trust in all kinds of authority, and there is no one even to say what might distinguish “disinformation” from any other kind of data. [...]

  The desire to return to consensus reality is hopelessly nostalgic. Yes, there are still hard limits: The “cloud” is a physical place, scooping out mountains for raw materials and venting heat and carbon dioxide out of gargantuan data centers; political power still grows out of the barrel of a gun. But the layer of the stack in which our subjectivities are formed, the place where our beliefs about the world are shaped, is also a battleground. We must teach ourselves to navigate the torrent that is replacing consensus reality, this turbulent, treacherous mediatized flow. There is no shore to swim back to, but in the new age of magic, when reality is labile and can be recoded by the power of signs, by narrative and memes and vibes and compelling images, art becomes a truly political technology. This is not art as critique. Critique is just sincere-posting, dutifully pointing out yet again that the Medbed isn’t “real.” Art can mess with our masters in ways we don’t yet fully understand. It makes culture. It is a transmitter of values. It is the lava out of which future realities will congeal.

  Tags: misinformation disinformation facts reality future ai slop hari-kunzru steve-bannon flooding-the-zone-with-shit art media propaganda

• Multiplying our way out of division — Matt Godbolt’s blog

  A very silly optimisation for the “binary to decimal” conversion problem:

  The compiler has turned division by a constant ten into a multiply and a shift. There’s a magic constant 0xcccccccd and a shift right of 35! Shifting right by 35 is the same as dividing by 235 - what’s going on? [..]

  What’s happening is that 0xcccccccd / 2**35 is very close to ? (around 0.10000000000582077). By multiplying our input value by this constant first, then shifting right, we’re doing fixed-point multiplication by ? - which is division by ten. The compiler knows that for all possible unsigned integer values, this trick will always give the right answer.

  Tags: hacks optimization bit-hacking binary decimal fixed-point arithmetric tricks

• Large Language Models As The Tales That Are Sung

  A thought-provoking read on LLMs, poetry, the oral tradition, and Gene Wolfe:

  "Even if LLMs are made out of poetry, they are incapable of producing poems. Or in Wolfe’s language, both the epic form and LLMs are story, but are incapable of telling stories. That requires the marriage of structure and intention that human mediation provides. LLMs are a kind of composite of the singing of tales, but are not singers, even if we sometimes misconstrue them as such."

  Tags: llms text poetry words language gene-wolfe ascians storytelling structure culture

• 'Unauthorized' Edit to Ukraine's Frontline Maps Point to Polymarket's War Betting

  A live map that tracks frontlines of the war in Ukraine was edited to show a fake Russian advance on the city of Myrnohrad on November 15. The edit coincided with the resolution of a bet on Polymarket, a site where users can bet on anything from basketball games to presidential election and ongoing conflicts. If Russia captured Myrnohrad by the middle of November, then some gamblers would make money. According to the map that Polymarket relies on, they secured the town just before 10:48 UTC on November 15. The bet resolved and then, mysteriously, the map was edited again and the Russian advance vanished.

  Tags: polymarket betting war future cyberpunk fraud ukraine russia

• WallBonito

  A recommended frame vendor from Poland, thanks to mags on ITC

  Tags: framing frames home decoration shopping pictures walls

• Building a Medallion architecture with ClickHouse

  Walkthrough of the "Medallion" architecture concept, which comprises three layers (or stages), each serving distinct purposes in the data pipeline:

  • Bronze layer - This layer acts as the landing area for raw, unprocessed data directly from the source system: simply put a "staging area". This data is stored in its original structure with minimal transformations and additional metadata. This layer is optimized for fast ingestion, and can provide an historical archive of source data that is always available for reprocessing or debugging. Whether the bronze layer should store all data is a point of contention, with some users preferring to filter the data and apply transformations, e.g., flattening JSON, renaming fields, or filtering out poorly formed data. We're not overly opinionated here but recommend optimizing the storage for consumption by the silver layer only - not other consumers.

  • Silver layer - Here, data is cleansed, deduplicated, and conformed to a unified schema, with raw data from the previous Bronze layer being enriched and transformed to provide a more accurate and consistent view. This data can be consistent and usable for enterprise-wide use cases such as machine learning and analytics. The data model should emerge at this layer with a focus placed on ensuring primary and foreign keys are consistent to simplify future joins. While not common, applications and downstream consumers can read from this layer. These are typically business-wide applications that need the entire cleansed dataset, e.g., ML workflows. Importantly, data quality will not improve after this stage only the ease at which it can be queried efficiently.

  • Gold layer - This later aims to have fully curated, business-ready, and project-specific datasets that make the data more accessible (and performant) to consumers. These datasets are often denormalized, or pre-aggregated, for optimal read performance and may have been composed of multiple tables from the previous silver stage. The focus here is on applying final transformations and ensuring the highest data quality for consumption by end-users or applications, such as reporting and user-facing dashboards.

  This layered approach to data pipelines aims to efficiently address challenges like data quality, duplication and schema inconsistencies. By transforming raw data incrementally, the Medallion architecture aims to ensure a clear lineage and progressively refined datasets that are ready for analysis or operational use.

  Tags: medallion-architecture data architecture pipelines clickhouse

• PocketBase

  "Open Source backend in 1 file". This is nice; it's a little OSS sqlite database, authentication, file storage and admin dashboard for web apps.

  Tags: golang opensource database pocketbase web-apps sqlite

• Questioning an Interface: From Parquet to Vortex

  Interesting -- a new, GPU-optimised storage format:

  Like Parquet, Vortex minimizes bytes on disk. However, Vortex is also designed with a core use-case in mind: decoding and querying data directly from object storage on GPUs. This key idea translates very well to our use-case even though we don’t run our queries on GPUs (yet?). Specifically, the file format is designed to maximize throughput and parallelism from the metadata format to the SIMD/SIMT friendly encodings used.

  Crucially, it also acknowledges that part of making queries fast is not only good filter pushdown, but also general-purpose compute pushdown. If anything cannot be pushed down, Vortex’s encodings can be tuned to offer zero-copy conversion to Arrow for further query execution using any general-purpose query execution engine.

  Vortex also learns from Parquet’s limitations around extensibility and aims to be as future-proof as possible. New encodings can ship with WASM decoders so encoding adoption is not limited by reader libraries having to implement support. The main Rust library is also designed to be fully extensible, so you can write your own layouts/encodings and plug them in as first-class citizens.

  Given how well Vortex’s design matched our needs, we tried it out and got a 70% average performance improvement on all our queries. With the newer encodings that Vortex offers, we got 10% better uncompressed storage size and only 3% larger compressed storage size compared to snappy-compressed Parquet.

  Tags: gpu vortex parquet compression storage file-formats files pushdown simd