-
This is a nice term from Katryna Peart to describe one of the corrupting factors LLMs introduce to documents: "narrative flattening":
"When US cities began deploying AI to process civic documents — meeting minutes, public histories, commemorative records, policy documents — the failures that emerged weren’t the dramatic kind. The AI didn’t invent facts wholesale [...] it smoothed."
AI-driven "summarisation" results in radically modified narrative meanings:
Contested decisions became consensus. Dissenting voices disappeared into summaries that read as agreement. And the output read as authoritative because it was produced from an authoritative source.
In testing AI systems against a municipal commemorative report from Newark, New Jersey, I asked each system how the initiative compared to similar programmes in the region — using only the document provided.
ChatGPT invented a comparative framework, asserting the programme stood out from “traditional anniversary programmes” and “typical county-level commemorations.” Neither category exists in the document. The system didn’t hallucinate a date or misattribute a quote. It constructed an entire analytical frame from nothing and presented it as document-based retrieval.
In the same test, the document contained outreach tactics — radio, direct mail, community networks — that worked because they aligned with how civic organising functions in Newark’s Black, ageing community. An AI system extracting those tactics would present them as applicable elsewhere, while stripping away the demographic and political context that made them effective in that specific city. [...]
To test whether the same failure modes appear in UK civic documents, I applied the same protocol to the Somerset Council Plan 2023–2027 — the post-reorganisation vision document produced when five predecessor councils merged into a single unitary authority in April 2023. I tested four AI systems: Gemini, ChatGPT, Microsoft Copilot, and Claude.
Every model hardened aspirational language into apparent commitments. The plan states the council would “demonstrate leadership around the whole range of housing issues” and “strive to develop an inclusive culture.” No targets, no timelines, no delivery mechanisms. Every model converted those statements into bullet-pointed commitment lists. A council officer reviewing those outputs would have no way of knowing the specificity came from the model, not the plan.
Copilot — the model most UK councils are currently deploying through existing Microsoft 365 contracts, often without separate AI governance review — described Somerset as committed to “co-design” with communities and “fair access” to education, housing, jobs, and services. Neither phrase appears in the document. Copilot synthesised fragments from three separate passages into a single clean commitment the council never made. On the comparison question, it added that Somerset’s emphasis on rural inequality “is less prominent in many urban unitary authorities” — a comparative claim with no basis in the source. It did not flag any of this as inference.
ChatGPT fabricated a comparative framework and constructed a tension narrative. Asked how Somerset’s approach compared to other UK unitary authorities, it responded that “unlike more fragmented models, Somerset frames inequality as interconnected” with multiple service areas. No other authority is described anywhere in the document. It also described the “main tensions” in the reorganisation process — a framing the document never uses. The word tensions does not appear in the Somerset Council Plan.
[...]
Across original research testing three civic documents against three major AI systems, failure modes were structurally predictable based on document type:
- Celebratory documents get reproduced uncritically — institutional PR becomes authoritative historical record, success metrics are cited without methodological context, and dissenting voices go unmarked.
- Accountability documents get softened — AI systems introduce balance and healing language that the original document explicitly rejects, restoring a both-sides framing the institution deliberately refused.
- Pre-event planning documents get filled in — aspirational inclusion language invites AI to supply the racial history, equity frameworks, and comparative data the institution implied but never produced.
In each case the output reads as grounded in the source. In each case something the document actually said — or deliberately did not say — has been quietly rewritten.
Standard AI procurement frameworks test for hallucination, data security, and cost. They do not test for narrative flattening — because it doesn’t look like an error. It looks like a summary.
(via gwire)
Tags: narrative-flattening via:gwire summarisation summarization ai llms corruption katryna-peart hallucination confabulation errors uk documents