Auditing AI Chatbots During the Galway West and Dublin Central Byelections
"In the weeks leading up to the byelections in Galway West and Dublin Central, we simulated citizen-AI interactions by asking [a] set of election-related test questions to four popular AI chatbots (Anthropic’s Claude, OpenAI’s ChatGPT, Google’s Gemini, and xAI’s Grok)":
We asked each chatbot a set of 194 questions covering a range of relevant topics on two separate occasions, 14 and 7 days before voters go to the polls on 22 May. Our aim was to assess:
- Do they provide citizens with accurate election information?
- Which sources do they rely on, and how does this vary across chatbots?
- Who do they platform when they answer questions, and who is left out?
.... The largest share of citations is directed towards mainstream news sites, and this is where the first evidence of source curation by chatbots can be found. While the Irish Times is a common source across all providers, ChatGPT and Gemini never refer to RTÉ. Meanwhile, despite not ranking among the top three news sources for the other chatbots, Gript is the number one news source cited by Gemini.
We find that xAI’s Grok is the most likely to use social media in responses, while Gemini most frequently refers to YouTube and content from Reddit. ChatGPT appeared less likely to rely on social media compared to the other chatbots.
IMO, this points to an undesirable side effect of paywalls in the news media. While it's vitally important for media companies to protect their means of income, an unwelcome side effect is that the introduction of paywalls has resulted in AI chatbots sidelining mainstream news media sources which they cannot access reliably, in favour of what is effectively disinformation from less trustworthy sites like Gript.
Tags: grok ai llms xai web news media ireland irish-times rte chatgpt gemini reddit youtube elections politics