Fact Checked by 
A new research study carried out by the BBC suggests “four major artificial intelligence (AI) chatbots are inaccurately summarising news stories.”
On Tuesday (February 11) the publisher announced the results of its study which included giving ChatGPT, Copilot, Gemini and Perplexity AI content from its website before asking questions about the content.
“It said the resulting answers contained “significant inaccuracies” and distortions,” the BBC explained.
It was fascinating to be part of this research into AI and news content at the BBC.
Top line findings include: 51% of all AI answers to questions about the news were judged to have significant issues of some formhttps://t.co/qkD1GwJ203
— George Bowden (@georgebowden) February 11, 2025
Each tool was asked to summarize 100 news stories, with the journalists who were the experts in the subject of the article then rating the quality of answers from the AI assistants.
“It found 51% of all AI answers to questions about the news were judged to have significant issues of some form.
“Additionally, 19% of AI answers which cited BBC content introduced factual errors, such as incorrect factual statements, numbers and dates.”
Some of the reported inaccuracies included OpenAI’s ChatGPT and Microsoft’s Copilot saying Rishi Sunak and Nicola Sturgeon were still in office even after they had left. Rishi Sunak was the Prime Minister of the United Kingdom between 2022 to 2024, while Nicola Sturgeon was the First Minister of Scotland from 2014 to 2023.
“In general, Microsoft’s Copilot and Google’s Gemini had more significant issues than OpenAI’s ChatGPT and Perplexity, which counts Jeff Bezos as one of its investors,” the article said.
CEO of BBC News expands on study looking into AI chatbots
The CEO of BBC News and Current Affairs, Deborah Turness, published a blog on the topic on the same day as the results. Within it, she expanded on the results: “The team found ‘significant issues’ with just over half of the answers generated by the assistants.
“The AI assistants introduced clear factual errors into around a fifth of answers they said had come from BBC material.”
When diving into the issues, Turness suggested part of the problem appears to be that “AI assistants do not discern between facts and opinion in news coverage; do not make a distinction between current and archive material; and tend to inject opinions into their answers.”
While the publisher typically blocks its content from AI chatbots, it opened it up while the testing for the study was taking place. This occurred during December, 2024.
An OpenAI spokesperson told BBC News in the article: “We’ve collaborated with partners to improve in-line citation accuracy and respect publisher preferences, including enabling how they appear in search by managing OAI-SearchBot in their robots.txt. We’ll keep enhancing search results.”
Featured Image: AI-generated via Ideogram
