How often AI search engines cite the wrong source: what two audits measured

Two audits give the clearest numbers on citation accuracy in AI search. The Tow Center asked chatbots to "identify the corresponding article’s headline, original publisher, publication date, and URL" for an excerpt.¹ Liu et al. start from the principle that "each provided citation should support its associated statement".² The tasks differ, and so do the systems tested. The findings point the same way.

The Tow Center test: find the source of this excerpt

The Tow Center tested "OpenAI’s ChatGPT Search, Perplexity, Perplexity Pro, DeepSeek Search, Microsoft’s Copilot, xAI’s Grok-2 and Grok-3 (beta), and Google’s Gemini".¹ It chose 20 news publishers, some that permit search crawlers in robots.txt and some that block them.¹ Some had licensing or revenue-share deals with AI companies, while others were suing them.¹

From each publisher the team randomly selected ten articles, then picked direct excerpts by hand.¹ Each chatbot was asked to identify the article's headline, original publisher, publication date and URL.¹ The excerpts were chosen so that a traditional Google search returned the original source within the first three results.¹ The study states the total as twenty publishers times ten articles times eight chatbots.¹

Responses were graded on three attributes: the correct article, the correct publisher and the correct URL.¹ Labels ran from Correct through Partially Incorrect and Completely Incorrect to Not Provided and Crawler Blocked.¹ The tests were conducted in February 2025.¹

What the Tow Center found

Collectively, the chatbots gave incorrect answers to more than 60 percent of queries.¹ Perplexity answered 37 percent incorrectly, while Grok 3 answered 94 percent incorrectly.¹

Confidence was a separate problem. ChatGPT incorrectly identified 134 articles, but signaled a lack of confidence just fifteen times out of its two hundred responses, and never declined to answer.¹ Copilot was the exception: it declined more questions than it answered, and every other tool was more likely to give an incorrect answer than to acknowledge limits.¹

Wrong attribution was common. DeepSeek misattributed the source of the excerpts 115 out of 200 times.¹ Even when a chatbot identified the right article, it often failed to link to the original source.¹

Several failure types matter for publishers:

• Syndicated copies. Chatbots sometimes sent users to syndicated versions on platforms like Yahoo News or AOL instead of the original, even when the publisher had a licensing deal with the AI company.¹ Perplexity Pro cited syndicated versions of Texas Tribune articles for three of ten queries, despite a partnership.¹
• Fabricated or broken URLs. More than half of responses from Gemini and Grok 3 cited fabricated or broken URLs that led to error pages.¹ For Grok 3, 154 of 200 citations led to error pages.¹
• Blocked crawlers. Perplexity's free version correctly identified all ten excerpts from paywalled National Geographic articles, even though the publisher has disallowed Perplexity's crawlers.¹ USA Today blocks ChatGPT's crawler, yet the chatbot cited a version of its article that was republished by Yahoo News.¹
• Licensing deals. The San Francisco Chronicle permits OpenAI's search crawler and is part of Hearst's content partnership with the company.¹ Yet "ChatGPT only correctly identified one of the ten excerpts we shared from the publisher".¹ Time, which has deals with both OpenAI and Perplexity, was among the most accurately identified publishers, though no model identified its content correctly 100 percent of the time.¹

Paying more did not reduce errors. Perplexity Pro and Grok 3 answered more prompts correctly than their free equivalents but also showed higher error rates, mainly because they gave definitive wrong answers rather than declining.¹

The Stanford audit: does the citation support the sentence?

Liu et al. measured something narrower and more general. They audited four commercial generative search engines (Bing Chat, NeevaAI, perplexity.ai and YouChat) using human evaluation.² Responses were scraped between late February and late March 2023.²

Each system was evaluated on 1450 queries drawn from varied sources, including Google user queries and open-ended Reddit questions.² In total, 34 annotators took part.² The paper defines two metrics. "Citation recall is the proportion of verification-worthy statements that are fully supported by their associated citations".² Citation precision is the proportion of generated citations that support their associated statements.²

On average, a mere 51.5% of generated sentences were fully supported by citations, and only 74.5% of citations supported their associated sentence.² The authors call these results "concerningly low."²

Systems varied widely. Bing Chat had the highest average precision (89.5), while YouChat had the lowest (63.6).² YouChat's average recall was 11.1, against 68.7 for perplexity.ai.² Recall also moved with the query type: the lowest average recall (44.3) came on open-ended AllSouls essay questions, which generally have no extractive answer online.²

Responses were nonetheless rated fluent and helpful, with average ratings of 4.48 for fluency and 4.50 for perceived utility.² The authors "find that citation precision is inversely correlated with perceived utility in existing generative search engines (r=−0.96)".² The authors hypothesize this happens because systems that copy or closely paraphrase their cited pages score high on precision but may answer the query less helpfully.²

How the two studies compare

The Tow Center asked whether a chatbot can name the right publisher and URL for a known excerpt. Liu et al. asked whether a cited page supports the sentence it sits beside. The first is an attribution test on news. The second is a support test on general queries.

Both found low accuracy, and both found that a confident, fluent presentation hides it. The Tow Center's chatbots rarely hedged. Liu et al. found responses that appear informative but contain unsupported statements. The Stanford data is from early 2023 and the Tow Center data from February 2025, on largely different systems, so the numbers should not be compared directly.

What this means for publishers

The findings suggest a few practical checks.

• Test your own content. The Tow Center method is easy to copy: take a distinctive excerpt from an article and ask each chatbot for the headline, publisher, date and URL. Record all three attributes separately, as the study did.
• Check where citations point. Look for syndicated copies and republished versions being cited in place of your original. The Tow Center saw this even for publishers with licensing deals.
• Click the links. Fabricated and broken URLs were common for some tools, so a citation that names you may still lead nowhere.
• Do not assume robots.txt is a full control. Syndicated copies can keep content visible even when a crawler is blocked.
• Do not assume a deal fixes attribution. Time's Howard said the AI companies did not commit to being 100 percent accurate.¹

An earlier Tow Center test of ChatGPT Search alone found that "ChatGPT returned partially or entirely incorrect responses on a hundred and fifty-three occasions, though it only acknowledged an inability to accurately respond to a query seven times."³ The study adds: "Even publishers that permit OpenAI’s search crawlers are not always cited correctly."³

These studies carry limits. The Tow Center says its findings are not intended to be extrapolated to all models or news organizations.¹ It also notes that its findings represent just one occurrence of each excerpt being queried.¹ Liu et al. add that "verifiability is not factuality".² Treat the figures as a baseline for your own spot checks, not as a fixed error rate.