{"name":"AEO and AI search statistics, with sources","description":"Generative engine optimization (GEO), AEO and AI search statistics from public studies, each with its sample, method, sponsor and a link to the primary source. Free CSV and JSON under CC BY 4.0.","url":"https://aeofootnotes.com/lab/aeo-statistics","publisher":"AEO Footnotes","license":"https://creativecommons.org/licenses/by/4.0/","updated":"2026-10-30","count":37,"topics":{"clicks":"Clicks and CTR","citations":"Citations and accuracy","crawlers":"Crawlers and llms.txt","geo":"GEO experiments","adoption":"Adoption"},"studies":[{"footnote":1,"id":"pew-ai-summary-clicks","topic":"clicks","claim":"Google users who saw an AI summary clicked a traditional search result on 8% of visits, against 15% of visits without one, and clicked a link inside the summary on 1% of visits.","metric":"Share of Google search page visits followed by a click on a result link","sample":{"n":900,"unit":"U.S. adults (68,879 Google searches)"},"method":"Metered browsing panel (KnowledgePanel Digital): every Google search on tracked devices and what the user did next","period":"March 1 to 31, 2025","published":"2025-07-22","publisher":"Pew Research Center","sponsor":"None stated","conflict":false,"engines":["Google AI Overviews"],"title":"Google users are less likely to click on links when an AI summary appears in the results","url":"https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/","caveat":"Google and U.S. adults only, over one month; longer and question-style searches trigger summaries more often, so query type and summary presence are mixed.","topicLabel":"Clicks and CTR"},{"footnote":2,"id":"pew-ai-summary-share","topic":"adoption","claim":"18% of the Google searches in Pew's panel produced an AI summary, and 88% of those summaries cited three or more sources.","metric":"Share of Google searches with an AI summary","sample":{"n":68879,"unit":"Google searches by 900 U.S. adults"},"method":"Metered browsing panel (KnowledgePanel Digital), all Google searches in the month","period":"March 1 to 31, 2025","published":"2025-07-22","publisher":"Pew Research Center","sponsor":"None stated","conflict":false,"engines":["Google AI Overviews"],"title":"Google users are less likely to click on links when an AI summary appears in the results","url":"https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/","caveat":"A one-month snapshot from one panel; Google has changed how often it shows AI summaries since.","topicLabel":"Adoption"},{"footnote":3,"id":"ahrefs-aio-ctr-2025","topic":"clicks","claim":"The presence of an AI Overview correlated with a 34.5% lower average clickthrough rate for the top-ranking page, comparing March 2024 with March 2025.","metric":"Position 1 desktop CTR, actual against forecast","sample":{"n":300000,"unit":"keywords (150,000 with an AI Overview, 150,000 informational without)"},"method":"Before and after CTR from aggregated Google Search Console data, with informational keywords as the baseline","period":"March 2024 and March 2025","published":"2025-04-17","publisher":"Ahrefs","sponsor":"Ahrefs (sells SEO tools and Brand Radar for AI visibility)","conflict":true,"engines":["Google AI Overviews"],"title":"AI Overviews Reduce Clicks by 34.5%","url":"https://ahrefs.com/blog/ai-overviews-reduce-clicks/","caveat":"Desktop CTR only, from Search Console data Ahrefs can see; the effect is a forecast-based correlation, not a controlled test.","topicLabel":"Clicks and CTR"},{"footnote":4,"id":"ahrefs-aio-ctr-2026","topic":"clicks","claim":"Rerun with December data, the presence of an AI Overview correlated with a 58% lower average clickthrough rate for the top-ranking page.","metric":"Position 1 desktop CTR, actual against forecast","sample":{"n":300000,"unit":"keywords (150,000 with an AI Overview, 150,000 informational without)"},"method":"Same method as the April 2025 study: aggregated Google Search Console CTR, before and after","period":"December 2023 and December 2025","published":"2026-02-04","publisher":"Ahrefs","sponsor":"Ahrefs (sells SEO tools and Brand Radar for AI visibility)","conflict":true,"engines":["Google AI Overviews"],"title":"Update: AI Overviews Reduce Clicks by 58%","url":"https://ahrefs.com/blog/ai-overviews-reduce-clicks-update/","caveat":"CTR for the baseline keywords without AI Overviews also fell (0.076 to 0.039), so the forecast absorbs other changes over two years.","topicLabel":"Clicks and CTR"},{"footnote":5,"id":"seer-aio-ctr","topic":"clicks","claim":"Organic CTR on informational queries that showed an AI Overview fell from 1.76% in June 2024 to 0.61% in September 2025, a 61% decline.","metric":"Monthly average organic CTR","sample":{"n":3119,"unit":"search terms across 42 client organizations"},"method":"Client organic and paid search data (25.1M organic and 1.1M paid impressions), split by AI Overview presence and citation","period":"June 2024 to September 2025","published":"2025-11-04","publisher":"Seer Interactive","sponsor":"Seer Interactive (search agency that sells GEO services)","conflict":true,"engines":["Google AI Overviews"],"title":"AIO Impact on Google CTR: September 2025 Update","url":"https://www.seerinteractive.com/insights/aio-impact-on-google-ctr-september-2025-update","caveat":"Informational queries from Seer's clients only; organic CTR on queries without an AI Overview also fell 41% over the same period.","topicLabel":"Clicks and CTR"},{"footnote":6,"id":"seer-aio-cited","topic":"citations","claim":"When a brand was cited in an AI Overview it earned 35% more organic clicks and 91% more paid clicks than when it was not cited.","metric":"Organic and paid CTR, cited against not cited","sample":{"n":3119,"unit":"search terms across 42 client organizations"},"method":"Client search data split by whether the brand was cited or mentioned in the AI Overview","period":"June 2024 to September 2025","published":"2025-11-04","publisher":"Seer Interactive","sponsor":"Seer Interactive (search agency that sells GEO services)","conflict":true,"engines":["Google AI Overviews"],"title":"AIO Impact on Google CTR: September 2025 Update","url":"https://www.seerinteractive.com/insights/aio-impact-on-google-ctr-september-2025-update","caveat":"Seer states it cannot show that citation causes the higher CTR; brands with stronger baseline CTR may simply be cited more.","topicLabel":"Citations and accuracy"},{"footnote":7,"id":"semrush-zero-click","topic":"clicks","claim":"For the same keywords measured before and after they began showing an AI Overview, the zero-click rate fell from 33.75% to 31.53%.","metric":"Zero-click rate per keyword","sample":{"n":200000,"unit":"keywords with search volume over 100 (200K+)"},"method":"Clickstream panel from Datos (a Semrush company), keywords that gained an AI Overview between May and October","period":"January to October 2025","published":"2025-12-15","publisher":"Semrush","sponsor":"Semrush (sells SEO and AI visibility tools); data from Datos, a Semrush company","conflict":true,"engines":["Google AI Overviews"],"title":"Semrush AI Overviews Study: What 2025 SEO Data Tells Us About Google's Search Shift","url":"https://www.semrush.com/blog/semrush-ai-overviews-study/","caveat":"A zero-click rate is not a clickthrough rate to any one page; the page does not give the size of the before and after keyword set.","topicLabel":"Clicks and CTR"},{"footnote":8,"id":"semrush-aio-share","topic":"adoption","claim":"AI Overviews appeared for 6.49% of keywords in January 2025, 24.61% in July and 15.69% in November.","metric":"Share of tracked keywords that trigger an AI Overview","sample":{"n":10000000,"unit":"keywords (10M+)"},"method":"SERP tracking across the Semrush keyword database, month by month","period":"January to November 2025","published":"2025-12-15","publisher":"Semrush","sponsor":"Semrush (sells SEO and AI visibility tools)","conflict":true,"engines":["Google AI Overviews"],"title":"Semrush AI Overviews Study: What 2025 SEO Data Tells Us About Google's Search Shift","url":"https://www.semrush.com/blog/semrush-ai-overviews-study/","caveat":"A share of keywords in one vendor's database, not a share of the searches people actually run.","topicLabel":"Adoption"},{"footnote":9,"id":"ahrefs-ai-traffic","topic":"clicks","claim":"AI chatbots sent 0.17% of the average site's visitors, and 63% of sites received at least one visit from an AI chatbot.","metric":"Share of monthly visitors referred by AI chatbots","sample":{"n":3000,"unit":"websites"},"method":"Referral data from an anonymized sample of sites running Ahrefs Web Analytics, seven chatbots","period":"Not stated on the page","published":"2025-02-06","publisher":"Ahrefs","sponsor":"Ahrefs (sells SEO tools and Brand Radar for AI visibility)","conflict":true,"engines":["ChatGPT","Perplexity","Gemini","Claude","Copilot","Mistral","Jasper"],"title":"63% of Websites Receive AI Traffic (New Study of 3,000 Sites)","url":"https://ahrefs.com/blog/ai-traffic-study/","caveat":"Sites that use Ahrefs Web Analytics, not a random sample; visits that arrive without an AI referrer are not counted as AI traffic.","topicLabel":"Clicks and CTR"},{"footnote":10,"id":"sparktoro-zero-click-2024","topic":"clicks","claim":"For every 1,000 Google searches in the U.S., 360 clicks went to the open web, and just under 60% of searches ended without a click.","metric":"Clicks to non-Google properties per 1,000 searches","sample":{"n":null,"unit":"devices in a multi-million device clickstream panel"},"method":"Clickstream panel (Datos), desktop and mobile searches weighted by device share","period":"January to May 2024 (mobile data from January 2024)","published":"2024-07-01","publisher":"SparkToro","sponsor":"Data from Datos, a Semrush company","conflict":false,"engines":["Google Search"],"title":"2024 Zero-Click Search Study: For every 1,000 EU Google Searches, only 374 clicks go to the Open Web. In the US, it's 360.","url":"https://sparktoro.com/blog/2024-zero-click-search-study-for-every-1000-us-google-searches-only-374-clicks-go-to-the-open-web-in-the-eu-its-360/","caveat":"Collected before AI Overviews reached most U.S. searches; a baseline for zero-click behavior, not a measurement of AI answers.","topicLabel":"Clicks and CTR"},{"footnote":11,"id":"adobe-genai-retail-traffic","topic":"adoption","claim":"Traffic to U.S. retail sites from generative AI sources rose 1,200% in February 2025 compared with July 2024.","metric":"Growth in visits referred by generative AI sources","sample":{"n":1000000000000,"unit":"visits to U.S. retail sites (more than 1 trillion)"},"method":"Adobe Analytics visit and transaction data, plus a survey of 5,000 U.S. consumers","period":"July 2024 to February 2025","published":"2025-03-17","publisher":"Adobe","sponsor":"Adobe (sells Adobe Analytics and Adobe Brand Visibility, formerly LLM Optimizer)","conflict":true,"engines":["Generative AI sources (not broken out)"],"title":"Adobe Analytics: Traffic to U.S. retail websites from Generative AI sources jumps 1,200 percent","url":"https://blog.adobe.com/en/publish/2025/03/17/adobe-analytics-traffic-to-us-retail-websites-from-generative-ai-sources-jumps-1200-percent","caveat":"Growth from a small base; Adobe says generative AI traffic remains modest next to channels such as paid search or email.","topicLabel":"Adoption"},{"footnote":12,"id":"pew-chatgpt-use","topic":"adoption","claim":"34% of U.S. adults say they have ever used ChatGPT, roughly double the share in 2023, including 58% of adults under 30.","metric":"Share of U.S. adults who have ever used ChatGPT","sample":{"n":5123,"unit":"U.S. adults"},"method":"Survey of Pew's American Trends Panel, recruited by random address sampling and weighted","period":"February 24 to March 2, 2025","published":"2025-06-25","publisher":"Pew Research Center","sponsor":"None stated","conflict":false,"engines":["ChatGPT"],"title":"34% of U.S. adults have used ChatGPT, about double the share in 2023","url":"https://www.pewresearch.org/short-reads/2025/06/25/34-of-us-adults-have-used-chatgpt-about-double-the-share-in-2023/","caveat":"Self-reported ever-use; it does not measure how often people use ChatGPT or whether they use it to search.","topicLabel":"Adoption"},{"footnote":13,"id":"tow-ai-search-citations","topic":"citations","claim":"Eight AI search tools gave incorrect answers to more than 60% of 1,600 queries asking them to identify a news article's headline, publisher, date and URL.","metric":"Share of queries answered incorrectly","sample":{"n":1600,"unit":"queries (20 publishers, 10 articles each, 8 tools)"},"method":"Prompt runs with direct article excerpts, each answer graded by hand for article, publisher and URL","period":"February 2025","published":"2025-03-06","publisher":"Tow Center for Digital Journalism, Columbia Journalism Review","sponsor":"None stated","conflict":false,"engines":["ChatGPT Search","Perplexity","Perplexity Pro","DeepSeek Search","Copilot","Grok 2","Grok 3","Gemini"],"title":"We compared eight AI search engines. They're all bad at citing news.","url":"https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php","caveat":"One narrow task (finding the source of an excerpt); it does not measure answer quality for ordinary questions.","topicLabel":"Citations and accuracy"},{"footnote":14,"id":"tow-chatgpt-search-publishers","topic":"citations","claim":"ChatGPT search returned partially or entirely incorrect sources for 153 of 200 publisher quotes and acknowledged it could not answer only 7 times.","metric":"Responses with incorrect source attribution","sample":{"n":200,"unit":"quotes from 20 publications"},"method":"Prompt runs: block quotes from 10 articles per publisher, answers graded for publisher, date and URL","period":"November 2024","published":"2024-11-27","publisher":"Tow Center for Digital Journalism, Columbia Journalism Review","sponsor":"None stated","conflict":false,"engines":["ChatGPT Search"],"title":"How ChatGPT Search (Mis)represents Publisher Content","url":"https://www.cjr.org/tow_center/how-chatgpt-misrepresents-publisher-content.php","caveat":"An early test of a new product; OpenAI called it an atypical test, and 40 of the quotes came from publishers that blocked its crawler.","topicLabel":"Citations and accuracy"},{"footnote":15,"id":"liu-verifiability","topic":"citations","claim":"Across four generative search engines, 51.5% of generated sentences were fully supported by their citations and 74.5% of citations supported their sentence.","metric":"Citation recall and citation precision","sample":{"n":1450,"unit":"queries"},"method":"Human evaluation of each sentence and citation in the engines' answers","period":"2023","published":"2023-04-19","publisher":"Liu, Zhang and Liang, Stanford University (arXiv 2304.09848)","sponsor":"None stated","conflict":false,"engines":["Bing Chat","NeevaAI","perplexity.ai","YouChat"],"title":"Evaluating Verifiability in Generative Search Engines","url":"https://arxiv.org/abs/2304.09848","caveat":"2023 systems, some since discontinued; today's engines were not measured.","topicLabel":"Citations and accuracy"},{"footnote":16,"id":"bbc-ai-assistants-news","topic":"citations","claim":"51% of AI assistant answers to questions about the news were judged to have significant issues, and 19% of answers citing BBC content introduced factual errors.","metric":"Share of answers with significant issues, as rated by journalists","sample":{"n":362,"unit":"AI responses reviewed by 45 BBC journalists"},"method":"Four assistants given access to the BBC website answered news questions; expert journalists rated each answer","period":"December 2024","published":"2025-02-11","publisher":"BBC","sponsor":"None stated","conflict":false,"engines":["ChatGPT","Copilot","Gemini","Perplexity"],"title":"Representation of BBC News content in AI Assistants","url":"https://www.bbc.co.uk/aboutthebbc/documents/bbc-research-into-ai-assistants.pdf","caveat":"One publisher's content, rated by its own journalists; news questions only.","topicLabel":"Citations and accuracy"},{"footnote":17,"id":"ebu-news-integrity","topic":"citations","claim":"45% of AI assistant answers to news questions had at least one significant issue, and 31% showed serious sourcing problems.","metric":"Share of answers with at least one significant issue","sample":{"n":3000,"unit":"responses (more than 3,000)"},"method":"Journalists from 22 public service media organizations in 18 countries and 14 languages graded answers","period":"2025","published":"2025-10-22","publisher":"European Broadcasting Union and BBC","sponsor":"None stated","conflict":false,"engines":["ChatGPT","Copilot","Gemini","Perplexity"],"title":"Largest study of its kind shows AI assistants misrepresent news content 45% of the time","url":"https://www.bbc.co.uk/mediacentre/2025/new-ebu-research-ai-assistants-news-content","caveat":"News questions only, graded by journalists at the broadcasters whose content was being tested.","topicLabel":"Citations and accuracy"},{"footnote":18,"id":"ahrefs-aio-citations-top10","topic":"citations","claim":"76.10% of pages cited in AI Overviews ranked in Google's top 10 for the query, and 14.40% did not rank in the top 100.","metric":"Organic rank of AI Overview cited pages","sample":{"n":1900000,"unit":"citations from 1M AI Overviews"},"method":"Rank lookup for the three most visible citations in each AI Overview","period":"Not stated on the page","published":"2025-07-21","publisher":"Ahrefs","sponsor":"Ahrefs (sells SEO tools and Brand Radar for AI visibility)","conflict":true,"engines":["Google AI Overviews"],"title":"76% of AI Overview Citations Pull From the Top 10","url":"https://ahrefs.com/blog/search-rankings-ai-citations/","caveat":"Only the top three citations per answer, ranked for the tracked keyword rather than the fan-out queries behind the answer.","topicLabel":"Citations and accuracy"},{"footnote":19,"id":"ahrefs-brand-visibility","topic":"citations","claim":"YouTube mentions showed the strongest correlation with brand mentions in ChatGPT, AI Mode and AI Overviews (about 0.737), ahead of branded web mentions (0.66 to 0.71).","metric":"Spearman correlation with AI brand mentions","sample":{"n":75000,"unit":"brands"},"method":"Spearman correlation between brand signals in Ahrefs' indexes and mention counts in AI answers","period":"Not stated on the page","published":"2025-12-12","publisher":"Ahrefs","sponsor":"Ahrefs (sells SEO tools and Brand Radar for AI visibility)","conflict":true,"engines":["ChatGPT","Google AI Mode","Google AI Overviews"],"title":"Top Brand Visibility Factors in ChatGPT, AI Mode, and AI Overviews (75k Brands Studied)","url":"https://ahrefs.com/blog/ai-brand-visibility-correlations/","caveat":"Correlation, not causation, as the study says; both the signals and the AI mentions come from Ahrefs' own data.","topicLabel":"Citations and accuracy"},{"footnote":20,"id":"profound-citation-share","topic":"citations","claim":"In Profound's tracked data, Wikipedia was 7.8% of ChatGPT citations, and Reddit was 6.6% of Perplexity citations and 2.2% of Google AI Overviews citations.","metric":"Share of total citations by source domain","sample":{"n":680000000,"unit":"citations"},"method":"Citation counts from the prompts Profound tracks across three platforms","period":"August 2024 to June 2025","published":"2025-06-05","publisher":"Profound","sponsor":"Profound (sells an AI visibility platform)","conflict":true,"engines":["ChatGPT","Google AI Overviews","Perplexity"],"title":"AI Platform Citation Patterns: How ChatGPT, Google AI Overviews, and Perplexity Source Information","url":"https://www.tryprofound.com/blog/ai-platform-citation-patterns","caveat":"Shares of a vendor's tracked prompt set, which is not published; not a sample of what users ask.","topicLabel":"Citations and accuracy"},{"footnote":21,"id":"toronto-earned-media","topic":"citations","claim":"For ranking questions about well-known brands, 93.5% of the sources ChatGPT cited were earned media, against 67.4% for Perplexity and 63.4% for Gemini.","metric":"Share of cited domains classed as earned, brand or social","sample":{"n":null,"unit":"ranking-style brand queries in two sets (size not stated for this experiment)"},"method":"Prompt runs through each engine's API; cited domains classified as brand, earned or social","period":"2025","published":"2025-09-10","publisher":"Chen, Wang, Chen and Koudas, University of Toronto (arXiv 2509.08919)","sponsor":"None stated","conflict":false,"engines":["ChatGPT","Claude","Gemini","Perplexity"],"title":"Generative Engine Optimization: How to Dominate AI Search","url":"https://arxiv.org/abs/2509.08919","caveat":"Domain labels rely partly on GPT-assisted classification, and API answers can differ from the consumer apps.","topicLabel":"Citations and accuracy"},{"footnote":22,"id":"yang-news-citations","topic":"citations","claim":"9% of the more than 366,000 citations in AI Search Arena responses referenced news sources, concentrated among a small number of outlets.","metric":"Share of citations pointing to news sources","sample":{"n":366000,"unit":"citations (over 366,000) in 65,000 responses"},"method":"Analysis of conversations from the AI Search Arena, a head-to-head evaluation platform","period":"2025","published":"2025-07-07","publisher":"Kai-Cheng Yang, Northeastern University (arXiv 2507.05301)","sponsor":"None stated","conflict":false,"engines":["OpenAI models","Perplexity models","Google models"],"title":"News Source Citing Patterns in AI Search Systems","url":"https://arxiv.org/abs/2507.05301","caveat":"Arena users and their prompts are not a representative sample of people searching.","topicLabel":"Citations and accuracy"},{"footnote":23,"id":"graphite-ai-content-citations","topic":"citations","claim":"82% of articles cited by ChatGPT and Perplexity were human-written and 18% AI-generated, as classified by an AI detector.","metric":"Share of cited articles classed as AI-generated","sample":{"n":31493,"unit":"keywords across 10 categories (100 per category asked of answer engines)"},"method":"AI detector (Surfer) run on 500-word chunks of ranking and cited articles","period":"June 2025","published":"2025-10-14","publisher":"Graphite","sponsor":"Graphite (growth agency that sells SEO and AEO work)","conflict":true,"engines":["ChatGPT","Perplexity","Google Search"],"title":"How Does AI-Generated Content Perform in Search and Answer Engines?","url":"https://graphite.io/five-percent/ai-content-in-search-and-llms","caveat":"Detector-based, with a 4.2% false positive rate Graphite reports; AI drafts edited by people were not evaluated.","topicLabel":"Citations and accuracy"},{"footnote":24,"id":"muckrack-earned-media","topic":"citations","claim":"About 84% of links cited by ChatGPT, Claude and Gemini came from earned media, and about 27% were journalistic.","metric":"Share of cited links by source type","sample":{"n":25000000,"unit":"links from AI responses (25,000,000+)"},"method":"Prompt runs across a large consumer prompt set; every cited URL categorized","period":"2026","published":"2026-05","publisher":"Muck Rack","sponsor":"Muck Rack (sells PR software and Generative Pulse for AI visibility)","conflict":true,"engines":["ChatGPT","Claude","Gemini"],"title":"What Is AI Reading? May 2026","url":"https://media.muckrack.com/documents/What_Is_AI_Reading__May_2026.pdf","caveat":"Earned media here includes Wikipedia, social platforms and third-party corporate content; the prompt set is not published.","topicLabel":"Citations and accuracy"},{"footnote":25,"id":"aggarwal-geo","topic":"geo","claim":"Content edits such as citing sources, adding quotations and adding statistics raised source visibility in generative engine answers by up to 40% on the GEO-bench benchmark.","metric":"Relative change in position-adjusted word count and subjective impression","sample":{"n":10000,"unit":"queries (GEO-bench)"},"method":"Controlled experiment: edits applied to one source, visibility measured in the engine's answer against an unedited baseline","period":"2023 to 2024","published":"2023-11-16","publisher":"Aggarwal et al., KDD 2024 (arXiv 2311.09735)","sponsor":"None stated","conflict":false,"engines":["Research generative engine","Perplexity.ai"],"title":"GEO: Generative Engine Optimization","url":"https://arxiv.org/abs/2311.09735","caveat":"Relative gains in a lab engine and on Perplexity (up to 37% there); the paper did not test effects on search rankings.","topicLabel":"GEO experiments"},{"footnote":26,"id":"cseo-bench","topic":"geo","claim":"Of 54 cases tested, only 3 showed statistically significant ranking gains from existing conversational SEO methods for a single adopter.","metric":"Statistically significant rank improvements (Wilcoxon signed-rank test)","sample":{"n":54,"unit":"method, task and domain cases"},"method":"Benchmark: C-SEO methods applied to documents, rank change measured in LLM answers for two tasks and six domains","period":"2025","published":"2025-06-06","publisher":"Puerto et al., Parameter Lab and TU Darmstadt (arXiv 2506.11097, NeurIPS 2025 Datasets and Benchmarks)","sponsor":"None stated","conflict":false,"engines":["gpt-4o-mini","Claude 3.5 Haiku","o3","o4-mini"],"title":"C-SEO Bench: Does Conversational SEO Work?","url":"https://arxiv.org/abs/2506.11097","caveat":"A simulated conversational search engine over fixed documents, not live AI search products.","topicLabel":"GEO experiments"},{"footnote":27,"id":"kumar-strategic-text","topic":"geo","claim":"Adding an optimized strategic text sequence to a product page raised the product's rank in about 40% of 200 randomized LLM evaluations and left it unchanged in about 60%.","metric":"Share of evaluations where the target product's rank improved","sample":{"n":200,"unit":"LLM evaluations with randomized product order"},"method":"Adversarial text optimization (GCG) on a catalog of fictitious coffee machines","period":"2024","published":"2024-04-11","publisher":"Kumar and Lakkaraju, Harvard University (arXiv 2404.07981)","sponsor":"None stated","conflict":false,"engines":["Llama-2 (open model)"],"title":"Manipulating Large Language Models to Increase Product Visibility","url":"https://arxiv.org/abs/2404.07981","caveat":"A fictitious catalog and an open model, not a live search product; it measures manipulation, not a recommended practice.","topicLabel":"GEO experiments"},{"footnote":28,"id":"cloudflare-crawl-to-refer","topic":"crawlers","claim":"For June 19 to 26, 2025, crawl-to-refer ratios ranged from 70,900 HTML page crawls per referral for Anthropic down to 0.1 for Mistral.","metric":"Crawler HTML requests per referred HTML request","sample":{"n":null,"unit":"requests across Cloudflare's network"},"method":"HTML requests from each platform's crawlers divided by HTML requests whose Referer is that platform","period":"June 19 to 26, 2025","published":"2025-07-01","publisher":"Cloudflare (Radar)","sponsor":"Cloudflare (sells AI Crawl Control and pay per crawl)","conflict":true,"engines":["Anthropic","OpenAI","Perplexity","Google","Mistral","and others on Radar"],"title":"The crawl before the fall... of referrals: understanding AI's impact on content providers","url":"https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radar/","caveat":"Native apps such as Claude's send no Referer header, so their referrals are undercounted, as Cloudflare notes.","topicLabel":"Crawlers and llms.txt"},{"footnote":29,"id":"cloudflare-crawl-purpose","topic":"crawlers","claim":"Over the 12 months to August 2025, 80% of AI crawling was for training, against 18% for search and 2% for user actions.","metric":"Share of AI crawler requests by declared purpose","sample":{"n":null,"unit":"requests across Cloudflare's network"},"method":"Crawler traffic classified by purpose from operator disclosures and industry sources","period":"12 months before August 29, 2025","published":"2025-08-29","publisher":"Cloudflare (Radar)","sponsor":"Cloudflare (sells AI Crawl Control and pay per crawl)","conflict":true,"engines":["AI crawlers seen by Cloudflare"],"title":"The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals","url":"https://blog.cloudflare.com/crawlers-click-ai-bots-training/","caveat":"Purpose comes from what operators declare about each crawler, not from how the data is used.","topicLabel":"Crawlers and llms.txt"},{"footnote":30,"id":"cloudflare-robots-ai-rules","topic":"crawlers","claim":"Of 3,816 top 10,000 domains with a robots.txt, 546 (about 14%) had allow or disallow directives aimed at AI bots.","metric":"Share of robots.txt files with AI bot directives","sample":{"n":3816,"unit":"domains with a robots.txt, from the top 10,000"},"method":"robots.txt files fetched by Cloudflare Radar, matched against a list of AI crawler names","period":"June 6, 2025","published":"2025-07-01","publisher":"Cloudflare (Radar)","sponsor":"Cloudflare (sells AI Crawl Control and pay per crawl)","conflict":true,"engines":["AI crawlers (GPTBot most named)"],"title":"From Googlebot to GPTBot: who's crawling your site in 2025","url":"https://blog.cloudflare.com/from-googlebot-to-gptbot-whos-crawling-your-site-in-2025/","caveat":"Following robots.txt is voluntary; a directive shows what a site asks, not what crawlers do.","topicLabel":"Crawlers and llms.txt"},{"footnote":31,"id":"reuters-news-blocking-2023","topic":"crawlers","claim":"By the end of 2023, 48% of the most widely used news websites in ten countries blocked OpenAI's crawlers and 24% blocked Google's AI crawler.","metric":"Share of top news sites blocking AI crawlers in robots.txt","sample":{"n":15,"unit":"most used news sites in each of 10 countries"},"method":"Archived robots.txt files from the Wayback Machine for every available day of 2023","period":"2023","published":"2024-02-22","publisher":"Reuters Institute for the Study of Journalism","sponsor":"None stated","conflict":false,"engines":["GPTBot","ChatGPT-User","Google-Extended"],"title":"How many news websites block AI crawlers?","url":"https://reutersinstitute.politics.ox.ac.uk/how-many-news-websites-block-ai-crawlers","caveat":"An end-of-2023 snapshot from before most publisher licensing deals; blocking rules do not show what crawlers obey.","topicLabel":"Crawlers and llms.txt"},{"footnote":32,"id":"buzzstream-news-blocking","topic":"crawlers","claim":"79% of top U.S. and UK news sites blocked at least one AI training bot in robots.txt, and 71% blocked at least one AI retrieval or live search bot.","metric":"Share of sites with disallow rules for AI bots","sample":{"n":100,"unit":"top news sites (50 U.S., 50 UK, deduplicated)"},"method":"robots.txt disallow directives checked for 11 AI crawlers","period":"2025 to 2026","published":"2026-04-08","publisher":"BuzzStream","sponsor":"BuzzStream (sells outreach and digital PR software)","conflict":false,"engines":["11 AI crawlers, including GPTBot, ClaudeBot and PerplexityBot"],"title":"Which News Sites Block AI Crawlers in 2025? [New Data]","url":"https://www.buzzstream.com/blog/publishers-block-ai-study/","caveat":"Sites chosen by SimilarWeb traffic share; robots.txt is a request, and the post notes bots can ignore it.","topicLabel":"Crawlers and llms.txt"},{"footnote":33,"id":"seranking-llms-txt","topic":"crawlers","claim":"10.13% of nearly 300,000 domains had an llms.txt file, and removing llms.txt from a model of AI citation frequency made its predictions more accurate.","metric":"llms.txt adoption and its value as a predictor of AI citations","sample":{"n":300000,"unit":"domains (nearly)"},"method":"Check for llms.txt on each domain, then Spearman correlation, XGBoost regression and SHAP analysis","period":"Not stated on the page","published":"2026-10-15","publisher":"SE Ranking","sponsor":"SE Ranking (sells SEO tools and an AI Search Toolkit)","conflict":true,"engines":["LLM responses (models not named)"],"title":"LLMs.txt: Why Brands Rely On It and Why It Doesn't Work","url":"https://seranking.com/blog/llms-txt/","caveat":"The citation sample and model are not published in detail; a weak predictor in one model is not proof of no effect.","topicLabel":"Crawlers and llms.txt"},{"footnote":34,"id":"rankability-llms-txt","topic":"crawlers","claim":"9.3% of the top 1,000 domains and 8.3% of the top 10,000 served a confirmed llms.txt or llms-full.txt file.","metric":"Confirmed llms.txt or llms-full.txt adoption","sample":{"n":10000,"unit":"domains from the Tranco list"},"method":"Root file check of each domain, with blocked or inconclusive checks kept in the denominator","period":"September 18, 2026","published":"2026-09-18","publisher":"Rankability","sponsor":"Rankability (sells AI search visibility software for agencies)","conflict":true,"engines":[],"title":"llms.txt Adoption Research Report","url":"https://www.rankability.com/data/llms-txt-adoption/","caveat":"Unknown results count as not confirmed (474 of the top 1,000), so the shares are floors for the ranked list.","topicLabel":"Crawlers and llms.txt"},{"footnote":35,"id":"commoncrawl-llms-txt","topic":"crawlers","claim":"11.72% of 5,167,831 sampled /llms.txt URLs returned a text body, and 68.27% of the 584,107 files collected were templated.","metric":"Share of sampled hosts serving llms.txt, and content of the files","sample":{"n":5167831,"unit":"/llms.txt URLs from a random sample of hosts"},"method":"Crawl experiment: /llms.txt added to the seed list for a random sample of hosts in the July 2026 crawl","period":"July 2026 crawl (CC-MAIN-2026-30)","published":"2026-08-31","publisher":"Common Crawl Foundation","sponsor":"None stated (nonprofit that operates CCBot)","conflict":false,"engines":[],"title":"A Content Analysis of llms.txt Files from the July 2026 Crawl Archive","url":"https://commoncrawl.org/blog/a-content-analysis-of-llms-txt-files-from-the-july-2026-crawl-archive","caveat":"Hosts Common Crawl can already fetch, not the whole web; the post says the rate is not comparable to the Web Almanac figure.","topicLabel":"Crawlers and llms.txt"},{"footnote":36,"id":"ahrefs-llms-txt-requests","topic":"crawlers","claim":"28% of 137,210 domains published an llms.txt file, and 97% of those files received zero requests in May 2026.","metric":"llms.txt adoption and requests to the file","sample":{"n":137210,"unit":"domains in Ahrefs Web Analytics with traffic"},"method":"Root file check plus bot and user-agent requests to the llms.txt path","period":"May 2026","published":"2026-06-15","publisher":"Ahrefs","sponsor":"Ahrefs (sells SEO tools and Brand Radar for AI visibility)","conflict":true,"engines":["AI bots and assistants that request llms.txt"],"title":"We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read","url":"https://ahrefs.com/blog/llmstxt-study/","caveat":"Ahrefs Web Analytics customers skew technical, so the study treats the 28% adoption figure as an upper bound.","topicLabel":"Crawlers and llms.txt"},{"footnote":37,"id":"aeofootnotes-lab-directory","topic":"crawlers","claim":"Of the 118 domains of the tools and agencies in the AEO Footnotes directory, 116 served a robots.txt and 64 served an llms.txt.","metric":"robots.txt and llms.txt presence, AI crawler blocking","sample":{"n":118,"unit":"domains"},"method":"Fetch of /robots.txt and /llms.txt on each domain; blocking judged by the RFC 9309 group that applies","period":"October 29, 2026","published":"2026-10-29","publisher":"AEO Footnotes (Footnotes Lab)","sponsor":"None; AEO Footnotes runs the directory the sites come from and sells no AEO product","conflict":false,"engines":["14 AI crawler user agents"],"title":"robots.txt and llms.txt across 118 AEO tools and agency sites: what we found","url":"https://aeofootnotes.com/lab/robotstxt-and-llmstxt-across-118-aeo-tools-and-agency-sites-what-we-found","caveat":"Sites in one directory, not a sample of the web; publishing llms.txt says nothing about whether AI systems read it.","topicLabel":"Crawlers and llms.txt"}]}