AI Search Visibility Audit Checklist (Copyable)
Audit AI search visibility in seven passes: confirm AI crawlers can fetch your pages (robots.txt and the CDN), publish a correct llms.txt, ship structured data in the served HTML, put the answer in each key page's first sentences, describe your company the same way everywhere, link your claims to sources and check the pages assistants cite, then measure a fixed prompt set on every assistant you care about. Copy the checklist below into a doc or an issue and work through it.
The checklist
It is markdown with task boxes, so it pastes into a GitHub issue, Notion or any doc. Work top to bottom: if a crawler cannot fetch a page, nothing later in the list matters for that page.
# AI search visibility audit
Site: [domain] Date: [YYYY-MM-DD] Auditor: [role]
## 1. Crawler access (robots.txt and the edge)
- [ ] https://[domain]/robots.txt returns 200 and names your sitemap
- [ ] A written policy exists for each crawler group: search crawlers, AI search crawlers, user-triggered fetchers, training crawlers
- [ ] Search crawlers allowed on the pages you want cited: Googlebot (Google Search and its AI features), Bingbot (Bing and Microsoft Copilot)
- [ ] AI search crawlers allowed on those pages: OAI-SearchBot (ChatGPT search), Claude-SearchBot, PerplexityBot
- [ ] Training crawlers set to match your policy: GPTBot, ClaudeBot, Google-Extended (a robots.txt token for Gemini; it does not affect Google Search), Applebot-Extended
- [ ] User-triggered fetchers not blocked at the CDN or WAF: ChatGPT-User, Claude-User, Perplexity-User (OpenAI and Perplexity say robots.txt may not apply to these)
- [ ] No inherited "Disallow: /" under "User-agent: *" or under a named agent
- [ ] Each key page fetched with each user agent (curl -A "GPTBot" URL) returns 200 and the real HTML, not a challenge page
## 2. llms.txt
- [ ] https://[domain]/llms.txt returns 200 as plain text, without redirecting to an HTML page
- [ ] Starts with an H1 (the name), then a blockquote summary
- [ ] Links sit in H2 sections as "- [name](url): notes"
- [ ] Every link is an absolute URL that returns 200
- [ ] Secondary links are under "## Optional"
- [ ] Prices, plans and "what we do not do" match the live site
## 3. Structured data
- [ ] JSON-LD is in the HTML the server sends (curl the URL and search for "application/ld+json")
- [ ] Every block parses and passes the Schema Markup Validator (validator.schema.org)
- [ ] Home page: Organization with name, url, logo and sameAs links to profiles you control
- [ ] Articles: Article or BlogPosting with headline, author, datePublished and dateModified
- [ ] Product or SoftwareApplication pages: offers match the pricing page
- [ ] FAQPage only where the same questions and answers are visible on the page
- [ ] BreadcrumbList on pages below the home page
- [ ] Nothing marked up that a reader cannot see
## 4. Answer-first content
- [ ] Each key page answers its main question in the first one or two sentences
- [ ] That answer is in the server-rendered HTML, not only after JavaScript runs
- [ ] Headings are phrased the way buyers ask, one question per heading
- [ ] Comparisons, specs and prices are HTML text or tables, not images
- [ ] Pages whose facts change show a visible "last updated" date
## 5. Entity consistency
- [ ] One company name, one product name and one short description, used on the site, in llms.txt, in schema and on your profiles
- [ ] Plan names, prices and limits identical everywhere they appear
- [ ] Profiles you control (LinkedIn, GitHub, app stores, directories) link back to the site and say the same thing
- [ ] Old product names and retired features redirect, or say plainly that they are retired
## 6. Citations and sources
- [ ] Claims on your pages link to their primary sources
- [ ] Your own facts (docs, pricing, changelog, policies) live at stable URLs others can cite
- [ ] List the third-party pages assistants cite for your category (from section 7)
- [ ] For each one: are you listed, and is what it says about you correct?
- [ ] Corrections requested where a cited page is wrong about you
## 7. Measuring prompts
- [ ] A fixed prompt set written down: category, problem, comparison and brand questions a buyer would type
- [ ] Each prompt run on every assistant you care about, more than once, because answers change between runs
- [ ] Recorded per answer: assistant, date, mentioned (yes/no), position of first mention, competitors named, URLs cited, tone
- [ ] Results reported per assistant, not blended
- [ ] Changes you ship logged with dates, so a later re-run can be read against them
- [ ] Next re-run date: [YYYY-MM-DD]
1. Crawler access
The crawlers that build AI answers are not one group, and blocking the wrong one has the opposite effect to the one intended. Decide a policy per group, then make robots.txt and your CDN or firewall implement it.
| User agent | Operator | What it does |
|---|---|---|
| OAI-SearchBot | OpenAI | Surfaces sites in ChatGPT search. Sites that block it do not appear in ChatGPT search answers. |
| GPTBot | OpenAI | Collects content that may be used to train OpenAI's models. Blocking it does not remove you from ChatGPT search. |
| ChatGPT-User | OpenAI | Fetches a page when a ChatGPT user asks. OpenAI says robots.txt rules may not apply to it. |
| Claude-SearchBot | Anthropic | Indexes content to improve Claude's search results. |
| ClaudeBot | Anthropic | Collects content that may be used to train Anthropic's models. |
| Claude-User | Anthropic | Fetches a page when a Claude user asks a question that needs it. |
| PerplexityBot | Perplexity | Surfaces and links sites in Perplexity's search results. Perplexity says it is not used for model training. |
| Perplexity-User | Perplexity | Fetches a page for a user's question. Perplexity says it generally ignores robots.txt. |
| Google-Extended | Not a crawler but a robots.txt token: controls use of your content for Gemini training and grounding. Google says it does not affect inclusion or ranking in Google Search. | |
| Googlebot, Bingbot | Google, Microsoft | Classic search crawlers. Google's AI features in Search and Microsoft Copilot draw on these indexes. |
robots.txt is only half of it. A CDN or firewall rule can challenge these agents before the request reaches your server, and nothing in your own logs will show it. Fetch your key pages with each user agent and confirm you get the real HTML.
2. llms.txt
An llms.txt file is a curated map of your site in markdown, following the proposal at llmstxt.org. It is cheap to publish and easy to get wrong in ways that matter: stale prices, dead links, a redirect to an HTML page. Our llms.txt template has the full format and a filled example.
3. Structured data
Structured data states facts in a form a machine reads without guessing: who you are, what a page is, when it changed, what it costs. Two failures are common and invisible in a browser: JSON-LD added by JavaScript that is missing from the served HTML, and markup that says something the page does not. Check both with curl and the Schema Markup Validator.
4. Answer-first content
An assistant building an answer looks for a sentence it can use. Pages that open with background and reach the answer three paragraphs down give it less to work with than pages that answer first and explain after. Check the first one or two sentences of every key page, in the HTML the server sends.
5. Entity consistency
If your site, your llms.txt, your schema and your profiles describe the company in different words, or list different prices, an assistant has to choose between them. Pick one name, one description and one set of facts, and use them everywhere you control.
6. Citations and sources
Assistants often cite pages that are not yours: comparisons, directories, reviews and forums. The measurement in section 7 tells you which ones. For each, check whether you are listed and whether what it says is correct, and keep your own facts at stable URLs that are easy to cite.
7. Measuring prompts
Visibility is measured in answers, not in files shipped. Keep a fixed set of the questions buyers ask, run each one on every assistant more than once because answers vary between runs, and record who is named and which URLs are cited. Report per assistant and compare only runs of the same prompt set.
How Fenn helps
Fenn covers several of these passes for you. Its readiness check tests your site's main page for AI-crawler access, llms.txt, structured data and FAQ markup, answer-first content, freshness and internal links, and turns each failure into a fix with a copyable prompt. Its measurement asks ChatGPT, Claude, Gemini, Perplexity and Grok your customers' questions when you start it, and shows how often each names you, who it names instead and which sources it reads. Results copy as text, print as a PDF or export to CSV.
FAQ
How often should I run an AI search visibility audit?
Run the full checklist after any replatform, CDN or firewall change, or robots.txt edit, and at least once a quarter. Re-run the prompt measurement after each change you ship, so you can read the results against it.
If I block GPTBot, will my site disappear from ChatGPT?
Not from ChatGPT search. OpenAI documents GPTBot as its training crawler and OAI-SearchBot as the crawler that surfaces sites in ChatGPT search, so the two are controlled separately.
Does passing every item guarantee AI citations?
No. The checklist makes your pages retrievable and easy to quote. Whether an assistant cites you also depends on the question, the competing pages and the sources it trusts, which is why the last section measures real answers.
Why do I get different answers to the same prompt?
Assistants generate each answer fresh, and some search the web for each request, so names and citations can change between runs. Run each prompt more than once and compare like with like.
Which AI assistants should I measure?
The ones your buyers use. Report each one separately, because an assistant can name you often while another never does. Fenn measures ChatGPT, Claude, Gemini, Perplexity and Grok.
Sources
Checked on 14 October 2026.