How to Check if ChatGPT and Perplexity Can See Your Website

Published 2026-11-03 · ipok.fyi

More people now ask an AI assistant instead of typing a query into Google: "best merino socks", "which IP checker is free", "Shopify stores like Allbirds". If your site can't be fetched, read or understood by these systems, you simply won't appear in the answer. Optimizing for this is often called GEO (Generative Engine Optimization) or AI visibility.

This guide shows what to check, how to check it manually, and how to run all of it in one go with the free ipok.fyi GEO checker.

How AI assistants find your content

Roughly, there are two ways your pages reach an AI answer:

Training and indexing crawls. Bots such as OpenAI's GPTBot, Anthropic's ClaudeBot and PerplexityBot fetch pages to build their models or search indexes.
Live retrieval. When a user asks a question with web search on, the assistant fetches pages in real time (for example, OpenAI's ChatGPT-User / OAI-SearchBot or Perplexity's fetchers) and cites them.

Both depend on the same basics: the bot must be allowed, the page must return content without JavaScript tricks, and the content must be easy to interpret.

Check 1: robots.txt rules for AI user-agents

Open https://yoursite.com/robots.txt and look for these user-agents: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended. A rule like:

User-agent: GPTBot
Disallow: /

blocks that bot entirely. Many sites added such blocks in 2023–2024 without realizing it might also remove them from AI search answers. Also watch for a broad User-agent: * / Disallow: / and CDN-level bot blocking (a firewall can return 403 to AI bots even if robots.txt allows them).

Check 2: Is your content in the raw HTML?

AI fetchers frequently do not run JavaScript the way a browser does. If your product descriptions, prices or articles only appear after client-side rendering, a bot may see an empty shell. Test it:

curl -s https://yoursite.com/page | grep -i "a sentence from your page"

If the text isn't there, the bot probably can't see it either. Server-side rendering or static generation fixes this.

Check 3: Structured data (JSON-LD)

Schema.org markup tells machines exactly what a page is. For stores, Product (with offers, price, availability) and Organization matter most; for content, Article and FAQPage. Validate with Google's Rich Results Test or the Schema Markup Validator.

Check 4: llms.txt and sitemap

llms.txt is a proposed convention: a Markdown file at /llms.txt that summarizes your site and links key pages for language models. It isn't an official standard and support varies, but it's cheap to add. (ipok.fyi publishes its own at ipok.fyi/llms.txt.)
sitemap.xml referenced from robots.txt helps every crawler discover your pages.

Check 5: Basic technical SEO still matters

AI search systems often build on traditional search indexes, so titles, meta descriptions, canonical tags, status codes, HTTPS and speed still count. Run the free ipok.fyi SEO checker to cover these.

Check 6: Ask the AI directly

Technical checks tell you whether you can be seen. To learn whether you are mentioned, ask assistants with web search the questions your customers ask ("best [category] brands", "[your brand] alternatives") and note whether you're cited. Doing this by hand across several models is tedious, which is what the live AI probe in the GEO checker automates.

Run all checks at once with ipok.fyi

Go to ipok.fyi/geo and enter your URL. No signup.
The tool runs 16 weighted technical checks — including robots.txt rules for GPTBot, ClaudeBot and PerplexityBot, llms.txt, sitemap, JSON-LD, Product schema and raw-HTML content — and gives an overall score.
Each failed check comes with a copy-paste fix (robots.txt lines, Product JSON-LD, an llms.txt template).
Optionally run the live multi-model AI visibility test to see whether models with web search mention your brand. One test per day is free; extra tests use credits (see pricing).

We used the same checks on nine well-known Shopify stores; the write-up is in How to get your Shopify store recommended by ChatGPT.

Quick fixes that usually help

Remove unintended Disallow rules for AI search bots you want to appear in (you can still block training-only bots if you prefer).
Check your CDN/WAF bot settings, not only robots.txt.
Make sure key text and prices are server-rendered.
Add Product, Organization and FAQPage JSON-LD where relevant.
Publish llms.txt and keep sitemap.xml current.
Write clear, factual pages that answer specific questions — assistants quote concise, well-structured content.

FAQ

What is GEO?

Generative Engine Optimization: making your site easy for AI assistants and AI search to crawl, understand and cite.

Does blocking GPTBot remove me from ChatGPT?

GPTBot is OpenAI's crawler for training. OpenAI documents separate agents (OAI-SearchBot, ChatGPT-User) for search and user-triggered fetches. Check your rules for each, and review OpenAI's current bot documentation since policies change.

Is llms.txt required?

No. It's an optional, emerging convention. It doesn't hurt and may help some tools summarize your site.

Can a site pass all technical checks and still not be mentioned?

Yes. Access is necessary but not sufficient — models also weigh relevance, authority and how often your brand appears across the web. That's why the live AI probe is separate from the technical score.

Is the GEO check free?

The technical GEO and SEO checks are free with no signup. The live multi-model AI test includes one free run per day.

Check your site now: ipok.fyi/geo · SEO checker

Run the free GEO / AI visibility check on your store →