Can AI read your site?
Check robots.txt for every major AI crawler, and whether your llms.txt is set up the way AI tools expect.
What it checks
AI crawlers
GPTBot, ChatGPT search and browsing, ClaudeBot, PerplexityBot and Google-Extended, allowed or blocked.
Google
Whether Googlebot can crawl the homepage, and which sitemaps robots.txt lists.
llms.txt
Found or not, and whether it has a title, a one-line summary and links to your key pages.
Questions
Should I block AI crawlers?
If you want to show up in ChatGPT, Perplexity or Gemini answers, their search crawlers need to read your pages. Blocking training crawlers like GPTBot is a separate choice.
What is llms.txt?
An optional markdown file at /llms.txt with a title, a short summary and links to your key pages, so AI tools can find what matters.
My crawlers are allowed. Why doesn't ChatGPT mention me?
Being readable is step one. Askline shows whether ChatGPT names you for the questions buyers ask, and what to change when it doesn't.
More free tools
Stop guessing. Start asking.
Add your website. See what ChatGPT says about you, and the Google SEO behind it. Nothing to install.
$35/mo · Claude + Cursor