Skip to content

Free, in your browser, no sign-up

Find out which AI crawlers can read your site.

Paste your robots.txt. The check runs the same RFC 9309 parser our crawler uses and shows, crawler by crawler and path by path, who may fetch what and which rule decides it.

Nothing is uploaded. The file never leaves your browser.

AI crawler check

Example: the demo store’s robots.txt

12 of 15 AI crawlers can fetch every path you listed; 2 are blocked from some; 1 from all.

  • GPTBotOpenAIAll 5 paths
  • OAI-SearchBotOpenAIAll 5 paths
  • ChatGPT-UserOpenAIBlocked from 2
  • ClaudeBotAnthropicAll 5 paths
  • Claude-SearchBotAnthropicAll 5 paths
  • Claude-UserAnthropicAll 5 paths
  • PerplexityBotPerplexityAll 5 paths
15 crawlers × 5 paths · RFC 9309 longest-match · computed in the browser

Newsletter and podcast.

The same findings in two more formats, each linking back to the page it comes from.

The newsletter

One finding a fortnight, with the number, its interval and where it came from. Published on Substack and archived here. No launches, no discounts.

Demo: nothing is sent from this page.

The podcast

Each white paper, read by its author and declared as such, as an episode on Apple Podcasts and YouTube. The audio is a version of the page, never a separate claim: the canonical text stays here.

Episodes appear here as they are recorded.

One entity, every format.

Where AwesomeSEO publishes. Each paper becomes an article, a walkthrough, an episode and an issue; every copy links to the canonical page here. These are channels we publish to, shown with their official marks, not endorsements.
LinkedIn
Each white paper as an article, with its data
YouTube
Walkthroughs on the demo workspace
Apple Podcasts
Audio versions of the papers, read by the author
Substack
The newsletter: one finding a fortnight
Medium
Syndicated articles, canonical to this site
Instagram
The comparison infographic, sourced and dated
TikTok
One-minute cuts of the walkthroughs
X
Threads of the findings, with the numbers

AI use is declared in every asset that used it. No invented testimonials, no seeded engagement, no tactic from the playbook’s excluded tier (10 of them, listed on the principles page).

See all of it run on a real workspace.

The demo workspace runs every module on a fictional Swedish linen brand, with example data. Nothing to install.