Free · No sign-up
AI Crawler Checker
Find out which AI crawlers your robots.txt lets in, and which ones it shuts out, before it costs you citations in ChatGPT, Claude and Perplexity.
How to use it
Three steps to a clear answer.
- 01
Enter your domain
Type the site you want to check. Only its public robots.txt and llms.txt are read.
- 02
We apply your rules
Each crawler is matched to its group in robots.txt and the most specific rule for your home page decides, the way the crawlers do it.
- 03
Fix what blocks you
If an AI search crawler is blocked, remove or narrow the rule that matched it and run the check again.
Go further · AI Visibility
Crawlable is step one. Getting cited is the goal.
RanklyBee AI Visibility tracks the questions buyers ask ChatGPT, Gemini, Claude and Perplexity, shows whether you are mentioned, and which sources they cite instead.
FAQ