AI Search Brief

Free tools for SEO and AI search. No sign-up, and nothing you check is stored.

← All tools

AI Crawler Access Checker

Enter a domain and we will read its robots.txt the way a crawler does — resolving the same group and longest-match rules — to show which AI crawlers can reach it. Nine agents, grouped by what a block actually costs you.

We read two files — /robots.txt and /llms.txt — and store nothing.

Two things this tool will not pretend

Blocking a crawler is not always wrong. Whether your content should train someone's model is a rights decision, not an SEO one, and it is yours. The report separates the crawlers that affect whether you are cited in AI answers from the ones that only affect training, so the two decisions stay apart.

robots.txt is a request, not a fence. It tells well-behaved crawlers what to skip. It does not enforce anything, and operators vary in how closely they honour it — so “Blocked” here means you have asked, not that you have been obeyed.

Never lose a story.

Sign in to save it for later.