Free tool

Indexability checker

Can Google, Bing, ChatGPT search, Perplexity and Claude index this page? Check noindex, canonical, robots.txt and sitemap in one go.

Free, no signup. One page: status, noindex, canonical, robots.txt and sitemap. Nothing is stored.

How the indexability check works

  1. We load the page as a browser, follow any redirects, and read the status code, the X-Robots-Tag header and the raw HTML.
  2. We read every robots meta tag and the canonical (from the HTML and the Link header). If the canonical points elsewhere, we load that URL too.
  3. We read robots.txt and apply the rules for Googlebot, Bingbot, OAI-SearchBot, PerplexityBot and Claude-SearchBot to this path.
  4. We request the page as each AI search crawler, to catch firewalls that refuse them, and look for the URL in your sitemap.

What the indexability checker looks at

Signals read by the indexability checker
SignalWhat we look forWhy it matters
HTTP statusThe page answers 200 (after any redirects)Error pages are dropped from the index
X-Robots-Tag headernoindex, none, nosnippet in the response header, for all bots or oneInvisible in the page source, often set by a CDN
Meta robots<meta name="robots">, plus googlebot, bingbot and AI bot meta tagsThe usual way pages get hidden by accident
CanonicalSelf, another URL, missing, or more than one; the target's status and noindexEngines index the canonical URL, not yours
robots.txtRules for Googlebot, Bingbot, OAI-SearchBot, PerplexityBot and Claude-SearchBot on this pathA blocked crawler can't read the page
FirewallA live request as each AI search crawlerCDNs often refuse bots that robots.txt allows
SitemapThe URL is listed in your XML sitemapHelps engines find and recrawl it
Raw HTML textWords on the page before JavaScript runsAI crawlers don't run JavaScript

How to read the matrix

  • Crawl allowed: robots.txt lets that engine’s crawler fetch this path, and for the AI crawlers, the live request wasn’t refused.
  • Indexable: the page answers 200, no noindex applies to that engine, and (for the AI crawlers) the text is in the raw HTML.
  • Canonical OK: the canonical points to this page, or there is none. A canonical to another URL means engines index that one instead.

Redirects in the way? The redirect checker shows every hop. To check a whole site rather than one page, run the website SEO checker.

FAQ

What makes a page indexable?

Four things: the page answers 200, crawlers are allowed to fetch it (robots.txt and the firewall let them in), nothing tells engines not to index it (no noindex in a meta tag or the X-Robots-Tag header), and the canonical points to the page itself, or is missing. If any of these fails, engines either skip the page or index a different URL instead.

Why check Bing for ChatGPT search?

ChatGPT search draws on Bing's index along with OpenAI's own crawler, OAI-SearchBot. A page that Bing can't index, or that blocks OAI-SearchBot, is much harder for ChatGPT to find and cite. That's why the Bing row checks both bingbot and OAI-SearchBot.

Where can a noindex hide?

In three places: a <meta name="robots"> tag, a bot-specific meta tag like <meta name="googlebot">, and the X-Robots-Tag HTTP header. The header is easy to miss because it never shows in the page source; it is often set by a CDN, a server rule or a plugin. This check reads all three.

My canonical points to another URL. Is that bad?

Not always. On a duplicate (a filtered product list, a print version, a tracking URL) it is exactly right. On a page you want found, it tells engines to index the other URL instead, so this one won't show up. A canonical that points to a broken, redirected or noindex page is always a mistake.

Does robots.txt stop a page from being indexed?

Not exactly. robots.txt stops crawling, not indexing. Google can still list a blocked URL without its content if other sites link to it, but it can't read the page, so it can't see a noindex tag on it and AI assistants can't quote it. To keep a page out of the index, allow crawling and use noindex.

Why does it matter if the text is in the raw HTML?

Google renders JavaScript before indexing. The crawlers behind ChatGPT, Claude and Perplexity mostly don't. If the text only appears after JavaScript runs, those crawlers see an almost empty page, even though Google indexes it fine.

Is this indexability checker free?

Yes. It doesn't use any AI model, you don't need an account, and nothing about the page is stored. To keep it fair, each visitor can run a few checks every 10 minutes.

Learn more

More free tools

By Daniel Borodin, GEOHATS founder · Last updated