URL Status & SEO Checker

See the HTTP status, robots.txt rule, meta robots, X-Robots-Tag and canonical of a URL, fetched from our server.

Analysis Results
Final URL
Indexing Status
Robots.txt
Meta Robots
X-Robots-Tag
Canonical URL
Content Type
Server
Whois Details

What the checker does

Enter a URL and press Check Status. Our server, not your browser, requests the page with cURL under the user agent UrlStatusChecker/1.0, follows up to five redirects and gives up after 10 seconds without a connection or 15 seconds in total. It then requests /robots.txt from the host you entered. From the two responses it reports:

From these it labels the URL Indexable or Non-Indexable with a one-line reason. If the site's TLS certificate cannot be verified, the fetch is repeated without verification so that you still get a result. The tool does not report certificate problems, so an HTTPS result here says nothing about the certificate.

This is not Google's index

The verdict is an inference about what a well-behaved crawler is allowed to do with the URL. Nothing on this page queries Google, Bing or any other search engine, so "Indexable" does not mean indexed and "Non-Indexable" does not mean absent from results. Google may crawl an indexable page and still leave it out. Google also documents that robots.txt is not a mechanism for keeping a page out of Google, so a blocked URL can still appear in results, without a description, if other pages link to it. To see whether Google has indexed a URL, use the URL Inspection tool in Search Console, which also shows the page as Googlebot rendered it.

How to read each row

HTTP status

The badge shows the status of the final response after redirects. A chain of 301 to 200 therefore shows HTTP 200, with the Final URL row telling you where it landed. 404 and 410 mean the page is gone, 5xx means the server failed, and a 3xx appears only when the chain is longer than five hops. A 403 may mean the site refuses unknown bots such as ours while serving Googlebot normally, so treat it as a prompt to check in Search Console rather than as proof of a problem. The tool cannot detect a soft 404, a "not found" message delivered with status 200.

Indexing status

The URL is marked Indexable (Accessible) when the status is below 400, no robots.txt rule blocks it, and neither the meta tag nor the header contains noindex. Otherwise the reason shown is the first of these that applies: Blocked by robots.txt, Meta noindex found, X-Robots-Tag: noindex found, HTTP error status. The order matters: a robots.txt block hides everything else, which mirrors real crawling. Google explains that a noindex rule only works when the page is not blocked by robots.txt, because a crawler that may not fetch the page never sees the tag.

Robots.txt

Allowed or Blocked is decided the way Google's robots.txt specification describes: among the rules in the User-agent: * group that match the start of your path, the longest rule wins, an Allow beats a Disallow of equal length, and the wildcards * (any characters) and $ (end of URL) are honoured. Groups aimed at a named crawler are ignored, so a site that disallows Googlebot specifically while allowing everyone else shows Allowed here. "Not found or not readable" means the robots.txt request did not return 200; crawlers treat a missing file as permission to crawl everything, while a server error on robots.txt is handled differently by each crawler. The file is always read from the host you typed, even when the URL redirects to another host.

Meta robots and X-Robots-Tag

These rows show the raw values. Only noindex affects the verdict; nofollow, noarchive, max-snippet and similar directives are displayed but not interpreted. The tool reads the HTML as the server sent it and does not run JavaScript, so a meta tag added by a script is not seen, and it reads only the tag named robots, not one named googlebot. A header aimed at one crawler, such as bingbot: noarchive, is shown as it is.

Canonical URL, Content-Type, Server, Whois

The canonical row shows the first canonical link in the HTML without checking that it resolves; compare it with the Final URL, since a canonical pointing elsewhere tells search engines to credit that other address. Content-Type and Server are the headers as sent; Server often names a CDN such as Cloudflare rather than the software behind it. The Whois row links the host name to our WHOIS lookup.

What this tool cannot tell you

Related tools: the robots.txt validator checks the syntax of a whole robots.txt file, the robots.txt generator writes one, and the meta tag generator produces a correct robots meta tag for a page.