Noindex Header Test

SEOptimer’s Noindex Header Test scans your webpage to check for the presence of a "Noindex" directive in the HTTP header, helping you identify and remove it to ensure your page is properly indexed by search engines.

SEO Audit Any Site in Seconds
+ Comprehensive SEO / GEO Tools for Every Function

Run over 100 checks on your website across On-Page SEO, Usability, Performance, Social and Links. Now with GEO in every report. SEOptimer also offers keyword research, rank tracking, backlinks and a full site crawler.

Start 14 Day Free Trial

About Noindex Headers

What is a noindex header?

A noindex header is an instruction to keep a page out of search results, delivered with the HTTP response rather than written into the page HTML. It does the same job as a noindex meta tag; it simply arrives by a different route.

The header that carries it is called X-Robots-Tag, and you will see that name used interchangeably with noindex headers in most documentation. It can carry other directives too, such as nofollow or noarchive, but noindex is the one with consequences worth checking for.

Because the instruction travels in the response, it can apply to files that have no HTML to put a meta tag in: PDFs, images, spreadsheets, downloads. It is also how server-level rules get applied across whole directories at once.

Do noindex headers still matter in 2026?

They matter disproportionately because they are invisible. A noindex in the page source can be found by anyone viewing it; one in a header cannot, and it can quietly suppress a page for months while everyone inspects the HTML and finds nothing wrong.

They are also set at the server or CDN layer, which means they often sit outside the content management system entirely. The person debugging a page that will not index frequently has no access to the place the directive lives.

Used deliberately they remain the correct tool for non-HTML files, which is a genuinely common need on sites publishing documents.

How noindex headers relate to AI search

A noindex header excludes content from indexes in exactly the same way a meta tag does, and therefore from the material AI answers are built on. The cost of an accidental one has grown accordingly.

It is also increasingly used to apply AI-specific directives at the file level, which makes it worth understanding rather than treating as a legacy header nobody sets any more.

For documents in particular this cuts both ways. A PDF you would rather was not quoted is best excluded here; a PDF you want found should not be.

Noindex header best practices

Check the response headers, not just the HTML, when a page will not index.
Use them for non-HTML files, where a meta tag is not an option.
Keep server rules narrow. A directory-wide rule easily catches more than intended.
Make sure the header does not contradict the meta tag on the same page.
Do not pair a noindex header with a robots.txt block; a blocked page is never fetched, so the header is never read.
Review after CDN or hosting changes, which frequently reintroduce defaults.

Common mistakes

Debugging an indexing problem entirely in the page source, where a header directive is invisible.
Applying a rule to a directory that also contains pages you want indexed.
Leaving a staging-era header in place after launch, at the server rather than in the code.
Assuming a CDN passes headers through unchanged when it may add or strip them.
Setting a header and a meta tag that disagree, then being surprised by which one wins.

What this tool checks

The tool inspects the HTTP response headers for the page and reports whether an X-Robots-Tag noindex directive is present. Finding one is reported as a failure, on the basis that a page you are testing is a page you want indexed.

It examines the response only. A noindex written into the page HTML is a separate check, and the two are worth running together when a page will not appear in results.

Where to go next

Indexing problems usually come down to one of three instructions. Check the noindex meta tags in the page HTML, review your robots.txt for a crawl block that would stop the page being read at all, and confirm the page is listed in your XML sitemap.

Comprehensive SEO Toolbox with over 55 Tools

Check Titles, Meta Descriptions, Headings, Schema, Core Web Vitals, SSL and Robots.txt. Generate Meta Tags, Sitemaps and .htaccess Rules, minify CSS, JS and HTML, and draft copy with our AI Writing Tools - instantly and for free.