Meta Robots vs X-Robots-Tag Comparator

The same indexing instruction can arrive by two routes: the meta tag in the HTML and the HTTP header. Having both is common — one comes from the CMS, the other from the server or CDN — and that is exactly where the page nobody can explain the disappearance of comes from. Enter both values and the page says what actually applies.

The rule is simple and worth memorising: the most restrictive wins. There is no precedence by origin — the header does not beat the meta tag, nor the other way round. If one says index and the other says noindex, the result is noindex. The page applies that resolution separately for indexing and for link following, because the two directives are independent.

Worth remembering the defaults: with no directive at all, the behaviour is index and follow. Declaring those two words explicitly therefore changes nothing — it is noise taking up space. And there is a trap the comparison cannot catch: a page blocked in robots.txt is never fetched, so the noindex inside it is never read, and the URL can end up indexed with no content. To remove it from the index, crawling must be allowed.