Search

All tools
SEO & servers

robots.txt and Sitemap Validator

Paste a robots.txt or sitemap XML file and check directives, URLs, and lastmod dates. Optionally fetch a public URL.

Examples

  • Allow Google, block all.
  • Missing User-agent.
  • Sitemap loc.

How it works

robots.txt is matched top-down by User-agent groups. Disallow and Allow only apply after a User-agent line. Sitemap lines should be absolute URLs. XML sitemaps need a urlset or sitemapindex, each loc must be a URL, and lastmod must be a valid date. Fetch only from public http(s) hosts; private and loopback addresses are blocked.

Related tools

Related reading

Your experience on this site will be improved by allowing cookies Cookie Policy