llms.txt Checker
Use the free llms.txt Checker to inspect file availability and a Markdown title heading, with clear limits on what the result proves.
Open llms.txt Checker →The public toolkit
Free AI search and crawler tools
Shared analysis, focused questions. Start with a complete readiness scan or investigate one part of your website.
AI Search Readiness Checker
Use the free AI Search Readiness Checker to review crawler policy, indexability, raw HTML and technical findings for up to five pages.
Open AI Search Readiness Checker →AI Crawler Checker
Use the free AI Crawler Checker to compare robots.txt rules for documented search, training and user-requested crawlers.
Open AI Crawler Checker →Robots.txt AI Checker
Use the Robots.txt AI Checker to inspect crawler groups, wildcard rules, Allow and Disallow matches, and sitemap declarations.
Open Robots.txt AI Checker →ChatGPT Search Readiness Checker
Use the ChatGPT Search Readiness Checker to review OAI-SearchBot policy, noindex directives, canonicals and raw content.
Open ChatGPT Search Readiness Checker →OAI-SearchBot Checker
Use the OAI-SearchBot Checker to find the matched robots.txt rule for OpenAI search discovery on a public URL.
Open OAI-SearchBot Checker →GPTBot Policy Checker
Use the GPTBot Policy Checker to review OpenAI training-crawler rules separately from your ChatGPT Search preferences.
Open GPTBot Policy Checker →Claude Crawler Checker
Use the Claude Crawler Checker to compare Claude-SearchBot, ClaudeBot and Claude-User robots.txt policies.
Open Claude Crawler Checker →Perplexity Crawler Checker
Use the Perplexity Crawler Checker to inspect PerplexityBot robots.txt policy and understand user-requested retrieval limits.
Open Perplexity Crawler Checker →Sitemap Health Checker
Use the Sitemap Health Checker to discover XML sitemaps and review bounded structure, host consistency and sample pages.
Open Sitemap Health Checker →Structured Data Checker
Use the Structured Data Checker to inspect JSON-LD syntax and declared schema types in public HTML, with evidence for malformed JSON.
Open Structured Data Checker →Agent Readiness Checker
Use the Agent Readiness Checker to inspect native HTML controls, labels and semantic signals before a rendered accessibility audit.
Open Agent Readiness Checker →AI Robots Policy Generator
Express search and training preferences using documented crawler tokens.
Build a robots snippet →llms.txt Generator
Create a convenience index for selected public documentation. No ranking promises.
Create a text index →