AI Agent
Readiness Hub
AI agents are crawling websites to answer questions, book appointments, and make decisions. These are the signals that determine if your site gets found — or ignored.
All Readiness Signals
Organized into 5 categories. Each check explains why it matters and how to fix it.
Discoverability
Can AI agents find and crawl your site?
Robots.txt exists
Your robots.txt file is accessible and valid, with correctly structured allow/disallow rules.
AI bot rules in robots.txt
Explicit rules for GPTBot, ClaudeBot, Googlebot and other AI crawlers are present.
XML Sitemap
A valid sitemap.xml is linked in robots.txt or at /sitemap.xml, helping agents index your site.
LLMs.txt
Machine-readable content guidance for AI systems.
LLMs.txt present
A /llms.txt file exists at your domain root, providing AI systems with structured information about your content.
Correct format
The LLMs.txt follows the specification with required sections: name, description, and at least one URL.
LLMs-full.txt
An optional but recommended extended file with full content for LLM indexing is present.
MCP Protocol
Model Context Protocol — the standard for agent-to-tool communication.
MCP Server Card
A well-formed /.well-known/mcp.json is present and returns a valid MCP server card.
OAuth Discovery
OAuth 2.0 authorization server metadata is discoverable at the standard .well-known endpoint.
API Catalog
An API catalog or OpenAPI spec is accessible, helping agents understand what your site can do.
Agent Skills
Declare what capabilities your site exposes to AI agents.
Skills manifest
An agent-skills.json (or link header) declares what AI skills your site provides in the agentskills.io format.
Structured data
Schema.org markup (Organization, WebSite, Product, FAQ, etc.) is present and valid on key pages.
WebMCP endpoint
A WebMCP interface allows agents to interactively query or perform tasks on your site.
Content Access
Headers and signals that control how AI reads your content.
Markdown negotiation
Your server responds to Accept: text/markdown headers with clean Markdown — reducing token waste for LLMs.
Content Signals header
The CF-Content-Signals response header is present, indicating content classification via Cloudflare.
Web Bot Auth
Cryptographic bot authentication is supported, allowing legitimate AI crawlers to prove their identity.
How does your site score?
Run a free audit and see exactly which AI readiness checks your site passes — and what to fix first.
Scan My Site Free →