Home › Search & Research › crawl-readiness-mcp
crawl-readiness-mcp
Check what AI crawlers may read and what a page permits, generate llms.txt and schema.
Topics: Search & Data Extraction
Connect
Review any command before running it. Package names and URLs come from the server's own registry entry.
Package (npm 0.2.9)
npx -y [email protected]
Or add to your MCP client config:
{
"mcpServers": {
"crawl-readiness-mcp": {
"command": "npx",
"args": [
"-y",
"[email protected]"
],
"env": {
"CRAWL_READINESS_API_KEY": "<YOUR_VALUE>"
}
}
}
}
CRAWL_READINESS_API_KEYsecret — Free API key from https://www.crawlreadiness.com/dashboard. The four audit tools and check_permission work without it; the three generators need it, and get_monitor_trend needs one on a plan with Answer Analyses.CRAWL_READINESS_API_BASE— Override the API base URL. Only useful for testing against a preview deployment; defaults to https://www.crawlreadiness.com.
Related servers
modelcontextprotocol/servers
Direct local file system access.
wigolo
Local-first web intelligence MCP server for AI coding agents
exa-labs/exa-mcp-server
Connect AI agents to Exa for web search, content fetching, and multi-step research.
blazickjp/arxiv-mcp-server
Search arXiv papers, download full text, semantic search, citation graphs, and alerts via MCP.
trieve
Crawl, embed, chunk, search and retrieve information from datasets through Trieve.
brightdata-mcp
Bright Data's Web MCP server enabling AI agents to search, extract & navigate the web
Listed in punkpeye/awesome-mcp-servers (MIT)
Data from the Official MCP Registry