Read Website Fast MCP Server

Page reading and site crawling share the documented read_website_fast workflow: supply an HTTP or HTTPS URL, then optionally set crawl depth and robots.txt handling. Mozilla Readability extracts content, and Turndown turns the HTML into Markdown with GFM support. The conversion keeps links available for knowledge graph workflows, while optional chunking supports later processing. For LLM pipelines, the aim is to remove page noise and reduce the token footprint of the content being read.

Local crawls can use concurrent fetching and rate limiting, backed by disk caching keyed to hashed URLs. Cache resources let clients inspect statistics or clear stored content. The command-line interface offers Markdown output, JSON with metadata, or a URL alongside its Markdown, plus settings for concurrency, request timeouts, and cross-origin crawling. A streaming approach is intended to limit memory use. The server communicates over stdio, with MCP client support documented for Claude Desktop, Cursor, VS Code, and JetBrains IDEs.