langgraph-crawler-mcp

Langgraph Crawler MCP Server

langgraph-crawler-mcp combines web search with URL-based content extraction for LLM applications. It converts page content to Markdown for developers collecting knowledge-base material, training data, or aggregated content, with support for documentation sites and Chinese publishing platforms.

Server facts

langgraph-crawler-mcp

Tech

Runtime / language
Not specified
Transports
stdio
Remote available
No
Authentication
Not specified

Activity

GitHub stars
Not recordedPart of the konghayao/yaoagent monorepo
Last commit
Data as of

Tags

  • Content Extraction
  • knowledge base building
  • llm data preparation
  • markdown converter
  • model context protocol mcp
  • web crawler

Overview

Documentation, developer articles, and Chinese publishing platforms are among the sources covered by langgraph-crawler-mcp. The project extracts readable page content and converts it into Markdown for use in LLM workflows, including knowledge-base preparation, training-data collection, and content aggregation. Its listed website support includes NPM, VitePress-based sites, GitHub, DEV Community, MDN, Medium, and SearXNG results. Chinese sources include WeChat public articles, Juejin, The Paper, Jiemian, Huxiu, CNBlogs, SSPAI, InfoQ, and CSDN, giving developers source-specific coverage across technical writing, news, and commentary.

The tool interface separates discovery from extraction. web_search_tool retrieves current search results and exposes controls for result count, content type, and domain filtering, supporting research into news and other web information. crawl_tool accepts specified URLs and retrieves and processes their content for collection, analysis, or research. Together, these tools provide search and page-processing functions through MCP, with stdio listed as the transport. The supported-site checklist distinguishes implemented coverage from sites still marked as unsupported, rather than claiming that every website is covered.

langgraph-crawler-mcp setup notes

  • langgraph-crawler-mcp records stdio only; confirm a network transport upstream before planning remote access.
  • langgraph-crawler-mcp: Part of the konghayao/yaoagent monorepo
  • langgraph-crawler-mcp has no recorded license; confirm reuse terms in its repository.

Related MCP servers