scrapegraph-mcp

A production-ready Model Context Protocol (MCP) server that provides seamless integration with the ScapeGraph AI API. This server enables language models to leverage advanced AI-powered web scraping capabilities with enterprise-grade reliability.

The server provides the following enterprise-ready tools:

• markdownify(website_url: str): Transform any webpage into clean, structured markdown format
• smartscraper(user_prompt: str, website_url: str): Leverage AI to extract structured data from any webpage
• searchscraper(user_prompt: str): Execute AI-powered web searches with structured, actionable results

To utilize this server, you'll need a ScapeGraph API key. Follow these steps to obtain one:

1. Navigate to the ScapeGraph Dashboard
2. Create an account and generate your API key

For automated installation of the ScrapeGraph API Integration Server using Smithery:

npx -y @smithery/cli install @ScrapeGraphAI/scrapegraph-mcp --client claude

Update your Claude Desktop configuration file with the following settings (located on the top rigth of the Cursor page):

(remember to add your API key inside the config)

{
    "mcpServers": {
        "@ScrapeGraphAI-scrapegraph-mcp": {
            "command": "npx",
            "args": [
                "-y",
                "@smithery/cli@latest",
                "run",
                "@ScrapeGraphAI/scrapegraph-mcp",
                "--config",
                "\"{\\\"scrapegraphApiKey\\\":\\\"YOUR-SGAI-API-KEY\\\"}\""
            ]
        }
    }
}

The configuration file is located at:

• Windows: %APPDATA%/Claude/claude_desktop_config.json
• macOS: ~/Library/Application\ Support/Claude/claude_desktop_config.json

Add the ScrapeGraphAI MCP server on the settings:

The server enables sophisticated queries such as:

• "Analyze and extract the main features of the ScapeGraph API"
• "Generate a structured markdown version of the ScapeGraph homepage"
• "Extract and analyze pricing information from the ScapeGraph website"
• "Research and summarize recent developments in AI-powered web scraping"
• "Create a comprehensive summary of the Python documentation website"

The server implements robust error handling with detailed, actionable error messages for:

• API authentication issues
• Malformed URL structures
• Network connectivity failures
• Rate limiting and quota management

When running on Windows systems, you may need to use the following command to connect to the MCP server:

C:\Windows\System32\cmd.exe /c npx -y @smithery/cli@latest run @ScrapeGraphAI/scrapegraph-mcp --config "{\"scrapegraphApiKey\":\"YOUR-SGAI-API-KEY\"}"

This ensures proper execution in the Windows environment.

This project is distributed under the MIT License. For detailed terms and conditions, please refer to the LICENSE file.

Special thanks to tomekkorbak for his implementation of oura-mcp-server, which served as starting point for this repo.

Made with ❤️ by ScrapeGraphAI Team