Happy slice of mcp pizzaMCP.pizza
crawlie

crawlie

MCP.Pizza Chef: spronta

Crawlie lets you scan any website to find broken links, redirects, missing metadata, and over 40 SEO and accessibility issues. It provides easy-to-understand advice on how to fix problems and scores your site’s technical health, AI search readiness, and accessibility. You can run it locally or use it within apps like Claude Desktop and ChatGPT without needing complicated setup or API keys. It also offers a hosted cloud option for instant use.

Other
Web/Research

Use This MCP server To

Scan my website for broken links and redirects Check my site’s SEO health and get fix suggestions Audit pages for missing metadata and accessibility issues Get AI-search readiness scores for my site Save and share detailed crawl reports Run site audits directly from Claude or ChatGPT Use hosted cloud crawling without local install

README

crawlie

The fast, free, open-source technical SEO + GEO crawler — built for humans and agents.

Crawl any site for broken links, redirects, missing metadata, and 40+ SEO & Generative-Engine checks — with plain-English guidance on every fix. Runs locally, ships a CLI and an MCP server, and costs nothing.

npm CI License: MIT

Setup · CLI · MCP & agents · Changelog · Use cases · Why I built this · Desktop app · Checks · Compare · Architecture

Read the docs → crawlie.dev

💡Share feedback & suggestions here

Showcase of the example app demonstrating a report

Setup

The easy way — npm (installs the crawlie CLI and the crawlie-mcp server):

npm i -g crawlie

The macOS app — grab the signed .dmg from Releases.

From source — needs Rust (engine/CLI/MCP) and, for the desktop app, pnpm + Node:

git clone https://github.com/spronta/crawlie
cd crawlie
cargo build --release
# → target/release/crawlie  and  target/release/crawlie-mcp

# or install onto your PATH:
cargo install --path crates/crawlie-cli      # installs `crawlie`
cargo install --path crates/crawlie-mcp      # installs `crawlie-mcp`

How it ships: the CLI + MCP come only through npm — the right native binary installs automatically as a platform package (nothing to download or unblock). The desktop app is the only direct download: a signed, notarized .dmg on Releases.

crawlie FAQ

Can I use this to find broken links and SEO issues on my website?
Yes — crawlie scans sites for broken links, redirects, missing metadata, and over 40 SEO checks with clear guidance on fixes.
Can I use this to audit my site’s AI search readiness and accessibility?
Yes — it provides GEO scores for AI search and Accessibility scores for WCAG compliance.
Which apps can I use crawlie with?
It works with Claude Desktop, ChatGPT, Cursor, Cline, and any MCP-compatible client.
Do I need an API key to use crawlie locally?
No — the local MCP server runs without any API key or account.
Is setting up crawlie difficult?
Setup is easy — just paste a small config snippet or install via npm globally.
Can I run crawlie without installing anything?
Yes — use the hosted Crawlie Cloud MCP server with an API key for cloud-based crawling.