crawlie

Fast, free, open-source technical SEO + GEO crawler: built for humans and agents.

spronta/crawlie

View on GitHub ↗

Fast, free, open-source technical SEO + GEO crawler: built for humans and agents.

crawlie grew +14.8% in the last 30 days, ranks 3951st of 4,996 in Tools & Integrations, last commit 26 days ago.

Star trajectory

82 days tracked
2026-08-31132 stars today2026-11-22
Momentum

+14.8% in 30d, above the Tools & Integrations median of +1.8%.

Maintenance

Last commit 26 days ago, staler than the median of 14d.

How it compares in Tools & Integrations

crawlie
132
stars
+14.8% 30d
commit 26 days ago
Tools & Integrations median
281
stars
+1.8% 30d
commit ~14d ago
Leader · rea
473.2k
stars
+28.6% 30d
commit today

Ranked 3951st of 4,996 by stars.

Ranked by shared tags, then keywords, then stars
  1. Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.

    +216 (+23%) in 30d · Committed yesterday · Rust · AGPL-3.0

    Shares Tools & Integrations, Protocols (MCP / A2A)

  2. Web research for Claude over MCP: quality-scored Google search, clean extraction, deep research. Hosted…

    +4 (+1.6%) in 30d · Committed 50d ago

    Shares Protocols (MCP / A2A), Tools & Integrations

  3. The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no…

    +56 (+1.0%) in 30d · Committed today · TypeScript · Other license

    Shares Tools & Integrations, Protocols (MCP / A2A)

  4. Self-hosted web search for AI agents and SEO. Google, Bing, Yandex, Baidu, DuckDuckGo & Ecosia, rendered in a…

    Committed 2d ago · Go · MIT

    Shares Protocols (MCP / A2A), Tools & Integrations

  5. 動漫花園 镜像站 | 动画 BT 资源聚合站 | 动画 BT 资源开放接口

    Committed today · TypeScript · AGPL-3.0

    Shares Tools & Integrations, Protocols (MCP / A2A)

  6. Self-hosted search gateway for AI agents. Unified search, news and page extraction with provider fallback…

    +4 (+0.3%) in 30d · Committed 5d ago · TypeScript · MIT

    Shares Tools & Integrations, Protocols (MCP / A2A)

  7. EveryInfra MCP server & agent plugin: scraper API for TikTok, Instagram, Amazon, YouTube, Xiaohongshu &…

    Committed today · Python · Apache-2.0

    Shares Protocols (MCP / A2A), Tools & Integrations

  8. Rust MCP server for Grok web search and Tavily-backed source retrieval

    -6 (-1.3%) in 30d · Committed 73d ago · Rust · MIT

    Shares Tools & Integrations, Protocols (MCP / A2A)

  9. Kindly Web Search MCP Server: Web search + robust content retrieval for AI coding tools (Claude Code, Codex…

    +5 (+1.2%) in 30d · Committed 33d ago · Python · MIT

    Shares Protocols (MCP / A2A), Tools & Integrations

  10. Lightweight scraper for Google News

    +0 (+0.0%) in 30d · Committed 11d ago · JavaScript · MIT

    Shares Tools & Integrations, Protocols (MCP / A2A)

Language
Rust
License
Other
Contributors
1
Forks
17
Watchers
2
Open issues
3
Releases
10
Created
2026-06-18
Stars this week
+5

You've seen where crawlie stands today.

Get the moves before everyone else: the 5 Tools & Integrations repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain crawlie?

Claim this page to add your own call-to-action.

Claim this page