awesome-ai-web-scraping

A curated list of AI-powered web scraping tools, LLM-friendly crawlers, MCP servers, and infrastructure for turning the web into data.

h4ckf0r0day/awesome-ai-web-scraping

View on GitHub ↗

A curated list of AI-powered web scraping tools, LLM-friendly crawlers, MCP servers, and infrastructure for turning the web into data.

awesome-ai-web-scraping grew +16.5% in the last 30 days, ranks 362nd of 441 in Tool & Framework Collections, last commit 187 days ago.

Star trajectory

82 days tracked
2026-08-31134 stars today2026-11-22
Momentum

+16.5% in 30d, above the Tool & Framework Collections median of +3.2%.

Maintenance

Last commit 187 days ago, staler than the median of 16d.

How it compares in Tool & Framework Collections

awesome-ai-web-scraping
134
stars
+16.5% 30d
commit 187 days ago
Tool & Framework Collections median
337
stars
+3.2% 30d
commit ~16d ago
Leader · awesome-copilot
41.4k
stars
+2.7% 30d
commit 2 days ago

Ranked 362nd of 441 by stars.

Ranked by shared tags, then keywords, then stars
  1. An unofficial, categorized directory of 118 CoreClaw Worker APIs for web scraping, automation, lead…

    +1 (+0.6%) in 30d · Committed today · Python

    Shares Tool & Framework Collections

  2. Continuously updated catalogue of open-source OSINT tools, skills, plugins, MCP servers, and agentic…

    +28 (+34%) in 30d · Committed 5d ago · Python · MIT

    Shares Tool & Framework Collections

  3. A mega collection of 2,070 AI agent data APIs: MCP servers, LLM tools, RAG pipelines, live web data, vision &…

    Committed 33d ago · MIT

    Shares Tool & Framework Collections

  4. AI 编程福利导航:汇总 Claude Code、Codex、Cursor 中转站与公益站,整理免费额度、注册福利、签到活动和评测记录,提供可视化看板与 Python 数据采集工具。

    +391 (+177%) in 30d · Committed today · HTML

    Shares Tool & Framework Collections

  5. 🚀 A curated collection of AI engineering resources, roadmaps, tools, and projects covering LLMs, RAG, AI…

    +163 (+0.7%) in 30d · Committed 31d ago · MIT

    Shares Tool & Framework Collections

  6. Claude skills for marketing: 49 free marketing skills for Claude Code and claude.ai (Google Ads, Meta Ads…

    +3.1k (+59%) in 30d · Committed 25d ago

    Shares Tool & Framework Collections

  7. A collection of MCP clients.

    +32 (+0.5%) in 30d · Committed 167d ago · MIT

    Shares Tool & Framework Collections

  8. Independent directory of useful skills, plugins, memory providers, tools, surfaces, and guides for Nous…

    +176 (+3.0%) in 30d · Committed 25d ago · Other license

    Shares Tool & Framework Collections

  9. Awesome list for AI agent harness engineering: tools, patterns, evals, memory, MCP, permissions…

    +289 (+5.9%) in 30d · Committed today · Python · Other license

    Shares Tool & Framework Collections

  10. A collection of remote MCP servers.

    +2k (+114%) in 30d · Committed 5d ago · MIT

    Shares Tool & Framework Collections

Language
-
License
Creative Commons Zero v1.0 Universal
Contributors
1
Forks
73
Watchers
6
Open issues
1
Releases
0
Created
2026-05-18
Stars this week
+4

You've seen where awesome-ai-web-scraping stands today.

Get the moves before everyone else: the 5 Tool & Framework Collections repos climbing fastest, every Tuesday.

Free weekly digest. Double opt-in, unsubscribe anytime.

Maintain awesome-ai-web-scraping?

Claim this page to add your own call-to-action.

Claim this page