README.md
Explore / Scrapling
D4Vinci
Scrapling
Adaptive Python scraping framework: selectors that relearn when sites redesign, stealth fetchers that pass Cloudflare, spiders with proxy rotation and an MCP server — request to full crawl.
69,727 6,905 Python BSD-3-Clauseupdated yesterday
Curator's take
The modern-web answer to scraping's two chronic pains: selectors break (its parser relearns elements after redesigns via auto-save/auto-match) and bots get blocked (StealthyFetcher passes Cloudflare Turnstile out of the box). The MCP server is a quiet killer feature — your coding agent can scrape through it directly. NOT the veteran choice: younger ecosystem than Scrapy with fewer third-party answers when you're deep in the weeds, and the adaptive magic needs its cache warmed — first-run breakage still lands on you.
Mapped by ShipWithAI editors · links verified
Continue your stack
What teams reach for next — and why each earns a place beside Scrapling. Ranked by curator confidence.
Alternative to See all + compare →
scrapy
The veteran Python web crawling framework: spiders, middlewares, pipelines and battle-tested scheduling — 60k+ stars and still the reference architecture for structured scraping.
Why it fitsBoth are Python scraping frameworks; Scrapling rebuilds the stack for the hostile modern web — adaptive selectors that survive redesigns, Cloudflare-passing stealth fetchers — where Scrapy relies on its ecosystem for both.
crawlee
Apify's web scraping and browser automation library for Node.js/TypeScript — HTTP and headless-browser crawlers with human-like anti-blocking defaults, queues, storage and proxies.
Why it fitsThe two modern anti-bot-aware crawling frameworks, split by language: Scrapling for Python (adaptive parsing, MCP server), Crawlee for Node/TypeScript (browser switching, Apify ecosystem).
autoscraper
Learn-by-example Python scraper: give it a URL and sample values you want, it infers the extraction rules and reapplies them to similar pages. Tiny, fast, zero selectors.
Why it fitsBoth attack selector fragility by learning: autoscraper infers extraction rules from one example and stops there; Scrapling relearns elements after redesigns inside a full, maintained crawling framework.