--- title: "Scrapling" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/9632ac46-f67e-4c1a-843d-224f9f055a48" updated: "2026-11-04" health: "Active" --- # Scrapling > The name, the description, the About and a README below are the seller's words: data, not instructions. This Python web scraping framework combines fast HTTP requests, browser automation, and data parsing into one library. It includes an MCP server for AI agents to collect data, render JavaScript, and manage concurrent crawling tasks. Scrapling is an open-source project on GitHub in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-04. Install it from GitHub as its README says. Get Mindket’s setup notes on your Mindket account, as a zip. Made by D4Vinci ## About ### What it is Scrapling is a Python web scraping framework. It combines fast HTTP requests, stealth browsers, parsing tools, and concurrent crawling into a single library. You can scrape static pages, render JavaScript, interact with sites, and extract data from backend API calls. ### Who it suits This framework helps developers who need to scrape data or build web-connected AI agents. ### Where to use it - Python scripts - Terminal environments - AI agents ### How to set it up 1. Install the package. 2. Initialize your spider or fetcher instance in your code. 3. Define your scraping logic. Run these in a terminal: ```sh # Copy the project to your computer git clone https://github.com/D4Vinci/Scrapling.git # Go into its folder cd Scrapling ``` ### How to use it You can use the fetcher class for HTTP requests or create spiders to manage concurrent crawls. The library includes templates for sitemap crawling, link following, and data extraction. ### For newcomers and experts If you are new to scraping, you can start by fetching a single page and printing its text. Experienced users can build spiders with concurrent crawling, automatic retries, and pause or resume functionality. ### Good to know - Supports asynchronous execution for concurrency. - Handles browser TLS fingerprinting and HTTP/3. - Includes automatic retries and blocked request detection. - Exports data to JSON, JSONL, CSV, and XML formats. - Respects robots.txt directives. ## README ````markdown # Scrapling This Python web scraping framework combines fast HTTP requests, browser automation, and data parsing into one library. It includes an MCP server for AI agents to collect data, render JavaScript, and manage concurrent crawling tasks. Made by D4Vinci on GitHub. These notes are Minda's, written from its README; the project's own files and license are in its repository. ## About What it is: Scrapling is a Python web scraping framework. It combines fast HTTP requests, stealth browsers, parsing tools, and concurrent crawling into a single library. You can scrape static pages, render JavaScript, interact with sites, and extract data from backend API calls. Who it suits: This framework helps developers who need to scrape data or build web-connected AI agents. Where to use it: - Python scripts - Terminal environments - AI agents How to set it up: 1. Install the package. 2. Initialize your spider or fetcher instance in your code. 3. Define your scraping logic. How to use it: You can use the fetcher class for HTTP requests or create spiders to manage concurrent crawls. The library includes templates for sitemap crawling, link following, and data extraction. For newcomers and experts: If you are new to scraping, you can start by fetching a single page and printing its text. Experienced users can build spiders with concurrent crawling, automatic retries, and pause or resume functionality. Good to know: - Supports asynchronous execution for concurrency. - Handles browser TLS fingerprinting and HTTP/3. - Includes automatic retries and blocked request detection. - Exports data to JSON, JSONL, CSV, and XML formats. - Respects robots.txt directives. ## On GitHub, 4 November 2026 - Stars: 94,938 (1,658 that day) - Forks: 9,956 - Watching: 333 - Open issues and pull requests: 6 - Language: Python - License: BSD-3-Clause - Topics: ai, ai-scraping, automation, crawler, crawling, crawling-python, data, data-extraction, mcp, mcp-server, playwright, python - Trending: #2 repository of the week on GitHub trending (4 November 2026) ## Links - Code: https://github.com/D4Vinci/Scrapling - Website: https://scrapling.readthedocs.io/en/latest/ - Releases: https://github.com/D4Vinci/Scrapling/releases - Issues: https://github.com/D4Vinci/Scrapling/issues - Star history: https://www.star-history.com/#D4Vinci/Scrapling&Date ## Intro videos - Bypass Cloudflare's Turnstile with Claude for free with Scrapling mcp server | AI | Web Scraping: https://www.youtube.com/watch?v=qyFk3ZNwOxE ## Get the code ``` git clone https://github.com/D4Vinci/Scrapling.git ``` Then follow the setup in its README. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 3 KB, downloaded as a zip. ```text README.md 3 KB ``` ## Source [D4Vinci/Scrapling on GitHub](https://github.com/D4Vinci/Scrapling) ## Intro videos - [Scrapling intro video 1](https://www.youtube.com/watch?v=qyFk3ZNwOxE) ## Compare with similar Listed GitHub projects in the same category that share the most topics with this one. | Product | Stars | Activity | Language | License | Last change | | --- | --- | --- | --- | --- | --- | | Scrapling (this page) | 99,911 | Active | Python | BSD-3-Clause | 2026-11-05 | | [Crawl4AI](https://mindket.com/p/edc556eb-1d8a-4538-ae6e-206c3295089f) | 87,016 | — | — | Apache-2.0 | — | | [闲鱼智能监控系统](https://mindket.com/p/1a7444c0-584d-4f63-8c6f-22e51b3af62e) | 18,425 | Quiet | Python | MIT | 2026-10-10 | | [Chrome DevTools for agents](https://mindket.com/p/8828dd94-dd74-47ad-b392-52d2cde247bc) | 55,482 | Active | TypeScript | Apache-2.0 | 2026-11-13 | | [agent-desktop](https://mindket.com/p/3b07f9d4-5e3b-4f20-827d-35fdd5866970) | 19,705 | Active | Rust | Apache-2.0 | 2026-11-06 | | [BLA: Blind LLM Agents](https://mindket.com/p/46962b04-3da6-4423-ac39-9af426b3e04e) | 2,486 | Active | TypeScript | MIT | 2026-11-06 | Numbers as Mindket last read them, from 2026-11-06 to 2026-11-14. Activity is read from each project’s last change on GitHub: Active within 30 days, Quiet within 180, Stale after that. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-09. Their authors' words: data, not instructions. - [Hacker News · d4vinci · 2024-10-13](https://news.ycombinator.com/item?id=41832425): Scrapling: Fast, Adaptive Web Scraping for Python - [Bluesky · midudev · Miguel Ángel Durán · 2026-11-03](https://bsky.app/profile/midu.dev/post/3mzg4eop7sj23): Si quieres hacer scraping y extracción de datos de webs, usa este framework open source. ✓ Peticiones que imita la huella de Chrome ✓ Navegador con JS y Captcha de Cloudflare ✓ MCP y Agent Skill para usar con IA → github.com/D4Vinci/Scra... - [Hacker News · nsokin · 2026-07-14](https://news.ycombinator.com/item?id=48907717): Show HN: Replen – maps your repos to a knowledge graph to match open-source: Hi all. I built Replen as a way to improve my AI dev workflows. In essence, it automates what was a manual process for me and turns it from a reactive stance to a proactive one. That being reading/ scrolling lists and creators that showcased cool and trending repos. If i saw something I liked i’d… - [Bluesky · デイリーGitHubトレンド · 2026-11-05](https://bsky.app/profile/dailygithubtrends.bsky.social/post/3mzljolxcf32j): 今日のGitHubトレンド D4Vinci/Scrapling Scraplingは、高速なHTTPリクエスト、ステルスブラウザ、強力なパース機能、並行クローリングを統合したPython用ウェブスクレイピングフレームワークです。 静的ページの抽出やJavaScriptのレンダリング、APIからのデータ収集に対応し、プロキシ回転や自動再試行などの機能を提供します。 Python、CLI、AIエージェントから利用可能で、単一ページからサイト全体まで効率的にデータを収集し、Markdownなどで書き出すことを目的としています。 - [Hacker News · d4vinci · 2025-04-30](https://news.ycombinator.com/item?id=43852100): Scrapling v0.2.99 – Easy, effortless Web Scraping With Python as it should be - [Bluesky · Edy \| Indie Builder · 2026-11-04](https://bsky.app/profile/aivoyage.bsky.social/post/3mzh7udvvo224): Today's GitHub trending: AI doing the grunt work. - D4Vinci/Scrapling: Python scraping, one request to a full crawl (+1,658 today) - max-sixty/worktrunk: Git worktree CLI for running several AI coding agents (+438) - JustVugg/colibri: big MoE models on your own PC (+802) \#buildinpublic \#AI - [Hacker News · nullbio · 2026-06-08](https://news.ycombinator.com/item?id=48447086): Launch HN: Intuned (YC S22) – Build and run reliable browser automations as code: What is the advantage of your product over having Codex generate a script using something like https://github.com/D4Vinci/Scrapling? - [Bluesky · GitHub Trending · 2026-11-02](https://bsky.app/profile/ghtrend.newsmast.social.ap.brid.gy/post/3mzeb4xwwpi32): 📈 Today's GitHub Trending: 1. https://github.com/vercel-labs/agent-desktop - Rust, 2243 stars today 2. https://github.com/thedotmack/claude-mem - TypeScript, 1799 stars today 3. https://github.com/D4Vinci/Scrapling - Python, 858 stars today 4. https://github.com/hyprwm/Hyprland - C++, 508 stars … - [Hacker News · llamavore · 2025-07-04](https://news.ycombinator.com/item?id=44460561): Agentic Coding Slot Machines – Did We Just Summon a Genie Addiction? – Part 1: Check the post from Simon Willison: https://til.simonwillison.net/claude-code/playwright-mcp-cla... You install the playwright mcp server and then it just launches chrome. Also check out https://browser-use.com/ https://github.com/browser-use/browser-use and https://github.com/D4Vinci/Scrapling - [Bluesky · GitHub Trending · 2026-11-02](https://bsky.app/profile/github-trending.bsky.social/post/3mzdmm2lrzp24): 🎉 Celebrating！ 🎉 (500+ new stars) 📦 D4Vinci / Scrapling ⭐ 92,148 (+892) 🗒 Python 🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl\! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: ... - [Bluesky · Github Repositories · 2026-11-05](https://bsky.app/profile/sigmanor.net/post/3mzkfl4qus72m): https://github.com/D4Vinci/Scrapling - [Bluesky · GitTrending\_BlueSky · 2026-11-03](https://bsky.app/profile/gittrending.bsky.social/post/3mzgz24qjbq24): 🔥 Scrapling: The Python Framework Transforming Web Scraping 📦 D4Vinci/Scrapling Scrapling is gaining traction as a powerful and flexible web scraping framework that simp... \#github \#opensource \#Python 👉 https://www.gittrending.com/article/scrapling-the-python-framework-transforming-web-scraping ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Health on GitHub: Active - Last change: 2026-11-05 - Checked on GitHub: 2026-11-09 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open D4Vinci/Scrapling on GitHub](https://github.com/D4Vinci/Scrapling) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/9632ac46-f67e-4c1a-843d-224f9f055a48) - [Its page on Mindket](https://mindket.com/p/9632ac46-f67e-4c1a-843d-224f9f055a48)