scraping AI Agent Skills

Browse 218 skills related to scraping

tmux

Remote-control tmux sessions for interactive CLIs by sending keystrokes and scraping pane output.

tmux

Remote-control tmux sessions for interactive CLIs by sending keystrokes and scraping pane output.

browser-automation

Browser automation powers web testing, scraping, and AI agent interactions. The difference between a flaky script and a reliable system comes down to understanding selectors, waiting strategies, and anti-detection patterns. This skill covers Playwright (recommended) and Puppeteer, with patterns for testing, scraping, and agentic browser control. Key insight: Playwright won the framework war. Unless you need Puppeteer's stealth ecosystem or are Chrome-only, Playwright is the better choice in 202

firecrawl-scraper

Deep web scraping, screenshots, PDF parsing, and website crawling using Firecrawl API

Cloudflare Browser

Control headless Chrome via Cloudflare Browser Rendering CDP WebSocket. Use for screenshots, page navigation, scraping, and video capture when browser automation is needed in a Cloudflare Workers environment. Requires CDP_SECRET env var and cdpUrl configured in browser.profiles.

tmux

Remote-control tmux sessions for interactive CLIs by sending keystrokes and scraping pane output.

zellij

Openclaw Skills Zellij

Remote-control zellij sessions for interactive CLIs by sending keystrokes and scraping pane output.

chrome-devtools

Browser automation, debugging, and performance analysis using Puppeteer CLI scripts. Use for automating browsers, taking screenshots, analyzing performance, monitoring network traffic, web scraping, form automation, and JavaScript debugging.

reddit

Read and search Reddit posts via web scraping of old.reddit.com. Use when Clawdbot needs to browse Reddit content - read posts from subreddits, search for topics, monitor specific communities. Read-only access with no posting or comments.

brightdata

Web scraping and search via Bright Data API. Requires BRIGHTDATA_API_KEY and BRIGHTDATA_UNLOCKER_ZONE. Use for scraping any webpage as markdown (bypassing bot detection/CAPTCHA) or searching Google with structured results.

sponge-wallet

Manages crypto wallets, transfers tokens, swaps on DEXes, checks balances, and accesses paid APIs (search, image gen, prediction markets, web scraping, document parsing, sales prospecting) via x402 micropayments. Use when the user asks about wallet balances, token transfers, swaps, blockchain payments, or paid API services.

asustor-pro-adaptive-suite

A hardware-aware, hybrid (SMB + SSH) suite for ASUSTOR NAS metadata scraping. Functions as a versatile Coder, Project Manager, and System Architect while maintaining strict read-only safety and i3-10th Gen resource throttling.

serper

Google search via Serper API with full page content extraction. Fast API lookup + concurrent page scraping (3s timeout). One well-crafted query returns rich results — avoid multiple calls. Two modes, explicit locale control. API key via .env.

vpn-rotate-skill

Bypass API rate limits by rotating VPN servers. Works with any OpenVPN-compatible VPN (ProtonVPN, NordVPN, Mullvad, etc.). Automatically rotates to new server every N requests for fresh IPs. Use for high-volume scraping, government APIs, geo-restricted data.

apify-lead-generation

Generates B2B/B2C leads by scraping Google Maps, websites, Instagram, TikTok, Facebook, LinkedIn, YouTube, and Google Search. Use when user asks to find leads, prospects, businesses, build lead lists, enrich contacts, or scrape profiles for sales outreach.

browser-cash

Spin up unblocked browser sessions via Browser.cash for web automation. Sessions bypass anti-bot protections (Cloudflare, DataDome, etc.) making them ideal for scraping and automation.

read-github

Read GitHub repos the RIGHT way - via gitmcp.io instead of raw scraping. Why this beats web search: (1) Semantic search across docs, not just keyword matching, (2) Smart code navigation with accurate file structure - zero hallucinations on repo layout, (3) Proper markdown output optimized for LLMs, not raw HTML/JSON garbage, (4) Aggregates README + /docs + code in one clean interface, (5) Respects rate limits and robots.txt. Stop pasting raw GitHub URLs - use this instead.

irish-takeaway

Find nearby takeaways in Ireland and browse menus via Deliveroo/Just Eat. Uses Google Places API for discovery and browser automation for menu scraping.

phantombuster

Control PhantomBuster automation agents via API. List agents, launch automations, get output/results, check status, and abort running agents. Use when the user needs to run LinkedIn scraping, Twitter automation, lead generation phantoms, or any PhantomBuster workflow.

tmux

Remote control tmux sessions for interactive CLIs (python, gdb, etc.) by sending keystrokes and scraping pane output.

arxiv-viewer

View, search, and download academic papers from arXiv. Supports API queries, web scraping via Actionbook, and HTML paper reading via ar5iv. Use when user asks about arxiv papers, academic papers, research papers, paper summaries, latest papers, or wants to search/download/read papers.

openakita/skills@apify-scraper

Web data extraction using 55+ verified Apify Actors for AI-driven scraping. Supports Instagram, Facebook, TikTok, YouTube, Google (SERP, Maps, Reviews) and more. Automatically selects the best Actor for each task and returns structured JSON/CSV output. Features include pagination and incremental crawling, rate limiting and retry strategies, proxy and authentication support (requires APIFY_TOKEN), scheduling and page-change monitoring, and guidance for ethical scraping and ban mitigation.

apify-actor-development

Develop, debug, and deploy Apify Actors - serverless cloud programs for web scraping, automation, and data processing. Use when creating new Actors, modifying existing ones, or troubleshooting Actor code.

cloudflare-browser-rendering

Add headless Chrome automation with Puppeteer/Playwright on Cloudflare Workers. Use when: taking screenshots, generating PDFs, web scraping, crawling sites, browser automation, or troubleshooting XPath errors, browser timeouts, binding not passed errors, session limits, page.evaluate __name errors, or waitForSelector timeout issues.

firecrawl-scraper

Convert websites into LLM-ready data with Firecrawl API. Features: scrape, crawl, map, search, extract, agent (autonomous), batch operations, and change tracking. Handles JavaScript, anti-bot bypass, PDF/DOCX parsing, and branding extraction. Prevents 10 documented errors. Use when: scraping websites, crawling sites, web search + scrape, autonomous data gathering, monitoring content changes, extracting brand/design systems, or troubleshooting content not loading, JavaScript rendering, bot detection, v2 migration, job status errors, DNS resolution, or stealth mode pricing.

playwright-local

Build browser automation and web scraping with Playwright on your local machine. Prevents 10 documented errors including CI timeout hangs, extension testing failures, and Ubuntu compatibility issues. Includes stealth mode for anti-bot bypass, authenticated sessions, infinite scroll handling, screenshot/PDF generation, and v1.57 Speedboard performance analysis. Use when: automating browsers, scraping protected sites, testing with real IPs, bypassing bot detection, generating screenshots/PDFs, or troubleshooting "target closed", "page.pause() hangs CI", "permission prompts block tests", or "Ubuntu 25.10 installation" errors.

browser

Automate web browser interactions using natural language via CLI commands. Use when the user asks to browse websites, navigate web pages, extract data from websites, take screenshots, fill forms, click buttons, or interact with web applications. Supports remote Browserbase sessions with automatic CAPTCHA solving, anti-bot stealth mode, and residential proxies — ideal for scraping protected websites, bypassing bot detection, and interacting with JavaScript-heavy pages.

browser

Iamzhihuix Happy Claude Skills Browser

Minimal Chrome DevTools Protocol tools for browser automation and scraping. Use when you need to start Chrome, navigate pages, execute JavaScript, take screenshots, or interactively pick DOM elements. Triggers include "browse website", "scrape page", "take screenshot", "automate browser", "extract DOM", "web scraping".

tmux

Remote-control tmux sessions for interactive CLIs by sending keystrokes and scraping pane output.

firecrawl-scraper

Complete knowledge domain for Firecrawl v2 API - web scraping and crawling that converts websites into LLM-ready markdown or structured data. Use when: scraping websites, crawling entire sites, extracting web content, converting HTML to markdown, building web scrapers, handling dynamic JavaScript content, bypassing anti-bot protection, extracting structured data from web pages, or when encountering "content not loading", "JavaScript rendering issues", or "blocked by bot detection". Keywords: firecrawl, firecrawl api, web scraping, web crawler, scrape website, crawl website, extract content, html to markdown, site crawler, content extraction, web automation, firecrawl-py, firecrawl-js, llm ready data, structured data extraction, bot bypass, javascript rendering, scraping api, crawling api, map urls, batch scraping

browser-automation

Non-testing browser automation - web scraping, form filling, screenshot capture, PDF generation, workflow automation. For TESTING with Playwright, use e2e-playwright skill instead. Activates for web scraping, form automation, screenshot, PDF, headless browser, Puppeteer, Selenium, automation scripts, data extraction.

Master Orchestrator

Fully automated Master Orchestrator: an end-to-end capability that links trend scraping, content generation, and viral-hit verification.

enact-firecrawl

Scrape, crawl, search, and extract structured data from websites using Firecrawl API - converts web pages to LLM-ready markdown

web-scrapingcrawlingmarkdown+5

twscrape

Python library for scraping Twitter/X data using GraphQL API with account rotation and session management. Use when extracting tweets, user profiles, followers, trends, or building social media monitoring tools.

browser

Control a Chromium browser via CDP. Use when automating web browsing, scraping pages, testing web apps, monitoring network traffic, or interacting with page elements.

browsing-with-playwright

Browser automation using Playwright MCP. Navigate websites, fill forms, click elements, take screenshots, and extract data. Use for web browsing, form submission, web scraping, or UI testing. NOT for static content (use curl/wget).

browser

Minimal Chrome DevTools Protocol tools for browser automation and scraping. Use when you need to start Chrome, navigate pages, execute JavaScript, take screenshots, or interactively pick DOM elements.

brightdata

Progressive four-tier URL content scraping with automatic fallback strategy. USE WHEN user says "scrape this URL", "fetch this page", "get content from", "can't access this site", "use Bright Data", "pull content from URL", or needs to retrieve web content that may have bot detection or access restrictions.

firecrawl

Official Firecrawl CLI skill for web scraping, search, crawling, and browser automation. Returns clean LLM-optimized markdown. USE FOR: - Web search and research - Scraping pages, docs, and articles - Site mapping and bulk content extraction - Browser automation for interactive pages Must be pre-installed and authenticated. See rules/install.md for setup, rules/security.md for output handling.

chrome-devtools

Browser automation, debugging, and performance analysis using Puppeteer CLI scripts. Use for automating browsers, taking screenshots, analyzing performance, monitoring network traffic, web scraping, form automation, and JavaScript debugging.

cloudflare-browser-rendering

Guide for implementing Cloudflare Browser Rendering - a headless browser automation API for screenshots, PDFs, web scraping, and testing. Use when automating browsers, taking screenshots, generating PDFs, scraping dynamic content, extracting structured data, or testing web applications. Supports REST API, Workers Bindings (Puppeteer/Playwright), MCP servers, and AI-powered automation. (project)

using-webctl

Browser automation via webctl CLI in Claude.ai containers with authenticated proxy support. Use when users mention webctl, browser automation, Playwright browsing, web scraping, or headless Chrome in container environments.

BrightData

Progressive URL scraping. USE WHEN Bright Data, scrape URL, web scraping tiers. SkillSearch('brightdata') for docs.

Apify

Social media scraping, business data, e-commerce via Apify actors. USE WHEN Twitter, Instagram, LinkedIn, TikTok, YouTube, Facebook, Google Maps, Amazon scraping.

web-research-workflow

Unified decision tree for web research and competitive monitoring. Auto-selects WebFetch, Tavily, or agent-browser based on target site characteristics and available API keys. Includes competitor page tracking, snapshot diffing, and change alerting. Use when researching web content, scraping, extracting raw markdown, capturing documentation, or monitoring competitor changes.

researchbrowserwebfetch+6

browser-tools

OrchestKit orchestration wrapper for browser automation. Adds security rules, rate limiting, and ethical scraping guardrails on top of the upstream agent-browser skill. Use when automating browser workflows, capturing web content, or extracting structured data from web pages.

browserautomationplaywright+3

prometheus-monitoring

Set up Prometheus monitoring for applications with custom metrics, scraping configurations, and service discovery. Use when implementing time-series metrics collection, monitoring applications, or building observability infrastructure.

playwright-automation

Automate browser testing, web scraping, and user workflow testing with Playwright across multiple browsers. Use when writing end-to-end tests for web applications, automating repetitive browser tasks, scraping data from websites, testing across Chrome/Firefox/Safari, taking screenshots for visual regression testing, testing authentication flows, filling and submitting forms programmatically, testing responsive designs across viewports, or any browser automation requiring reliable, cross-browser testing capabilities.

Join Clawiskill today. The decentralized skill network for AI Agents.

Explore Network