Fetch and extract content from web pages, converting HTML to clean markdown. Use when users want to read web articles, extract information from URLs, scrape web content, or when the built-in WebFetch tool fails due to network restrictions. Trigger when user provides URLs to read, asks to fetch web content, or needs to extract text from websites.
Fetch web page content and convert to clean markdown format.
Run the fetch script to get web content:
python3 scripts/fetch_url.py <url> [options]
--timeout <seconds>: Request timeout (default: 30)--max-length <chars>: Maximum output length (default: 100000)--raw: Output raw HTML instead of markdownFetch single URL:
python3 scripts/fetch_url.py "https://example.com/article"
Fetch with custom timeout:
python3 scripts/fetch_url.py "https://example.com/article" --timeout 60
Fetch multiple URLs in parallel:
for url in "https://url1.com" "https://url2.com"; do
python3 scripts/fetch_url.py "$url" &
done
wait
fetch_url.py with the URLThe script converts HTML to clean markdown:
#, ##, ###, etc.- for unordered, 1. for ordered**bold**, *italic*403 Forbidden: Website blocks automated requests. Consider:
Timeout errors: Increase timeout with --timeout 60
Empty content: Website may require JavaScript to render content
Edit PDFs with natural-language instructions using the nano-pdf CLI.
Control Sonos speakers (discover/status/play/volume/group).
Terminal Spotify playback/search via spogo (preferred) or spotify_player.
Capture frames or clips from RTSP/ONVIF cameras.
CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).
Monitor blogs and RSS/Atom feeds for updates using the blogwatcher CLI.
Category:tools