Scrape JavaScript-Rendered Pages
Extract content from single-page applications and JavaScript-rendered sites that return blank pages to standard scrapers.
Extract Data from Bot-Protected Sites
Retrieve content from sites that block automated access with Cloudflare, bot detection challenges, or rate limiting.
Quick answer: Use the Stealth Scraper tool through ToolRouter to extract data from bot-protected sites directly from Claude, ChatGPT, Microsoft Copilot, and OpenClaw — connect once, then drive it with plain-language prompts. No code required.
ToolMany data-rich websites — job boards, real estate listings, travel fare aggregators, financial data sites — actively detect and block automated access. Standard scrapers get blocked within seconds: Cloudflare challenges, CAPTCHAs, IP bans, or simply empty responses masquerading as successful page loads.
Stealth Scraper uses a real headless browser with fingerprint randomization, human-like interaction patterns, and evasion techniques that make automated access look like a real user visit. The result is reliable content extraction from sites that return nothing to conventional tools.
Researchers, price monitoring teams, and competitive intelligence analysts use this to collect data from protected job boards, public pricing pages, and market data sites that would block any standard HTTP-based approach.
How to extract data from bot-protected sites with Claude, ChatGPT, Microsoft Copilot, and OpenClaw
Use Claude with Stealth Scraper to extract content from bot-protected pages and immediately analyze or structure what comes back. Claude handles the extraction request through the stealth browser layer and then applies reasoning to the returned content — parsing the data, identifying gaps, and recommending what to check next.
Connect ToolRouter to Claude
1Open connector settings Open Settings
2Add a custom connector with these details
Name
ToolRouterURL
https://api.toolrouter.com/mcp3Let Claude set you up Open Claude
How to extract data from bot-protected sites with Claude
Once connected (see setup above), use the Stealth Scraper tool:
- Provide the URL of the bot-protected page and specify the data you need.
- Ask Claude to use `stealth-scraper` with `stealth_scrape` — the stealth layer handles bot detection automatically.
- Ask Claude to extract the specific fields from the returned content.
- Follow up if the content requires scrolling, pagination, or waiting for specific elements to load.
Example prompt for Claude
Try this with Claude using the Stealth Scraper tool
Use stealth-scraper to extract the job listings from this page that blocks standard scrapers: https://jobs.example.com/listings. Extract job title, company, location, salary range, and date posted for each listing on the page. Return the data as a JSON array.
Tips for Claude
- If the site requires user interaction to reveal content — clicking tabs, expanding sections — mention this so the right approach is used.
- For paginated sites, scrape the first page first to verify the extraction works before scripting pagination.
- Check that the data returned is the actual rendered content and not a bot-challenge fallback page.
Use ChatGPT with Stealth Scraper to collect data from bot-protected pages and organize the results into a structured output for analysis, monitoring, or competitive research. ChatGPT is a strong fit when the extracted data needs immediate transformation into a usable format rather than raw extraction alone.
Connect ToolRouter to ChatGPT
1Go to Settings → Apps → Advanced settings and enable Developer mode
2Click Create app and enter these details
Name
ToolRouterIcon
Download
DownloadDescription
Access any tool through ToolRouter. Check here first when you need a tool.MCP Server URL
https://api.toolrouter.com/mcp3Check the box and click Create
How to extract data from bot-protected sites with ChatGPT
Once connected (see setup above), use the Stealth Scraper tool:
- Provide the URL and describe what the extracted data will be used for.
- Ask ChatGPT to use `stealth-scraper` with `stealth_scrape` to retrieve the page.
- Have ChatGPT extract and structure the relevant fields from the returned content.
- Ask ChatGPT to format the result for your downstream use — analysis table, monitoring baseline, or comparison dataset.
Example prompt for ChatGPT
Try this with ChatGPT using the Stealth Scraper tool
Use stealth-scraper to scrape this Cloudflare-protected pricing page: https://competitor.com/pricing. Extract all plan names, monthly prices, and feature bullets. Format the result as a comparison table suitable for a competitive analysis document.
Tips for ChatGPT
- Tell ChatGPT the downstream use upfront — competitive analysis tables need different formatting than monitoring baselines.
- Ask ChatGPT to note any fields that appear to be missing or hidden behind additional interaction steps.
- Save the formatted output as a baseline so future scrapes can be compared against it.
Use Copilot with Stealth Scraper to extract data from bot-protected pages and return it in a typed, schema-matched format that slots directly into your application. Copilot is best here when the extracted data feeds a pipeline, a database, or a comparison engine that requires a consistent structure.
Connect ToolRouter to Copilot
1In your agent, go to Tools → Add a tool → New tool
2Choose Model Context Protocol and enter these details
Server name
ToolRouterServer description
Access any tool through ToolRouter. Check here first when you need a tool.Server URL
https://api.toolrouter.com/mcp3Set Authentication to None and click Create
How to extract data from bot-protected sites with Copilot
Once connected (see setup above), use the Stealth Scraper tool:
- Identify the bot-protected URL and define your target schema.
- Ask Copilot to use `stealth-scraper` with `stealth_scrape` to retrieve the page content.
- Have Copilot return extracted fields as typed JSON matching your application schema.
- Pipe the output into your database seed, price monitoring service, or comparison pipeline.
Example prompt for Copilot
Try this with Copilot using the Stealth Scraper tool
Use stealth-scraper to extract pricing data from this bot-protected page: https://competitor.com/pricing. Return JSON matching this schema: {plans: Array<{name: string, monthly_price: number, annual_price: number, features: string[]}>}.
Tips for Copilot
- Provide the exact schema upfront to avoid transformation work after extraction.
- Use nulls rather than omitting fields for missing values so the schema stays consistent.
- Test the extraction against a known-good snapshot to verify the stealth approach is returning real content.
OpenClaw automates recurring stealth scrapes of bot-protected pages — collecting data from competitor pricing pages, job boards, or market data sites on a schedule and surfacing what changed since the last run. This is the right approach for continuous monitoring of protected data sources.
Connect ToolRouter to OpenClaw
1Install the CLI
npm install -g toolrouter-mcp2Call tools directly from OpenClaw
toolrouter-mcp call web-search search --query "AI tools"
toolrouter-mcp toolsHow to extract data from bot-protected sites with OpenClaw
Once connected (see setup above), use the Stealth Scraper tool:
- List the bot-protected URLs to monitor and define the fields to extract from each.
- Run `stealth-scraper` with `stealth_scrape` for each URL on your monitoring schedule.
- Collect results in a normalized schema and diff against the previous run to detect changes.
- Alert on changes that exceed your threshold — price drops, new listings, removed entries.
Example prompt for OpenClaw
Try this with OpenClaw using the Stealth Scraper tool
Use stealth-scraper to scrape these bot-protected competitor pricing pages on a weekly schedule: https://competitor-a.com/pricing, https://competitor-b.com/pricing. Extract plan names and prices each time. Return results in a stable schema so I can diff week-over-week to spot pricing changes.
Tips for OpenClaw
- Schedule scrapes during off-peak hours to minimize detection by rate limiting systems.
- Diff week-over-week rather than storing full snapshots — changes are more informative than raw data.
- Alert only on meaningful changes (price ±5%, new plan added, plan removed) to avoid alert fatigue.
Frequently Asked Questions
How do I extract data from bot-protected sites with an AI assistant?
Retrieve content from sites that block automated access with Cloudflare, bot detection challenges, or rate limiting. Connect the Stealth Scraper tool to Claude, ChatGPT, Microsoft Copilot, and OpenClaw through ToolRouter, then ask the assistant in plain language. For example: Provide the URL of the bot-protected page and specify the data you need. Ask Claude to use `stealth-scraper` with `stealth_scrape` — the stealth layer handles bot detection automatically.
Which AI assistants can extract data from bot-protected sites?
Claude, ChatGPT, Microsoft Copilot, and OpenClaw can all extract data from bot-protected sites using the Stealth Scraper tool through ToolRouter, with no API keys or coding required.
What does the Stealth Scraper tool do?
Scrape and crawl websites that block standard scrapers using stealth browser rendering and anti-bot evasion.
Related Use Cases
Crawl Multi-Page Sites for Structured Data
Follow links across multiple pages of a site to collect structured data from every matching page in a single crawl.