Web Agent opens a real browser, goes to a website and carries out a task you describe in plain English. It clicks, types, fills in forms, moves between pages and reports back what it found.
Use it when a plain page scrape is not enough: the data sits behind a search box, a filter, pagination, a multi-step form or a "load more" button. Describe the goal and, if you want structured data back, the shape you want.
What you can do
- Fill in search forms and return the results
- Step through filters, tabs and pagination to collect data
- Read information that only appears after interacting with the page
- Return results as JSON matching a schema you supply
- Run through a residential connection in a chosen country, or a stealth browser for sites that block bots
Who it's for
Researchers, analysts and operators who need data from interactive websites, and agents that hit a wall with ordinary scraping.
How to use it
- Call run_task with the starting url and a specific goal, e.g. "Search for 'standing desk', sort by price low to high and return the first 5 product names and prices"
- Pass output_schema when you need structured fields back
- Set stealth or country only if the site blocks the default browser or shows region-specific content
- Keep max_seconds low for simple tasks; longer tasks take longer and cost more
- For a single static page, use the Web Scraper tool instead; it is faster
Getting started
Call run_task with a URL and a goal. Simple tasks finish in under a minute; complex multi-step tasks take a few minutes. The task runs in the background and returns a job to check.
