---
title: ToolRouter — Agent Use Case Guide
description: Step-by-step guide for completing a task with a specific AI agent using ToolRouter.
url: https://toolrouter.com/tools/geo/use-cases/check-ai-crawler-access/openclaw
---
# How to Check AI Crawler Access with OpenClaw

OpenClaw automates AI crawler monitoring across all your domains. Configure it to test GPTBot, ClaudeBot, PerplexityBot, and other AI crawlers on a recurring schedule, and alert you whenever a previously-allowed crawler becomes blocked due to configuration changes or CDN updates.

## Setup

openclaw mcp add toolrouter -- npx -y toolrouter-mcp

## Steps

1. Connect ToolRouter: openclaw mcp add toolrouter -- npx -y toolrouter-mcp
2. Ask OpenClaw: "Check if AI crawlers can access my site" and provide the URL
3. OpenClaw tests each major AI crawler
4. Review the results and identify any blocks
5. Update robots.txt to allow blocked crawlers

## Example Prompt

```
Test whether GPTBot and ClaudeBot can crawl https://mysite.com. I need to ensure AI search engines can index my content.
```

## Tips

- Check both your main domain and subdomains separately
- AI crawler policies may differ from regular search engine crawlers
- Run this check monthly to catch unintended configuration changes
