---
title: ToolRouter — Agent Use Case Guide
description: Step-by-step guide for completing a task with a specific AI agent using ToolRouter.
url: https://toolrouter.com/tools/geo/use-cases/check-ai-crawler-access/claude
---
# How to Check AI Crawler Access with Claude

Claude approaches AI crawler verification with analytical thoroughness, testing each major bot against your site and explaining the specific robots.txt rules or server responses causing blocks. It follows up with questions about your GEO goals to recommend exactly which crawlers to prioritize and what changes will have the most impact.

## Setup

claude mcp add toolrouter -- npx -y toolrouter-mcp

## Steps

1. Connect ToolRouter: claude mcp add toolrouter -- npx -y toolrouter-mcp
2. Ask Claude: "Check if AI crawlers can access my site using the geo tool" and provide your URL
3. Claude tests access for all major AI crawlers
4. Review which crawlers are blocked and which have access
5. Ask Claude to suggest robots.txt changes if crawlers are blocked

## Example Prompt

```
Check whether GPTBot, ClaudeBot, and PerplexityBot can crawl https://mysite.com. I want to make sure my content appears in AI search results.
```

## Tips

- Run this check after any robots.txt changes to avoid accidentally blocking AI crawlers
- Some CDN and security tools block AI crawlers by default -- check your WAF rules too
- Allowing AI crawlers is the prerequisite for all other GEO optimization work
