Build a Full Site Inventory
Crawl your entire website to build a complete inventory of every page, its status, and its metadata.
Audit URL Structure
Analyze your site URL hierarchy to ensure clean, SEO-friendly URL patterns and logical site architecture.
ToolYour URL structure is the backbone of your site architecture. Clean, descriptive URLs help search engines understand your content hierarchy, improve user navigation, and make your pages more shareable. Conversely, messy URLs with random parameters, excessive depth, or inconsistent patterns signal poor site organization.
ToolRouter's crawl_site skill maps every URL on your site, revealing the full URL hierarchy. This analysis exposes common problems: URLs that are too deep (more than 3-4 levels), inconsistent naming patterns between sections, URLs with unnecessary parameters or session IDs, mixed case URLs, and trailing slash inconsistencies.
A URL audit is particularly valuable before site migrations (plan your new URL structure), during SEO overhauls (identify structural improvements), and for large sites where URL patterns have become inconsistent as different teams added content over time.
Agent Guides
Claude
- Connect ToolRouter: claude mcp add toolrouter -- npx -y toolrouter-mcp
- Ask Claude: "Crawl mysite.com and analyze the URL structure"
- Claude maps the full URL hierarchy and flags structural issues
ChatGPT
- Configure ToolRouter in ChatGPT
- Ask: "Crawl mysite.com and analyze the URL structure for SEO issues"
- ChatGPT presents the URL hierarchy with flagged problems
Copilot
- Add ToolRouter to Copilot MCP config
- Ask: "Crawl mysite.com and audit the URL structure"
- Review the URL hierarchy and flagged issues
OpenClaw
- Connect ToolRouter: openclaw mcp add toolrouter -- npx -y toolrouter-mcp
- Ask: "Crawl mysite.com and audit the URL structure"
- OpenClaw maps the URL hierarchy and flags structural issues
Related Use Cases
Find 404 Errors Across Your Site
Crawl your website to discover all pages returning 404 errors and the internal links pointing to them.
Identify Orphaned Pages
Find pages on your site that have no internal links pointing to them, making them invisible to crawlers.
Check Internal Link Distribution
Analyze how internal links are distributed across your site to ensure important pages receive enough link equity.