A free aeogeo scan runs 26 deterministic checks in under 10 seconds on any public URL (ruleset v20).

This generator writes a robots.txt that allows the AI crawlers your AEO playbook expects — using the same allow-list baked into aeogeo at build time.

How do I generate an AI-aware robots.txt?

Enter your site URL, optional disallow paths, and copy or download the result. Crawler names come from site/public/robots.txt, not a hand-maintained list.

FAQ

robots.txt generator

Which AI crawlers should robots.txt allow?

Per the aeogeo crawler_ops playbook, explicitly allow: OAI-SearchBot, ChatGPT-User, GPTBot, PerplexityBot, ClaudeBot, Google-Extended.

Does this generator call a server?

No — your URL and paths are turned into robots.txt text entirely in the browser.