# AdRoast robots.txt # Canonical host: https://www.adroast.ai (apex 301s to www) User-agent: * Allow: / Disallow: /app Disallow: /api Disallow: /auth Disallow: /onboarding Disallow: /r/ Disallow: /roast? # AI assistants: https://www.adroast.ai/llms.txt # Explicitly allow image indexing (heroes + inline blog assets under /blog/) User-agent: Googlebot-Image Allow: / User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: anthropic-ai Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot-Extended Allow: / User-agent: Bytespider Allow: / User-agent: CCBot Allow: / # Submit only this sitemap in GSC (remove https://adroast.ai/sitemap.xml if still listed) Sitemap: https://www.adroast.ai/sitemap.xml