# https://www.robotstxt.org/robotstxt.html User-agent: * Disallow: /_ab/ # AI crawlers (OpenAI, Anthropic, Perplexity, Google, Apple, Bing, Common Crawl) User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Google-Extended Allow: / User-agent: Bingbot Allow: / User-agent: CCBot Allow: / User-agent: Applebot-Extended Allow: / # LLM context files (https://llmstxt.org) # - https://www.recall.it/llms.txt # - https://www.recall.it/llms-full.txt Host: https://www.recall.it Sitemap: https://www.recall.it/sitemap.xml