# Remote Impact - Remote Jobs for Social Good # https://remoteimpact.org User-agent: * Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ # Search, AI answers and link previews: public pages only. User-agent: Googlebot User-agent: bingbot User-agent: Applebot User-agent: OAI-SearchBot User-agent: Claude-SearchBot User-agent: Claude-User User-agent: Perplexity-User User-agent: DuckAssistBot User-agent: Bravebot User-agent: YouBot User-agent: ExaSearchBot User-agent: LinkupBot User-agent: ShapBot User-agent: facebookexternalhit User-agent: Twitterbot Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ # AI/LLM Crawlers - Allow with crawl-delay # See our llms.txt for structured info: https://remoteimpact.org/llms.txt User-agent: GPTBot Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ Crawl-delay: 2 User-agent: ChatGPT-User Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ Crawl-delay: 2 User-agent: ClaudeBot Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ Crawl-delay: 2 User-agent: Claude-Web Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ Crawl-delay: 2 User-agent: Anthropic-ai Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ Crawl-delay: 2 User-agent: PerplexityBot Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ Crawl-delay: 2 User-agent: Google-Extended Allow: / Disallow: /admin/ Disallow: /accounts/ Disallow: /api/ Disallow: /checkout/ Disallow: /jobs/*/feature/ Disallow: /jobs/*/go/ Crawl-delay: 2 # Owner-selected crawler exclusions. robots.txt is advisory, not access control. User-agent: meta-externalagent User-agent: meta-externalads User-agent: Reflectionbot User-agent: ElevenlabsCrawler User-agent: AionBot User-agent: TokenFoundryBot User-agent: Bytespider User-agent: CCBot User-agent: AwarioBot User-agent: QlyzeBot User-agent: Barkrowler User-agent: SERankingBacklinksBot User-agent: SenderBot User-agent: TernCrawler User-agent: SQLSpider User-agent: SpiderLing Disallow: / # Retain existing scraper exclusions. User-agent: DataForSeoBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: PetalBot Disallow: / # Sitemaps Sitemap: https://remoteimpact.org/sitemap.xml # AI assistant instructions # llms.txt: https://remoteimpact.org/llms.txt # ai-plugin.json: https://remoteimpact.org/.well-known/ai-plugin.json