# As a condition of accessing this website, you agree to abide by the following # content signals: # (a) If a Content-Signal = yes, you may collect content for the corresponding # use. # (b) If a Content-Signal = no, you may not collect content for the # corresponding use. # (c) If the website operator does not include a Content-Signal for a # corresponding use, the website operator neither grants nor restricts # permission via Content-Signal with respect to the corresponding use. # The content signals and their meanings are: # search: building a search index and providing search results (e.g., returning # hyperlinks and short excerpts from your website's contents). Search does not # include providing AI-generated search summaries. # ai-input: inputting content into one or more AI models (e.g., retrieval # augmented generation, grounding, or other real-time taking of content for # generative AI search answers). # ai-train: training or fine-tuning AI models. # use: how AI systems may consume the content (immediate, reference, or full). # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF # RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT # AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET. # BEGIN Cloudflare Managed content User-agent: * Content-Signal: search=yes,ai-train=no,use=reference Allow: / User-agent: Amazonbot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: meta-externalagent Disallow: / # END Cloudflare Managed Content # robots.txt for arowest.com # Western Crest Ventures LLP. All content (c) 2026, all rights reserved. # # The posture here is deliberate. Search engines and AI ANSWER engines are # welcome, because being cited is the whole point. Crawlers that exist only to # bulk-copy the site into a training corpus are not, because they take the work # and never send a reader back. Blocking a training crawler costs no visibility: # GPTBot, ClaudeBot, CCBot and Google-Extended feed model weights, not # citations, and Google has confirmed Google-Extended has no bearing on AI # Overviews, AI Mode or ordinary ranking. # # Licensing terms are machine-readable: the License: line below (RSL 1.0), # /.well-known/tdmrep.json (EU TDM reservation) and /terms/. # ---- Content signals for every crawler -------------------------------------- # Cloudflare Content Signals Policy: # search = index the page and show a link or snippet # ai-input = use the page to ground an AI answer that cites the source # ai-train = use the page to train or fine-tune a model Content-Signal: search=yes, ai-input=yes, ai-train=no User-agent: * Allow: / # ---- Search engines --------------------------------------------------------- User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Bingbot Allow: / User-agent: DuckDuckBot Allow: / User-agent: Applebot Allow: / User-agent: YandexBot Allow: / # ---- AI answer engines: welcome, they cite and send readers ------------------ User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: DuckAssistBot Allow: / User-agent: YouBot Allow: / User-agent: MistralAI-User Allow: / User-agent: Meta-ExternalFetcher Allow: / # ---- Model-training crawlers: no. They copy, they never cite. ---------------- User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: Google-Extended Disallow: / User-agent: Google-CloudVertexBot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: FacebookBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: cohere-ai Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: AI2Bot Disallow: / User-agent: Ai2Bot-Dolma Disallow: / User-agent: PanguBot Disallow: / User-agent: Kangaroo Bot Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: TimpiBot Disallow: / User-agent: img2dataset Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Scrapy Disallow: / User-agent: Diffbot Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: iaskspider/2.0 Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: Sidetrade indexer bot Disallow: / # ---- Machine-readable licence + sitemap ------------------------------------- License: https://arowest.com/rsl.xml Sitemap: https://arowest.com/sitemap.xml