# One group for every crawler: search engines and AI search and answer services are welcome # (llms.txt gives them a structured description of the site). A crawler named in a group of its # own follows only that group and skips the exclusions below. User-agent: * Allow: / # internal tools and the server-side statistics, not content Disallow: /calibrate.html Disallow: /database_viewer.html Disallow: /ulm_viewer.html Disallow: /podcast-lab.html Disallow: /podcast-lab-voices.html Disallow: /stats.php Disallow: /stats-data/ Disallow: /stats-geo/ # Crawlers that collect content to train AI models: not allowed. The content is reserved against # text and data mining (Section 44b(3) German Copyright Act, Art. 4(3) Directive (EU) 2019/790); # terms of use: https://tabula-peutingeriana.com/impressum.html#nutzung # AI search and answer services (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, # PerplexityBot, Perplexity-User and others) and search engines are not affected. User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: Applebot-Extended User-agent: CCBot User-agent: Bytespider User-agent: meta-externalagent User-agent: FacebookBot User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: Diffbot User-agent: AI2Bot User-agent: Ai2Bot-Dolma User-agent: omgili User-agent: omgilibot User-agent: Timpibot User-agent: ImagesiftBot User-agent: PanguBot Disallow: / Sitemap: https://tabula-peutingeriana.com/sitemap.xml