Introducing open source Cambrian Core. Explore
Legal & Compliance • Effective August 11, 2026

Parallel Web Systems Bot Policy

Information for webmasters and content owners regarding Parallel's web crawler user-agents and polite retrieval standards.

1. Crawler User-Agents

Parallel operates web retrieval user-agents including 'ParallelBot', 'ParallelBot-Extract', and 'ParallelBot-Monitor'. Our crawlers identify themselves via clear User-Agent headers linking to verification hostnames.

2. Respect for Robots.txt and Rate Limits

Our crawlers honor robots.txt directives and employ adaptive backoff to avoid imposing server load on origin hostnames.

3. Index by Parallel for Content Owners

Through index.cambrian.sh, content creators and publishers can register domains, configure indexing preferences, and participate in direct licensing and compensation programs when AI agents access their knowledge.