Perplexity Crawlers
| User Agent | Description |
|---|---|
| PerplexityBot | PerplexityBot is designed to surface and link websites in search results on Perplexity. It is not used to crawl content for AI foundation models. To ensure your site appears in search results, we recommend allowing PerplexityBot in your site's robots.txt file and permitting requests from our published IP ranges listed below.Full user-agent string: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)Published IP addresses: https://www.perplexity.com/perplexitybot.json |
| Perplexity‑User | Perplexity-User supports user actions within Perplexity. When users ask Perplexity a question, it might visit a web page to help provide an accurate answer and include a link to the page in its response. Perplexity-User controls which sites these user requests can access. It is not used for web crawling or to collect content for training AI foundation models.Full user-agent string: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)Published IP addresses: https://www.perplexity.com/perplexity-user.json Since a user requested the fetch, this fetcher generally ignores robots.txt rules. |
WAF Configuration
Section titled “WAF Configuration”If you're using a Web Application Firewall (WAF) to protect your site, you may need to explicitly whitelist Perplexity's bots to ensure they can access your content. Below are configuration guidelines for popular WAF providers.
Cloudflare WAF
Section titled “Cloudflare WAF”To configure Cloudflare WAF to allow Perplexity bots:
Create a new rule
Click on Custom rules and create a new rule to allow Perplexity bots.
Configure rule conditions
Set up a rule that combines both User-Agent and IP address conditions:
- Field: User Agent
- Operator: Contains
- Value:
PerplexityBotORPerplexity-User
AND
- Field: IP Source Address
- Operator: Is in
- Value: Use the IP ranges from the official endpoints listed below
Set rule action
Set the action to Allow to ensure these requests bypass other security rules.
AWS WAF
Section titled “AWS WAF”For AWS WAF configuration, create IP sets and string match conditions:
Create IP sets
In the AWS WAF console, create IP sets for both PerplexityBot and Perplexity-User using the IP addresses from the official endpoints listed below.
Create string match conditions
Create string match conditions for the User-Agent headers:
PerplexityBotPerplexity-User
Create allow rules
Create rules that combine the IP sets with the corresponding User-Agent strings, and set the action to Allow.
Associate with Web ACL
Associate these rules with your Web ACL and ensure they have higher priority than blocking rules.
IP Address Sources
Section titled “IP Address Sources”- PerplexityBot IP addresses: https://www.perplexity.com/perplexitybot.json
- Perplexity-User IP addresses: https://www.perplexity.com/perplexity-user.json