PurpleScanBot Crawler — Technical FAQ

1. What is PurpleScanBot?

PurpleScanBot is the crawler operated by PurpleScan. It audits e-commerce properties for technical SEO health, HTTP security headers, consent-banner compliance, performance, and uptime regressions.

If PurpleScanBot is hitting your site, someone in your organisation — or an agency or platform partner — has registered the domain with PurpleScan. Typical checks include:

• Critical SEO tags (canonical, hreflang, meta robots, structured data).
• HTTP security headers (CSP, HSTS, COOP/COEP, X-Frame-Options).
• Pre-consent cookies and outbound tracking pixels.
• Core Web Vitals and rendering screenshots for regression auditing.

2. User-Agent strings

PurpleScanBot uses three distinct User-Agent strings depending on the request type. All three contain the token PurpleScanbot (case-insensitive), so a single substring match is enough to identify our traffic in logs or WAF rules.

Desktop (headless Chrome)

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36 (compatible; PurpleScanBot/1.0; +https://www.prplscan.com/ua)

Mobile (headless Chrome, Pixel 8 emulation)

Mozilla/5.0 (Linux; Android 13; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Mobile Safari/537.36 (compatible; PurpleScanBot/1.0; +https://www.prplscan.com/ua)

Raw HTTP (robots.txt, sitemaps, static checks)

PurpleScanbot/1.0 (+https://www.prplscan.com/ua)

3. Originating IP addresses

All PurpleScanBot traffic originates from two static addresses. There is no rotating pool — a single allow-list entry per address family is sufficient.

Reverse DNS does not currently resolve to a PurpleScan-owned hostname, so please rely on the IP + User-Agent combination for attribution.

4. How we rate-limit requests

• Per-website cap: at most 1 request per 60 seconds per registered website, enforced with a sliding-window limiter.
• No parallel dispatch: only one crawl schedule can be in flight per website at a time.
• Automatic back-off: on HTTP 429 responses, crawling for that website is paused until a cool-down window elapses.
• robots.txt crawl-delay: if your robots.txt declares a Crawl-delay directive, we honour it in addition to the per-website cap.

In practice this means the aggregate load generated by PurpleScanBot on any single origin is well below 1 request per second, even across our desktop and mobile passes.

5. robots.txt & sitemaps

• We fetch /robots.txt daily and cache the result.
• We obey Disallow rules matching either User-agent: PurpleScanbot or User-agent: *.
• We honour the Crawl-delay directive if present.
• We auto-discover sitemaps referenced from robots.txt, and fall back to /sitemap.xml if none are declared.

To block us entirely, add the following to your robots.txt:

User-agent: PurpleScanbot
Disallow: /

6. How often we crawl

Crawl frequency depends on the type of page, not the size of the website. Deep e-commerce catalogues are sampled rather than fully re-crawled on every run.

7. Browser & device emulation

• Engine: Headless Chromium 147 driven over the Chrome DevTools Protocol via Selenium WebDriver.
• Desktop profile: viewport 1920 × 1080, Windows desktop UA.
• Mobile profile: Pixel 8 emulation (Android 13), mobile UA.
• JavaScript: enabled for rendering passes; disabled for raw static fetches such as robots.txt and sitemaps.
• Accept-Language: en-US,en;q=0.9 by default.

8. Safety guarantees

• PurpleScanBot does not submit forms, log in, or interact with carts.
• It does not issue POST, PUT, or DELETE requests to unauthenticated endpoints.
• It does not attempt to bypass authentication, paywalls, or geo-fencing.
• It does not download arbitrary binaries beyond what a real browser would fetch to render the page.

9. How to allow-list us

If your CDN, WAF, or bot-protection product is blocking PurpleScanBot, either of the following rules is enough on its own. You don’t need both.

Option A — by IP address

Allow-list both:

51.91.78.67
2001:41d0:404:200::92a7

Option B — by User-Agent

Match the substring PurpleScanbot (case-insensitive) in the User-Agent header.

10. Reporting issues

Questions or abuse reports? Reach the PurpleScan crawler team at [email protected]. Please include your domain and an approximate timestamp (with timezone) of the traffic you’re asking about, so we can correlate with our crawl logs. We reply within one business day.