Acceptable crawling policy

StorePulse is designed to be a polite observer of public ecommerce data.

What we do

• Identify our crawler with a stable user agent and contact address.
• Enforce robots.txt before requesting crawlable pages.
• Use bounded concurrency, delays, response limits, conditional requests, and backoff.
• Coalesce scans when multiple workspaces watch the same physical store.
• Store image URLs as references instead of downloading durable copies.

What we never do

• Authenticate to a target store's admin, customer, cart, checkout, or order areas.
• Bypass CAPTCHAs, bot protections, access controls, or disallowed paths.
• Use residential proxy rotation or attempt to evade rate limits.
• Fetch private, loopback, link-local, metadata-service, or internal network addresses.

See also: Privacy policy · Terms of service · How StorePulse works