FlintmereBot
The Flintmere scanner, explained.
FlintmereBot visits public Shopify storefronts to read the same catalog surfaces a buyer or an AI shopping agent would see. We publish anonymous, aggregate benchmarks so merchants can compare themselves to their vertical — never an individual-store list.
User agent string
FlintmereBot/1.0 (+https://audit.flintmere.com/bot)
RFC 7231–compliant. The +URL resolves to this page so any admin seeing the traffic can verify who we are.
What we fetch
- /robots.txt, /sitemap.xml, /llms.txt — to see what the store tells crawlers.
- /products.json — Shopify’s public product feed, which Shopify itself exposes on every storefront.
- A small sample of product pages — to read structured data, meta tags, and rendered markup.
We do not attempt to sign in. We do not submit forms. We do not read data behind authentication. We do not crawl customer or order pages.
Rate limits
One request every two seconds per host. A complete visit typically fetches fewer than twenty URLs. Each domain is revisited at most once per month.
How to block us
Add this to your /robots.txt:
User-agent: FlintmereBot Disallow: /
We respect robots.txt. New directives are picked up within 24 hours. To remove data from the benchmark entirely, email hello@flintmere.com. We reply within two working days.
Why we publish aggregates only
Naming individual stores in a league table would be unfair and unhelpful. We publish vertical medians (e.g. “beauty stores, median grade C”) and never a list of stores by grade. The research you see on /research is drawn from these aggregates.