For store owners
About SkincareCompareBot
Dewbirdy is an Australian skincare price index. Our crawler reads the public product feed your storefront already publishes so we can show shoppers what a product costs at your shop alongside others.
What it requests
One /products.json page per 250 products, roughly six to twelve requests for a whole catalogue. We read titles, prices, variants, stock status and images. We do not submit forms, follow checkout links, or request anything behind a login.
How often
Once a night, with at least a second between any two requests across all stores — most shops here are on shared infrastructure, so we pace globally rather than per host. We honour Crawl-delay when your robots.txt sets one, and we back off rather than retry hard on a 429.
How to block it
Disallow SkincareCompareBot in your robots.txt and we stop, permanently and without needing to be asked twice. We check before every request, we treat a 403 as a decision rather than an obstacle, and we do not rotate user agents or addresses to get around either.
Corrections
If a price of yours is shown wrongly, tell us and we will fix the listing. We would rather show nothing than something inaccurate.