For store owners
About DewBirdyBot
Dewbirdy compares skincare prices across Australian stores. Our crawler reads the public product feed your storefront already publishes so we can show shoppers what a product costs at your shop alongside others.
What it requests
One /products.json page per 250 products, roughly six to twelve requests for a whole catalogue. We read titles, prices, variants, stock status and images. We don’t submit forms, follow checkout links, or request anything behind a login.
How often
Once a night, with at least a second between any two requests across all stores — most shops here are on shared infrastructure, so we pace globally rather than per host. We honour Crawl-delay when your robots.txt sets one, and we back off rather than retry hard on a 429.
How to block it
Disallow DewBirdyBot in your robots.txt and we stop, permanently and without needing to be asked twice. We check before every request, we treat a 403 as a decision rather than an obstacle, and we don’t rotate user agents or addresses to get around either.
Corrections
If we’ve got one of your prices wrong, tell us and we’ll fix it. We’d rather show nothing than something inaccurate.