For site owners
PolyRevu is building a guide to restaurants and other places to eat and drink, based on what people have publicly written about them. The site is not yet open to the public. This page says what our crawler reads, how it behaves, and how to turn it away.
Identification
If you run a website, you may see requests from PolyRevuHonestFetchBot. Its full identification is:
PolyRevuHonestFetchBot/1.0 (+https://polyrevu.com/about/crawler-notice; contact: data@polyrevu.com)
It always identifies itself with this name. It never pretends to be a browser or another crawler.
What it reads
Manners
robots.txt, including Crawl-delay.Opt out
To stop our crawler from reading your site, add this to your robots.txt:
User-agent: PolyRevuHonestFetchBot Disallow: /
You can also write to us and we will stop fetching your site and remove what we derived from it.
Contact
Questions, concerns, or a request to stop: data@polyrevu.com. Please include your domain name.