About QeruenBot
QeruenBot is the automated crawler that builds the Qeruen search index.
User-Agent: QeruenBot/0.1 (+https://qeruen.kz/bot)
How we crawl sites
- We follow robots.txt, including Crawl-delay.
- We follow the robots meta tag (noindex, nofollow) and the X-Robots-Tag header.
- No more than 1 request per second and 1 concurrent connection per site.
- We only crawl sites in the .kz and .қаз zones.
How to limit or block crawling
Block User-agent: QeruenBot in robots.txt, or write to us — we'll remove the site or page from the index.