Back to Radar
AentsRadarBot
AentsRadarBot is the crawler Aents Radar uses to read a public website when somebody asks for an analysis of it. This page explains what it does, at what pace, and how to keep it out.
How it identifies itself
User-Agent
AentsRadarBot/1.0 (+https://radar.aents.net/bot)
What it does
- Reads public pages of the analyzed site, at most ten of them.
- Respects robots.txt, and stops when a server refuses or rate-limits it.
- Requests at most one page every two seconds, with two connections at most.
- Reads robots.txt and the declared sitemap.
- Runs a browser to measure performance and accessibility on the pages the visitor confirmed.
What it never does
- Sign in, or try to reach anything behind a login or a paywall.
- Submit forms, buy, book or add anything to a cart.
- Follow links outside the analyzed domain.
- Scan ports or probe for vulnerabilities.
- Store the pages it read beyond the seven days a report lives.
How to block it
Add this to your robots.txt. Radar reads it before every crawl and will stop visiting.
User-agent: AentsRadarBot Disallow: /
If the crawler is causing you trouble, write to Aents and it will be stopped for your domain.
Radar exists so a business owner can see what an automated review finds on their own website. The crawler is the part that reads it: it is not an index, it does not build a public database, and the pages it read are deleted with the report that used them.
Questions people ask before running it
Why did AentsRadarBot visit my website?
Somebody asked Aents Radar for a free analysis of that address. Radar reads a handful of public pages, measures them and shows the result to whoever requested it. Nothing is published and nothing is sold.
Will it slow my server down?
It should not. Radar makes at most one request every two seconds with two connections at most, reads up to ten pages, and stops immediately when the server answers with a block or a rate limit.
How do I stop it permanently?
Disallow AentsRadarBot in your robots.txt. Radar reads that file before every crawl and honours it, even when a visitor asks for the analysis again.