Robots SEO Analyzer
The robots.txt analysis API provides an efficient solution for examining and understanding robots.txt files on the web, streamlining the management of crawling directives.
18 subscribers
4 endpoints
The in-depth APIMemo review for this API hasn't been published yet —
the data below comes straight from the public marketplace listing.
Robots SEO Analyzer endpoints
| Method | Endpoint | Description |
|---|---|---|
| GET |
robots /robots |
Get raw robots data from a website. |
| GET |
sitemaps /sitemaps |
Return the list of sitemaps that are specified in robots.txt file. |
| GET |
crawlDelay /crawlDelay |
Get the crawl delay by user agent. If crawlDelay is null there is no crawl delay for the specified user agent. |
| GET |
allowed /allowed |
Returns true if crawling the specified URL is allowed for the specified user-agent. |
Robots SEO Analyzer pricing
| Plan | Price | Rate limit | Quotas |
|---|---|---|---|
| BASIC | Free | — |
|
| PRO Recommended | $5.99 / month | — |
|
| ULTRA | $20 / month | — |
|
