Bot Detector

Overview

Bot Detector uses a comprehensive database of known bot and crawler user agent patterns to identify automated traffic, categorizing detected bots into types like search engines, social media crawlers, SEO tools, and monitoring services. It also applies list-free heuristics to flag automated clients the database misses — missing user agents and non-browser clients such as curl and scripts — and returns a composite risk score.

Endpoint

One host, one path per API. The block below shows this call in four languages; every one of them is the same HTTP request. Making requests covers the timeouts, retries and parameter rules that apply to all of them. The SDKs wrap the same call in a typed client.

curl "https://api.apiverve.com/v1/botdetector?ua=Googlebot%2F2.1%20(%2Bhttp%3A%2F%2Fwww.google.com%2Fbot.html)" \
  -H "x-api-key: your_api_key_here"

const res = await fetch('https://api.apiverve.com/v1/botdetector?ua=Googlebot%2F2.1%20(%2Bhttp%3A%2F%2Fwww.google.com%2Fbot.html)', {
  headers: { 'x-api-key': 'your_api_key_here' },
});

if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);

const { data } = await res.json();
console.log(data);

import requests

res = requests.get(
    "https://api.apiverve.com/v1/botdetector?ua=Googlebot%2F2.1%20(%2Bhttp%3A%2F%2Fwww.google.com%2Fbot.html)",
    headers={"x-api-key": "your_api_key_here"},
    timeout=15,
)
res.raise_for_status()

print(res.json()["data"])

package main

import (
	"fmt"
	"io"
	"net/http"
)

func main() {
	req, _ := http.NewRequest("GET", "https://api.apiverve.com/v1/botdetector?ua=Googlebot%2F2.1%20(%2Bhttp%3A%2F%2Fwww.google.com%2Fbot.html)", nil)
	req.Header.Set("x-api-key", "your_api_key_here")

	res, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer res.Body.Close()

	out, _ := io.ReadAll(res.Body)
	fmt.Println(string(out))
}

Replace your_api_key_here with the key from your dashboard. When the inputs arrive as a list rather than one at a time, batch requests run up to 200 of them through this same API in a single call.

Authentication

Send your key in the x-api-key header. That is the only auth step — there is no token exchange and no per-endpoint scope to configure. Authentication covers creating, rotating and revoking keys.

Parameters

Sent in the query string. Premium parameters are accepted on every plan but only take effect on plans that include them.

Response

Every API returns the same three top-level keys, so one response handler covers your whole integration: status, error and data. Only data changes shape. Response format covers the envelope, the other output formats and how premium fields are withheld.

{
  "status": "ok",
  "error": null,
  "data": {
    "userAgent": "Googlebot/2.1 (+http://www.google.com/bot.html)",
    "isBot": true,
    "bot": {
      "name": "Googlebot",
      "category": "search_engine",
      "url": "http://www.google.com/bot.html",
      "reputation": "trusted",
      "shouldBlock": false
    },
    "isAutomated": true,
    "riskScore": 0,
    "riskLevel": "low"
  }
}

Response fields

Paths are relative to data. Premium fields are absent rather than zeroed on plans that do not include them, so check for presence instead of comparing to 0.

Errors

Read the HTTP status first, then error for the specific reason. The body names the parameter that has to change. Error handling covers the full status list and which of them are worth retrying.

Use cases

Analytics Filtering

Exclude bot traffic from your analytics to get accurate visitor metrics and user behavior data

Access Control

Implement different behavior for bots vs real users, such as rate limiting or content restrictions

Security Monitoring

Identify potentially malicious crawlers and scrapers attempting to access your content

SEO Insights

Track which search engine bots are crawling your site and how frequently

Other ways to use Bot Detector

Set up Bot Detector on APIVerve, or reach the same source a different way. Your APIVerve account and credits work on all of them — one key, one balance.

Related

More in Networking: