• dataset
  • script

Official crawler IP ranges: 18 lists and an IP checker

Where search and AI crawlers publish their IP ranges: 18 JSON lists with entry counts and dates, Amazon's three pages, and a script that checks an IP address.

  • v1.0
  • updated 8 November 2026
  • MIT
  • Vladislav Krivorutško

A request that calls itself Googlebot is real only if its IP address sits inside a range Google published. 10 operators publish such ranges: 18 JSON files hold 3,084 entries, and Amazon puts three more lists on its own pages. The table gives every address with its entry count and the date inside the file, all opened on 2026-11-08. The download is the same table as a CSV plus a short script that checks one IP against all 18 files.

Files in this resource

2 files · or all in one zip
  • crawler-ip-ranges.csv

    21 rows: operator, bots, class, list address, format, entry counts, the list's own date, check date, operator page

    3.9 KB
  • check_ip.py

    Python script: says which operator's official list contains an IP address

    2.6 KB

The 18 JSON lists

Every file has the same shape: a prefixes array whose items hold an ipv4Prefix or an ipv6Prefix.

OperatorBotsClassListIPv4IPv6File dated
GoogleGooglebot and other common Google crawlerssearch and crawldevelopers.google.com/static/crawling/ipranges/common-crawlers.json1701472026-11-06
GoogleGoogle special-case crawlers (AdsBot and others)special-case (ads and others)developers.google.com/static/crawling/ipranges/special-crawlers.json1361362026-11-06
GoogleGoogle user-triggered fetchersuser-triggereddevelopers.google.com/static/crawling/ipranges/user-triggered-fetchers.json5445442026-11-06
GoogleGoogle user-triggered fetchers (google.com addresses)user-triggereddevelopers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json2522522026-11-06
GoogleGoogle user-triggered agentsuser-triggereddevelopers.google.com/static/crawling/ipranges/user-triggered-agents.json1282026-11-06
Microsoft BingBingbotsearchwww.bing.com/toolbox/bingbot.json2902024-01-03
AppleApplebotsearchsearch.developer.apple.com/applebot.json2402026-09-15
OpenAIGPTBottrainingopenai.com/gptbot.json1802026-09-22
OpenAIOAI-SearchBotsearchopenai.com/searchbot.json3902026-01-02
OpenAIChatGPT-Useruser-triggeredopenai.com/chatgpt-user.json23702026-11-06
PerplexityPerplexityBotsearchwww.perplexity.ai/perplexitybot.json1302026-10-28
PerplexityPerplexity-Useruser-triggeredwww.perplexity.ai/perplexity-user.json1002026-10-28
AnthropicClaudeBot, Claude-SearchBot, Claude-Usertraining, search and user-triggeredclaude.com/crawling/bots.json1402026-10-20
OpenAIOAI-AdsBotadsopenai.com/adsbot.json202026-05-12
Common CrawlCCBotdataset crawlindex.commoncrawl.org/ccbot.json412026-08-11
DuckDuckGoDuckAssistBotsearchduckduckgo.com/duckassistbot.json48602026-09-01
Mistral AIMistralAI-Indexsearchmistral.ai/mistralai-index-ips.json202026-04-19
Mistral AIMistralAI-Useruser-triggeredmistral.ai/mistralai-user-ips.json402025-02-19

The date is the timestamp the operator put inside the file. Bing's file carries 2024-01-03 and still holds 29 ranges on 2026-11-08, so read the date as "last stamped by the operator", not as proof that a list is current.

Amazon: three pages, not JSON files

Amazon lists its addresses on three pages of its developer site. Each page embeds the data as JSON text, and the pages use different field names (ipv4Prefix for single addresses, ip_prefix for CIDR blocks), so the script does not read them. The counts are the number of entries found in the page text on 2026-11-08.

BotClassPageEntriesDated
Amazonbottrainingdeveloper.amazon.com/amazonbot/ip-addresses/1,2922026-09-08
Amzn-SearchBotsearchdeveloper.amazon.com/amazonbot/searchbot-ip-addresses/8162026-09-08
Amzn-Useruser-triggereddeveloper.amazon.com/amazonbot/live-ip-addresses/1,0232025-11-04

Who publishes no list

  • Meta: the crawler page gives user agent strings for meta-externalagent, meta-externalfetcher, Meta-WebIndexer and facebookexternalhit but no list of IP ranges (checked 2026-11-08). Requests that call themselves Meta can only be judged by their user agent.
  • ByteDance (Bytespider) and MistralAI-Training: no list documented on the date above.
  • Google-Extended and Applebot-Extended are robots.txt control tokens. They do not crawl, so they have no IP ranges.

Check one IP

bash
python check_ip.py 66.249.66.1 2001:4860:4801:10::1 132.196.86.1 8.8.8.8

The script downloads the 18 lists listed in the CSV and prints the matching list for each address. Output from the run on 2026-11-08:

text
66.249.66.1                              google-common (Googlebot and other common Google crawlers)
2001:4860:4801:10::1                     google-common (Googlebot and other common Google crawlers)
132.196.86.1                             openai-gptbot (GPTBot)
8.8.8.8                                  no match in 18 lists

Run it on the IPs behind a suspicious user agent in your own log. An address that calls itself Googlebot and matches nothing is not Google. In one 30-day audit of a single site, 531 of 2,278 requests that called themselves Google came from 21 IP addresses outside Google's published ranges.

Rather have this done for you?

Send the site address and what you want to know. We run the audit and send the report.

Send a request

Keep it fresh

Operators change these files, and some addresses move. The CSV is a snapshot of 2026-11-08. check_ip.py downloads the lists on every run, so only the counts and dates in the CSV age. The log audit kit does the same download for 13 of these lists before it classifies a log.

Limits

An IP inside a published range shows who runs the machine, not why the request was made. Google documents a second check as well, a reverse DNS lookup of the address; the lists are the faster one when you have thousands of log lines. The lists are for verifying requests that already reached you. Do not turn them into firewall rules without testing: operators add and drop ranges, and the CSV is a snapshot.

Tested on

On 2026-11-08 all 18 JSON addresses and the three Amazon pages answered HTTP 200 and were parsed with Python 3.13. check_ip.py ran against four addresses: an IPv4 and an IPv6 address inside Google's common crawler list, one inside OpenAI's GPTBot list, and 8.8.8.8, which matched nothing. The five operator pages below also answered 200.

Files are provided as is: run them on your own data and check the result.

Want an audit like this on your site?

If you would rather have it done for you, send a request and describe the site.

Send a request
Download crawler-ip-ranges.csv