dataset

Do rank trackers repeat themselves? Top-10 overlap data, 2026-11-12

Two runs of the same query share 7.6 of 10 domains in A-Parser, 6.0 in DataForSEO Estonia, 4.8 in DataForSEO US. CSV of 4,642 rows, CC BY 4.0.

v1.0
updated 12 November 2026
CC-BY-4.0
Vladislav Krivorutško

Download serp-repeatability.csv Download all (.zip)Jump to the text

0.76
top-10 overlap between two A-Parser runs, Estonia
0.60
the same for DataForSEO, Estonia
0.48
the same for DataForSEO, United States
4,642
result rows, 60 queries, 237 measurements

On 2026-11-12, two runs of the same Google query a few minutes apart shared on average 7.6 of their top-10 domains in A-Parser, 6.0 in DataForSEO for Estonia and 4.8 in DataForSEO for the United States. In A-Parser, 60% of the pairs of runs had 8 or more of 10 domains in common and 7% had 4 or fewer. In DataForSEO for the United States, 33% had 8 or more and half had 4 or fewer.

This is how a tool agrees with itself. It is not accuracy: there is no reference browser here, and a tool can repeat itself and be wrong. The data answers one practical question: how big is the noise of a single measurement, so that a position jump in your tracker can be compared with it. The script that acts on this noise (a repeat check with a ceiling) is SERP repeat-check.

Files in this resource

3 files · or all in one zip

serp-repeatability.csv
One row per result: source, market, run, query, position, domain. 237 query measurements of 60 queries
258 KB
analyze.py
Python 3.8+, standard library only; reproduces every number on this page, with an offline self-test
10 KB
README.md
Method, definitions, columns, the unreadable-domain rows, limits
12 KB

What was measured

30 neutral Estonian queries (weather, recipes, public services, transport) and 30 neutral English queries (how-to, health, personal finance, travel), none related to the sites of the author or of his clients. Every query was run several times in one afternoon, within roughly 15 minutes:

A-Parser, Estonia: SE::Google on google.ee, hl=et, Tallinn, mobile, about 30 results, the Bypass poisoned SERP option on, residential proxies targeted at the country and city. 4 runs.
DataForSEO, Estonia: live organic endpoint, country level, language et, mobile, depth 30. 2 runs.
DataForSEO, United States: the same endpoint, language en. 2 runs.

Each row of the CSV is one organic result: the source, the market, the run, the query, the rank among the distinct domains (1 to 30) and the domain. Every pair of runs of one source and market was compared query by query, on the set of top-10 domains. A query counts when a run returned at least 12 distinct domains.

Results

Source / market	Runs	Measured pairs	Mean top-10 overlap	8 or more of 10	4 or fewer of 10
A-Parser, Estonia	4	159	0.76	60%	7%
DataForSEO, Estonia	2	23	0.60	39%	26%
DataForSEO, United States	2	30	0.48	33%	50%
A-Parser against DataForSEO, Estonia	4 against 2	186	0.37	3%	69%

An overlap of 0.76 means that two runs of one query share 7.6 of 10 domains on average. With 8 or more shared, the page is the same give or take one result. With 4 or fewer, it is a different page.

What the numbers say

One measurement is not a position. Even the steadier tool returned a different top 10 (4 or fewer common domains) in 7% of the pairs, and the domains they share move too: the median shift of a result ranked 4 to 10 is 1 place in A-Parser, but 1 in 10 moves 7 places or more.
The noise depends on the setup. The same query set gave 0.76 on one tool and 0.60 on another, and the English set on DataForSEO gave 0.48. The data cannot say how much is the tool, how much the configuration (city proxies against country level) and how much the market.
Two tools disagree more than one tool with itself. The overlap between A-Parser and DataForSEO for Estonia is 0.37, and 69% of the pairs share 4 or fewer domains. Which of them looks like what a person sees is not in this data.
A repeat check helps on the jumps. A replay on the four A-Parser runs: when a check shares fewer than 5 of 10 domains with the previous check, measure again and keep the second measurement only if it is closer to the previous check. 44 of 624 cases (7%) were flagged, and on them the overlap with a probe run rose from 0.62 to 0.86. The probe is one noisy run too, so read this as a direction.

The rows with an unreadable domain

590 of the 4,642 rows (117 DataForSEO Estonia, 473 DataForSEO United States) are results whose API answer was a Google redirect and not a site: social posts, forum threads. They are kept with domain_resolved = no, a stand-in unresolved-<hash> for the domain and no title. In the United States set they are 29% of the top-10 rows and agree less between runs than the readable ones (Jaccard 0.23 against 0.36). analyze.py --resolved-only drops them: DataForSEO Estonia then reads 0.64 and DataForSEO United States 0.59, so the headline numbers above use all rows and are the more cautious reading.

Rather have this done for you?

Send the site address and what you want to know. We run the audit and send the report.

Send a request

Limits

DataForSEO has two runs per market, which is 23 and 30 measured pairs. One query moves the mean by about 0.02. A-Parser has four runs and 159 pairs.
One date and one afternoon. Another day can give other numbers.
No reference. Agreement with itself is not accuracy.
60 neutral queries chosen by the author, in two markets that differ in language and in size. Commercial, local and rare queries can behave differently.
The configurations are not equal. A-Parser used city-targeted residential proxies and its Bypass poisoned SERP option; DataForSEO was set to the country. A difference between the sources is a difference between these setups, not a verdict on the products.
Runs back to back. If two runs hit the same data centre or session, the agreement looks higher than it would hours apart.
Domains, not pages. Which page of a domain ranks is not compared.

Re-run it

bash

python3 analyze.py --selftest
python3 analyze.py serp-repeatability.csv

The script needs Python 3.8 or newer and nothing else. To measure your own tool, write a CSV with the same seven columns (source, market, run, query, position, domain, domain_resolved), at least 2 runs and better 4, and run the script on it.

Cite it

The CSV and this page are licensed CC BY 4.0, the script MIT. Name the source and link to this page. Suggested form:

ADLAB, "SERP repeatability dataset: do rank trackers repeat themselves? (data, 2026-11-12)", https://adlab.ee/en/resources/rank-tracking/serp-repeatability-dataset, CC BY 4.0.

Tested on

2026-11-12: Windows 11, Python 3.13.14. The self-test passed 13 of 13 checks and the analysis reproduced the table above from the CSV. Not run on other Python versions or operating systems.