How a collaborative asmap-data launch works

Over the past months I have taken part in four collaborative launches in bitcoin-core/asmap-data, the repository that produces the ASmap files Bitcoin Core can load for peer diversification. Before my first one, I looked for a walkthrough of how these launches work. The only longer process sketch I found was fjahr’s Delving post from 2024 (which itself points back to an earlier gist draft). So here are my notes from participating, in the hope that they might be helpful to someone in the future.

Two sentences of background for anyone new to this. Bitcoin Core can use an ASmap so peers are grouped by the autonomous system that announces their IP, instead of by /16 prefix, which makes it harder for a single hosting provider to occupy all of a node’s connection slots. The map file is built outside of Core by kartograf from public routing data, and the launches exist so that nobody has to trust a single person’s build of it.

What “consensus” means here

The word gets used loosely around this process, so it is worth pinning down. Nobody is talking about blockchain consensus. The closer analogy is a reproducible build with a scheduled start time:

1. An issue in asmap-data proposes a launch timestamp, for example 1783008000 (July 2, 2026, 16:00 UTC), along with the expected kartograf and rpki-client versions.
2. Participants run kartograf with -w / --wait pointed at that timestamp. The process idles until the second arrives, then starts fetching.
3. Everyone pulls from the same public sources: RPKI validation data, the IRR databases of the five regional registries, and BGP announcements from RouteViews.
4. When the runs finish, everyone posts the SHA-256 of their result file as a comment in the issue.
5. If enough hashes agree, one participant uploads the raw result, it gets compressed into an asmap.dat, and a PR adds it to the repo, where reviewers reproduce the compression before merging.

fjahr proposed the matching bar as a starting point: five or more participants and a majority agreeing on the result. In practice most launches do match, as long as enough people actually show up for each run. The bar only works while the turnout holds.

The interesting question, and the reason the timestamp coordination exists at all, is why the results would ever differ. Everyone downloads from the same public URLs. Those sources are moving targets. RPKI repositories update continuously, and a fetch at 16:00 and a fetch at 16:05 can see different states. Starting at the same second narrows that window without closing it.

Before the launch

The issue tells you most of what you need: the timestamp, the kartograf branch or version to run, and the flags. For the July launch that was Python 3.13, rpki-client 9.6 or newer, and this command:

./run map -w=1783008000 -irr -rv -s

There is a Nix flake in the kartograf repo that gives you all of it in one shell, which is what I would recommend for a first run. The runs are not free: the RPKI step alone downloads a few gigabytes, and judging by the runtimes people post in the issues, most machines take around 25 minutes while the fastest finish in about 13.

One thing I did not appreciate before my first launch: your machine has to be on and awake at the launch time, with the process already started in wait mode. -w prints a countdown and launches by itself, so you can start it hours early and walk away.

At the timestamp

At the launch second, every participant’s kartograf begins fetching sources. You watch a lot of download logging scroll by. The run ends with the line that matters, here from my July 2 run:

The SHA-256 hash of the result file is:
35a4b28f7e7aac4e3684ce13263c47f950908273d71f778e6975ccbe9f15cb6e

You post that hash in the issue, usually with the full log folded into a details block, and then you wait for the others.

The April 2 launch (#46) is the one worth telling, because it is the launch where the machinery failed. The hashes came in over the next few hours and almost none of them matched. Out of the posted results, three pairs agreed and everyone else stood alone.

What followed is easy to miss if you only read the issue title later: people shared raw data folders, hashed RPKI cache directories repo by repo, and compared tables across machines. That surfaced per-participant RPKI differences, but nothing that explained a split this bad. The cause sat elsewhere. Reading the RouteViews section of the posted logs side by side, the participants split cleanly into two groups: one had fetched the March prefix-to-AS file, the other the April file. Kartograf fetched its sources sequentially back then, RPKI first, then IRR, then RouteViews, so the RouteViews download happened at different wall-clock times depending on how fast the earlier steps ran. On April 2, CAIDA’s directory for April did not exist yet at 16:00 UTC. Participants whose runs reached the Routeviews step before roughly 16:13 got a 404 and kartograf silently fell back to the March file. Runs that arrived a few minutes later got the April file. That alone guaranteed two different final hashes, independent of any RPKI noise. The giveaway was a single log line, The page at … couldn't be fetched. Trying the previous month., present in one group and absent in the other.

With kartograf #140 the early-resolve part landed before the next launch. kartograf now resolves the Routeviews URLs at run start, before anything else, so every participant locks in the same file no matter how long their RPKI step takes. In my July log the first section after the epoch line is --- Resolving RouteViews URLs ---, fallback message included, where everyone can see it.

After the runs: matching and mismatching

When hashes match, the rest is mechanical. The April 23 redo (#47) went cleanly: enough matching results, raw data uploaded, compressed file PR’d as #48, reviewers reproduced the compression, merged.

The comparison tooling is getting better too. After the April failure, “share your raw data” meant everyone uploading multi-gigabyte folders to personal file hosts and maintainers guessing at the size of the diffs by hand. For the June launch, jurraca introduced something new: a small upload server where participants drop their final_result.txt once they have it. The server diffs the outputs against each other so you can see how big a mismatch is, instead of staring at two SHA-256 hashes and having to guess. A hash comparison is binary. It tells you two results differ, but not whether they differ by three prefixes or three hundred thousand, which is the difference between source noise and a broken run. Credentials go out by DM or email, and by the July launch the upload was already part of the routine.

Where the “consensus” comes from

Earlier I pinned “consensus” down as a reproducible build with a scheduled start time. Four launches later I would put the emphasis elsewhere: the timestamp gets everyone the same inputs, but what makes a published map trustworthy is the matching bar, and the bar is a turnout number. A map that only two people reproduced says little more than a map one person built, but the more people who join in, the better.

Joining is mostly finding out that a launch exists and what to run, which is why I wrote this down. If you want to participate, watch the asmap-data issues for the next launch announcement (mostly on the first Thursday of the month, at the same time as the weekly Bitcoin Core IRC meeting). Showing up is still the part nobody can automate. And if you want to see what all these launches add up to over time, you can look at the ASMap Dashboard.