Score calibration

What the basic scan does when pointed at real tokens on a real chain, what it got wrong, and what was changed because of it.

This document exists because a risk score that has never been checked against reality is a number, not a measurement. Everything below was run against Robinhood Chain mainnet on 2026-09-08 with the engine at 1.2.0, and every figure is reproducible with kay9-watchdog scan robinhood <address>.

There is no accuracy claim here, and there will not be one. "99 % accurate" would require a labelled ground truth for what a rug is, and no such thing exists. What can be said is narrower and checkable: which specific findings fired on which specific tokens, which of those were wrong, and what was done about the wrong ones.

---

1. The method

Tokens were not sampled at random. They were chosen to cover the states this chain actually produces, because a chain making tens of thousands of launches a day is mostly one state:

Each was scanned, the full signal list was read, and every finding was checked against the chain by hand — the pool it claimed did not exist, the holder it called a whale, and so on. A finding that survived that check is a true positive whatever the score is; one that did not is a defect.

---

2. What was wrong, and what changed

Three defects were found this way. All three were systematic — they would have fired on most of the chain, not on an unlucky token — and all three are now fixed and covered by tests in services/watchdog/test/liquidity-enumeration.test.ts.

2.1 A graduated token reported as having no liquidity

Observed. Token 0x20894263…1e8d ("Lemon") had graduated to a live Uniswap v4 pool. The scan reported EVM_NO_LIQUIDITY_FOUND — "None of the probed pools hold liquidity for this token" — and added 45 risk points with the LOW_LIQUIDITY flag.

Why. The scan derived a pool id from a guessed PoolKey: the token, a quote asset, and one of five standard fee tiers, with no hook. Reading the pool's own Initialize event shows what it really was:

On this chain a launch pool's fee is a launch tax, chosen per launch. No fixed probe list can enumerate 810000 with a tick spacing of 19988, and the tokens most worth scanning are exactly the ones such a list misses.

Changed. The scan now enumerates a token's pools from the PoolManager's own Initialize events — two eth_getLogs calls, filtered on the token in either currency position — and probes each pool id it finds. The fixed list survives only as a fallback. A pool id already known from discovery can be passed in, which skips the enumeration entirely.

Measured on mainnet: a 3,000,000-block enumeration (about three and a half days of chain time) returned in one second with 5 logs. A 10,000,000-block one was refused, so the lookback is 3,000,000.

Also changed, and more important: an empty pool list is now only a finding when the pools were enumerated. If only the guesses ran, the result is EVM_LIQUIDITY_NOT_ENUMERATED — unmeasured, raising INSUFFICIENT_DATA — because a guess that missed is not evidence about the asset. This is the same rule the engine already applied to refused RPC calls, extended to cover being wrong rather than being refused.

2.2 The Uniswap v4 singleton counted as a whale

Observed. Token 0x364e4f0e…8844 ("RWA Boxes") scored 80, high, on EVM_SINGLE_HOLDER_MAJORITY — "One address holds a majority of the circulating supply", critical, 70 points. The address holding 100 % was 0x8366a39c…0951.

That is the Uniswap v4 PoolManager. Uniswap v4 is a singleton: every pool's tokens sit in one contract, and there is no per-pool address to exclude the way a v2 pair can be. So the engine was not looking at a whale. It was looking at all pooled liquidity on the chain, credited to one holder.

Changed. The v4 PoolManager and PositionManager are excluded from the circulating base, as pool addresses, burn addresses and the token contract already were. Real concentration is unaffected: "Lemon" still reports EVM_HOLDER_CONCENTRATION with a top-10 share of 66 %, and its top-1 share corrected from 34.98 % to 15.09 % once the pool stopped counting as a holder.

2.3 A token nobody had traded, called concentrated

Observed. After 2.2, the same token still scored 80. With the pool excluded, its float was 764 tokens out of 1e27 — everything else was in the pool — and the one address that had bought held all of it. Technically a majority; substantively a token nobody has traded yet.

On a chain producing 20,000–50,000 launches a day, almost every one of them is in that state. The engine's highest severity would have been its least informative output.

Changed. When less than 1 % of supply is held outside pools, burn addresses and the token contract, concentration is not claimed at all. The scan reports EVM_NO_MEANINGFUL_FLOAT — unmeasured, raising INSUFFICIENT_DATA, with the explicit note that this is normal for a new token and is neither a finding against it nor any kind of reassurance. The score for that token moved from 80 to 25.

The guard cannot be used to escape a real finding: a token with 90 % of a real float in one address still reports EVM_SINGLE_HOLDER_MAJORITY, and there is a test for exactly that.

2.4 A crash the fixes exposed

Excluding the pool manager surfaced a latent defect: an excluded holder's share was computed against the circulating base it had been excluded from. A pool holding 1e27 of a token whose float is 1,000 came out at 130,853,707 — thirteen billion per cent — and a number that size cannot be canonicalised, so basicScan threw instead of reporting anything.

On this chain a tiny float is the normal case, so this would have taken out a large share of all scans. An excluded holder's share is now measured against total supply, where it is bounded and also means something: "the pool holds 99.9999 % of supply".

---

3. Where the engine is still wrong, and why it stays that way

The two tokens that should score lowest both score 60

A regulated stablecoin and a tokenised equity — the two things on this chain least likely to rug — read as elevated risk, for the same reason: the contract is upgradeable and has an owner, and any signal at high severity raises the overall score to at least 60.

The engine is not wrong about the facts. That contract can be upgraded, and whoever controls the proxy admin can change what the token does. What the engine cannot see is that the entity holding that power is a regulated issuer, and no amount of on-chain analysis will tell it.

This is not being "fixed", because every available fix is worse:

• A safelist of known-good tokens would make the score a function of KAY9's opinion rather than of the chain, and the whole product rests on the opposite.
• Down-weighting upgradeability generally would hide the single most common way a token rugs.

What is done instead: the score is never shown without its flags and its confidence, the report says in terms that a score describes capability rather than intent, and the site never renders a band as an endorsement. A reader who sees 60 and reads one line of why gets the true picture. A reader who sees only the number does not — and that is a design constraint on every surface, recorded in DESIGN.md.

Confidence is low, and honestly so

Every mainnet basic scan in this set reported confidence between 0.47 and 0.54. The reasons are the same each time and are named in the report:

• EVM_ARCHIVE_UNAVAILABLE — the public RPC is pruned, so the deployer cannot be established.
• EVM_ACTIVITY_NOT_SCANNED — trading activity is a deep-tier signal, not a basic one.
• EVM_CREATOR_NOT_SCANNED — creator history likewise.

A basic scan on a pruned public endpoint genuinely cannot see those things, and the number says so. An archive endpoint raises it; nothing else will.

---

4. What this exercise does not establish

• No false-negative rate. Every defect above is a false positive found by checking findings against the chain. Finding false negatives needs labelled rugs — tokens known to have rugged, with the date — and no such list exists for this chain yet. Until it does, no claim is made about what the engine misses. Building that set is the next calibration task.
• No sample size worth calling a sample. Four tokens is enough to expose a systematic defect and far too few to estimate a rate. Nothing here should be read as "the engine scores X % of tokens correctly".
• Nothing about deep or forensic tiers. Those need the auditor network running, and were not exercised here.

---

5. Reproducing this

cd services/watchdog
npx tsx src/cli.ts discover --blocks 20000        # what the chain is launching right now
npx tsx src/cli.ts scan robinhood <address>       # the full signal list for one token
npx tsx src/cli.ts scan robinhood <address> --json | jq .report.observations.concentration

The three fixes are pinned by test/liquidity-enumeration.test.ts, which encodes each defect as the scenario that produced it — a pool with an 81 % fee, a float of 1,000 units, a pool manager holding everything — so a regression fails the suite rather than quietly returning.