What the record promised, before it knew the answer

Integrity

This file, sha256

273077977b5f9aba5ff79c1ccaecfbc3a362f70cf3d9faad426c21676467ba4f

Hash the live record publishes

273077977b5f9aba5ff79c1ccaecfbc3a362f70cf3d9faad426c21676467ba4f

Verdict

matches the record

Checkpoints

GW10, GW19, GW38

Canonical path

docs/preregistration-2026-27.md (27307797…ba4f)

The document, verbatim

# Pre-registration — the 2026/27 accuracy record

Committed 12 September 2026, after three settled gameweeks. This file fixes, in advance, what the public accuracy record at sidelinequant.com/performance will report at three checkpoints, on which rows, against which comparators, and what would count as the model being beaten. Its SHA-256 is embedded in the record's payload (`provenance.preregistration.sha256`) and pinned by a test in the repository, so the record cannot quietly change what it measures after the fact.

## Checkpoints

Gameweeks 10, 19 and 38 of 2026/27. At each, the record publishes every figure below for every settled gameweek to date, per gameweek and pooled, and the homepage states which comparator leads on each pooled measure.

## What is measured

Per gameweek and pooled:

1. Mean absolute error and root-mean-square error of expected points (xP) against realised FPL points, per player-fixture.
2. Spearman rank correlation between xP and realised points.
3. Haul discrimination: the Mann-Whitney AUC of the retained P(≥6 points) and P(≥10 points) against realised hauls.
4. Reliability of the retained P(plays 60+ minutes) against realised minutes, and of P(≥6) and P(≥10) against realised hauls, in quantile bins with Wilson 95% intervals.
5. The model's own FPL entry (3502195) against the published average manager score, gross of transfer hits with hits stated separately, and against the sampled top-10k average where a capture predates the gameweek's deadline.

## On which rows

Every error figure is published on four stated populations, always together: all player-fixtures with a prediction; predicted E[minutes] > 0; played at least one minute (conditioned on the outcome, and labelled as such); predicted E[minutes] > 60 (the headline population). Rank and haul figures are on the predicted E[minutes] > 60 rows.

The prediction for a gameweek is the last model run before its deadline. Gameweek 1 predates run retention and is scored against the committed pre-season bundle of 18 August 2026, labelled `seed`, which carries no haul probabilities.

## Against what

Three comparators, each scored on the model's own rows, with the model re-scored on exactly the rows the comparator covers:

- Last season's points per game — a point prediction; error, rank and haul measures.
- Top-10k effective ownership at the last sampled capture before the deadline — a rank; rank and haul measures only.
- FPL's own published projection, `ep_next`, captured between the previous deadline and this one — a rank; rank and haul measures only. It accrues from the first snapshot after 9 September 2026 and is reported as unavailable before then.

Double-gameweek rows are excluded from the two rank comparators and counted.

## Intervals

95% percentile intervals from a seeded row-level bootstrap of 1,000 resamples for each gameweek's Spearman and haul AUC. Player-fixtures within a gameweek are dependent, so these intervals understate the true width; they are published as a floor on the uncertainty. A haul AUC on fewer than 50 hauls is published with the caption that it is too few to claim.

## Reproducibility

Each settled gameweek's pre-deadline prediction rows are served at `/performance/live/gw/{n}/predictions` and the record stamps their SHA-256 on that gameweek. The digest is of the `rows` array alone, serialised as compact JSON (`json.dumps(rows, separators=(",", ":"))`, UTF-8, no whitespace), rows sorted by (player_id, fixture_id), `null` for an absent value; the response states this beside the digest.

## What counts as being beaten

If at a checkpoint a comparator's pooled figure is better than the model's on any measure — lower error, higher rank correlation, higher AUC — the homepage and the record say so in those words, and the losing measure is not removed, reweighted or re-populated. Any change to this file after commit is a new pre-registration with a new hash, dated.