Completed example — Claim-Preservation Contract

Project: PISA moving-benchmark analysis
Research object / manuscript: Beyond PISA rankings
Analysis version: Mature empirical specification
Contract status: Retrospective representation of the authorised claim boundary

1. Verified evidence

Result / object ID	Verified statement	Source / location	Uncertainty / qualification
EV-PC1	PC1 explains approximately 97.31% of variation in the three published domain means	Frozen comparator output	Describes dominant global variation
EV-BM	Ball Mapper provides overlapping local neighbourhood/profile structure	Frozen Ball Mapper outputs	Sensitive to construction choices; not a replacement for global summary
EV-TRAJ	Ordered chain among 36 systems: 21 downward, 14 unchanged, 1 upward	Frozen trajectory output	Specific to identified ordered-state construction
EV-ROB	Across 250 landmark orders, median 19 downward, 14 unchanged, 1 upward	Robustness archive	Exact states vary; broad gradient persists
EV-UK4	England, Scotland, Wales and Northern Ireland are analysed separately; aggregate UK excluded	Final analytical specification	PISA 2022 sampling caveat remains relevant

2. Authorised claims

Claim ID	Authorised claim boundary	Supported by	Permitted variation in wording
C1	Rank movement, own-score movement and movement within a changing comparison field are non-equivalent	Empirical design + observed trajectories	May be restated without implying causality
C2	PCA is the preferred compact global summary of the three-domain structure	EV-PC1	“Preferred”, “compact”, “dominant global summary”
C3	Ball Mapper complements PCA by representing local overlapping neighbourhood/profile relationships	EV-BM + comparator evidence	Must retain complementary/local meaning
C4	Broad downward movement is robust across landmark-order sensitivity checks	EV-TRAJ + EV-ROB	Must retain robustness scope
C5	The four UK administrations should be interpreted separately in this analysis	EV-UK4	Must retain aggregate-UK exclusion rationale

3. Prohibited claims

Prohibited claim / claim class	Reason prohibited
Ball Mapper uncovers a fundamentally different global taxonomy that PCA misses	PC1 captures ~97.31% of global variation
17.35 percentage points moved directly from B4 to B9	Direct same-system coupling is not identified
Rank improvement demonstrates substantive educational improvement	Rank and own-score movement are non-equivalent
Robustness to landmark order establishes all forms of statistical uncertainty	Algorithmic sensitivity differs from survey/measurement uncertainty
The PISA case validates the reason-provenance framework	Case is developmental illustration, not validation

4. Inferential boundaries

• Endpoint occupancy changes do not identify direct same-system flows between overlapping neighbourhoods.
• Comparator robustness does not remove survey sampling and measurement uncertainty.
• Descriptive international trajectories do not by themselves establish causal mechanisms.
• Local neighbourhood structure and global dimensionality answer different questions.

5. Required qualifications

Claim ID	Required qualification
C2	Global summary refers to the three published domain means
C3	Ball Mapper is complementary rather than substitutive in this application
C4	Robustness refers to the specified landmark-order analysis
C5	PISA 2022 UK sampling-standard caveat remains visible

6. Frozen conceptual distinctions

Distinction	Must remain separate
Own-score change vs rank change	Do not use rank as a proxy for performance change
Global summary vs local neighbourhood structure	PCA and Ball Mapper answer different representational questions
Generated artefact vs verified empirical evidence	Code/text generation is not execution/verification
Endpoint occupancy vs direct flow	Do not infer unidentified coupling
Algorithmic robustness vs survey uncertainty	Do not collapse the two uncertainty sources