Seven worked research examples

Selected examples are drawn from a PISA research project. PISA (Programme for International Student Assessment) is an international assessment of 15-year-old students; the project examined how published education-system comparisons change over time. No expertise in the particular statistical method is required to understand the decision at issue. They are organised by methodological problem, not by application.

A separate constructed workflow illustration uses an invented survey example for teaching. It is not part of the documented PISA case set and does not purport to supply empirical or historical evidence.

The cases describe project episodes but do not provide the full private PISA record. The evidence available to public readers is stated on each case page; internal verification and public reproducibility are distinct. See Evidence-status rules.

Each example follows the same structure:

Research problem → consequential move → evidence → challenge → outcome → reason-provenance lesson → evidential boundary

The examples

  1. When AI proposes a consequential research question
    Methodological issue: useful origination inside a researcher-originated research process.

  2. Human contribution as specification, constraint and refusal
    Methodological issue: contribution can be governance rather than novelty.

  3. Generated analysis is not empirical evidence
    Methodological issue: generation and verification have different evidential status.

  4. When evidence makes the preferred method claim smaller
    Methodological issue: comparator evidence should be allowed to weaken the contribution.

  5. A plausible number with an unwarranted inference
    Methodological issue: correct computation and valid interpretation are different questions.

  6. Retrospective storytelling and the reason-provenance problem
    Methodological issue: later explanation is not automatically contemporaneous evidence.

  7. Direct engagement with the finished scholarly object
    Methodological issue: process control does not remove the need to inspect the final object.