Case 3 — Generated analysis is not empirical evidence

What a reader needs to know

An AI system can write an analysis script without running it. A ZIP is a packaged collection of files; a terminal runs commands on the researcher’s computer; a checksum checks whether the transferred file has changed; and a handback contains outputs and logs returned after execution. These steps establish whether a particular computation was run, not whether its findings are substantively right.

Research setting

Generative AI was used to draft analytical code and package empirical tasks.

The project imposed a strict distinction between generated analytical artefacts and empirical evidence.

Workflow

AI-generated package
↓
Versioned ZIP
↓
Manual transfer to the researcher’s Mac
↓
Local terminal execution
↓
Returned handback ZIP, checksum, logs and outputs
↓
Inspection
↓
Result becomes eligible to enter the evidence base

The research project did not assume that code had run merely because code had been supplied.

Why this matters

A fluent explanation of what code would do is not evidence that the computation happened.

Likewise, successful execution establishes what the supplied code did. It does not by itself establish that:

• the design was conceptually appropriate;
• the inputs were correct;
• the output supports the proposed substantive inference.

Those require additional checks.

Reason-provenance lesson

Execution evidence and scientific justification are different evidential layers.

A useful AI-assisted workflow can delegate code drafting while preserving a strong boundary around what counts as empirical evidence.

Evidence available to public readers

The page explains how execution and checking were separated in the project. No executable research code, execution logs, handback archives or underlying outputs are supplied here for independent inspection. Readers can inspect the decision rule, not reproduce the particular analyses from this page.

Public resource choice

No research code is required for this repository. The pedagogic object is the workflow and evidential distinction, not the software itself.