One analytical decision, three different kinds of warrant
Entirely constructed teaching illustration. Every proposal, decision and record extract below is invented to show a reasoning process. No survey dataset, AI transcript, model output, date, estimate or actual research action underpins the case. The example makes no empirical finding and does not establish the effectiveness of the framework.
Setting and pre-use decision
A social scientist plans a cross-sectional survey analysis involving reported job insecurity, self-reported wellbeing and employment-contract type. The researcher considers how insecurity and wellbeing may be associated within different contract-type groups. The design is observational; it cannot establish a causal effect merely by comparing groups.
Before consulting an AI service, the researcher must decide whether the proposed survey material may be shared under approvals, consent conditions, institutional policies, collaborator agreements and the specific service arrangement. For this invented exercise, the AI sees only a generic question and no actual respondent records. The same restriction would apply in real work unless permission for disclosure had been established.
A. Proposal and provisional decision
A first GenAI exchange (constructed, not a transcript)
Example researcher request — no participant data shared:
I am planning an observational survey study of reported job insecurity and self-reported wellbeing. I am not sharing respondent data. Suggest two or three comparisons worth considering, the assumptions or checks each would require, and one inference I should avoid. Do not claim to have run an analysis or invent references or findings.
Illustrative assistant reply (written for teaching; not actual model output):
- Consider whether the insecurity–wellbeing association differs among employment-contract groups. Check that contract types and measures are comparable and that the groups are adequately represented.
- Consider whether the association varies by an independently justified demographic characteristic. Check missingness, subgroup coverage and the theoretical rationale; avoid indiscriminate subgroup searching.
- Compare an overall descriptive relationship with group-specific descriptions, while recognising that compositional differences may explain an apparent contrast.
- Do not infer: the survey establishes that employment contracts cause mental-health harm.
Researcher’s first decision: I will provisionally investigate the contract-type comparison, but I will not report a difference until I have examined the measures, group coverage, survey design and uncertainty. The researcher does not assume the other suggestions are appropriate. No output is accepted as a finding.
The researcher may retain a permitted, contemporaneous record if the proposal changes the analytical plan. Confidential source material need not be stored with the record. The exchange demonstrates a bounded starting interaction, not a prompt that guarantees reliable suggestions.
Constructed AI proposal carried forward: compare the association between reported job insecurity and self-reported wellbeing across employment-contract types, rather than describing only an overall association.
Constructed miniature decision-log entry:
| Field | Example entry |
|---|---|
| Research move | Add a contract-type comparison to the proposed descriptive analysis. |
| Status at proposal | Candidate; not an empirical result. |
| Researcher disposition | Conditionally retain for evaluation, rather than automatically adopt. |
| Why the condition matters | Contract categories must be meaningful and comparable; the survey must measure all constructs suitably; group coverage and design must support interpretation. |
| What could reverse the decision | Sparse or ambiguous categories, incomparable measures, unsuitable weights/design treatment, or uninformative/unstable group estimates. |
| Trace needed for later attribution | If legally retainable, a contemporaneous version of the AI suggestion and the researcher’s decision; other possible contributors should not be erased. |
An actual researcher would write the particular evidence and decision at the time. The entries above are not a fabricated timestamped project log.
B. Verification keeps the same move in view
The researcher now decides whether the contract-type comparison itself, not merely the overall insecurity–wellbeing association, can be supported.
Verification plan: what must be checked (not a completed Evidence Ledger). No data, code or sources have been checked in this exercise. Each item has status proposed, not performed. A real Evidence Ledger would record the actual inspected source/output and resulting finding only after the check occurred.
| Question | Proposed check | Status | Consequence of failure |
|---|---|---|---|
| Are the constructs comparable across contract types? | Inspect questionnaire wording, response coding, category definitions and missingness. | Not performed | Redefine groups or abandon the comparison. |
| Is the analysis defensible for the observed sample? | Examine group sizes, survey design, relevant weights where appropriate, uncertainty and analytical specification. | Not performed | Narrow scope or withdraw unsupported group statements. |
| Does the hypothesised pattern survive meaningful challenges? | Compare suitable alternative specifications and contextual explanations; inspect sensitivity and possible selection or compositional differences. | Not performed | Report instability or an indeterminate finding rather than a strong result. |
| Does the design identify a causal mechanism? | Assess what the study design actually identifies. | Not performed | Do not infer causation from a cross-sectional comparison alone. |
Hypothetical branch, not a result: If the required checks were performed and the observed data supported different descriptive insecurity–wellbeing association patterns across contract groups, the researcher could consider the restricted statement below. No such checks have been run for this exercise; the conditional result is not presented as observed evidence. Conversely, failed checks would lead to rejection or narrower reporting of the original AI suggestion.
C. Claim authorisation determines what may be written
Claim authorisation means deciding precisely what a verified research object licenses the manuscript to say. A successful exploratory comparison does not automatically warrant a general or causal statement.
| Candidate wording | Decision | Why |
|---|---|---|
| “Insecure contracts cause declining mental health.” | Prohibited. | Contract category is not a causal identification strategy; wellbeing is not automatically equivalent to clinical mental health. |
| “Among respondents included in the analysis, the association between reported job insecurity and self-reported wellbeing differed across the contract-type groups examined.” | Conditional permission only. | Appropriate only if the group-specific association and differences were actually supported by checked measures, analyses and uncertainty; no finding is asserted here. |
| “Across the wider population, contract type changes the insecurity–wellbeing relationship.” | Not authorised by a sample comparison alone. | Population-level inference requires suitable sampling, survey-design and statistical justification as well as interpretation of the measures. |
| “Differences remain after adjusting for specified measured characteristics.” | Conditional permission only. | Requires that particular, checked analysis and does not eliminate unmeasured explanations. |
Constructed miniature claim-preservation contract:
- If evidenced, authorised: describe the checked within-sample associations by the named contract categories, including design and uncertainty limitations.
- Not authorised: causal claims; interchange of wellbeing and clinical mental health; generalisation beyond the analysed respondents without further support.
- Preserve during revision: the group comparison, its conditional evidential status and the distinction between descriptive heterogeneity and causal explanation.
- Reopen explicitly: only if a later source check, analysis, theory or design correction changes what can be inferred.
Editing test: better prose is not new evidence
Second hypothetical branch: Imagine a later, real study had performed the required checks and supported the narrow descriptive claim above. No such study is reported here. An AI-assisted editor then proposes:
“Precarious contracts drive a damaging mental-health effect.”
Reject: the sentence changes exposure, outcome, design and causal status. A permitted alternative would retain the bounded descriptive formulation about the association among analysed respondents, with measured constructs and qualifications intact. A paragraph that becomes shorter by deleting those constraints has not been improved in an evidential sense.
D. Contribution attribution requires a separate trace
The researcher later wishes to write that AI originated the contract-type comparison. Which statement survives depends on earlier records, not on how coherent the final paragraph sounds.
| Evidence that might survive | Bounded account it could support | What remains unknown |
|---|---|---|
| A preserved AI proposal and researcher acceptance recorded when the comparison was considered, with source and integrity checked | The recorded assistant proposed the comparison and the researcher conditionally selected it during the documented workflow. | Whether the idea existed previously, was suggested offline or was independently envisaged elsewhere. |
| Only a versioned analysis plan containing contract categories | The comparison was in that version of the plan. | Who originated it or why it was chosen. |
| A researcher’s later recollection | The researcher retrospectively recalls one history. | Whether that history is contemporaneously corroborated. |
Miniature reason-provenance conclusion: If the first type of evidence existed, an appropriately bounded contribution statement could describe the suggestion and the researcher’s evaluation as documented functions. Without it, the site should not claim definite AI origination. The present hypothetical case supplies no actual attribution trace.
Evidence that a proposal was generated is not evidence that it was scientifically sound. Evidence that the completed comparison was valid does not establish who first thought of it.
E. Finished-object inspection and release screening
Final inspection compares the manuscript’s abstract, main text, visualisations and conclusion against the claim-preservation contract. The researcher should reject a caption that implies causal change when the underlying results describe group-specific associations only. Any empirical record, participant material, AI trace or colleague document must pass a separate publication-permission and disclosure review. Pre-use permission to send material to a service and permission to make it public are distinct decisions.
Transferring the logic to other designs
Qualitative interview study: AI proposes a theme linking perceived employment insecurity to experiences of workplace belonging. The researcher must assess analytic memos, sampling context, extracts and discrepant accounts using standards appropriate to the project’s epistemology. A theme’s plausibility is not sufficient for a cross-case claim; the first appearance of the label in a codebook is not proof of intellectual origination.
Historical source study: AI suggests that a policy document demonstrates institutional consensus. Inspect document provenance, authorship, intended audience, date, omissions and competing materials before generalising. A source may reliably establish what an institution said, but not what all its members believed.
The evidence gate changes with the design; the distinction among proposed interpretation, supported research claim and supported contribution history remains useful.
Relation to the PISA cases
The moving-benchmark question, withdrawn movement inference and retrospective-storytelling case illustrate separate problems from an actual research programme. Their evidential boundaries are documented on their respective pages. None validates this invented survey trajectory or supplies evidence that the hypothetical checks were carried out.
For real projects, adapt the Decision Log, Evidence Ledger, Claim-Preservation Contract and Reason-Provenance Record only when those records serve an actual decision.