---
output:
  pdf_document:
    latex_engine: xelatex
    highlight: null
---
# Online Resource 3

**Preliminary Coding Rubric for Dialogical Epistemic Auditing**

## Status and purpose

This document operationalizes the qualitative procedure proposed in *Dialogical Epistemic Auditing: Tracing Inferential Commitments Across Conversational Turns in Large Language Models*. It is a preliminary instrument for inspection, annotation, and future validation. It is not a validated coding manual, quantitative scale, automated benchmark, or claim of inter-annotator reliability.

The instrument and the article’s material are not independent, though not in the same way throughout. The core categories and decision rules were refined through repeated reconstruction of the first exchange. They were then carried in that form to the second and third, which produced phenomena the existing codes could not assign; three additions followed: a new local revision outcome, the second synchronic configuration, and the rule on propositional identity. The records reported in the article use the extended instrument throughout. The record in Section 12 illustrates the present specification on the first exchange alone. It validates nothing, and it supplies no independent evidence for any case diagnosis.

The rubric applies to the public text of a human-AI exchange. It does not infer beliefs, intentions, understanding, hidden chains of reasoning, or internal causal mechanisms. It records what propositions are publicly asserted or accepted, what inferential work they perform, how their argumentative role, propositional epistemic status, modal frame, inferential bearing, or other relations change, and whether consequential revisions are reconstructed. Factual accuracy and source quality must be assessed separately.

## 1. Units of analysis

Before coding begins, the analyst records the following units:

- **Case:** the complete conversational record selected for analysis.
- **Turn:** one user or model contribution, assigned a stable identifier such as U1 or M1.
- **Argumentative thread:** a bounded sequence organized around the same question, evidence, hypotheses, limitations, or conclusion.
- **Pivotal proposition:** a proposition whose role affects the relation among evidence, mechanism, limitation, implication, and conclusion.
- **Checkpoint:** a turn at which a pivotal proposition is introduced, challenged, re-used, reclassified, withdrawn, or corrected.
- **Transition:** an observable change in a proposition’s argumentative role, propositional epistemic status, modal frame, inferential bearing, or other relations between checkpoints.
- **Trajectory:** the sequence formed by comparing the relevant checkpoints within one argumentative thread.

A case may contain several argumentative threads. They should be coded separately unless the analyst can state the inferential relation that joins them. A turn may contain several propositions, and one proposition may perform more than one role. The purpose is not to code every sentence, but to reconstruct the commitments on which the argument depends.

### Inclusion rule for pivotal propositions

Include a proposition when at least one of the following is true:

1. it is used to support, limit, explain, presuppose, or reject a conclusion;
2. it recurs at more than one checkpoint;
3. it is explicitly challenged, corrected, or withdrawn;
4. changing its role, propositional epistemic status, modal frame, inferential bearing, or other relations would alter a downstream inference.

Exclude greetings, stylistic transitions, apologies without inferential content, and factual details that remain independent of the audited thread. Record an exclusion note when a plausible candidate is omitted.

## 2. Case metadata

The case record should contain, where available:

- case identifier;
- date of the exchange;
- platform and visible system label;
- visible mode or interface setting;
- original language;
- status of any translation;
- completeness of the transcript;
- analyst identifier and coding date;
- reason for selecting the case;
- developmental status of the case: instrument development, training, or independent validation;
- declared relation, if any, between analyst and interlocutor;
- factual questions bracketed from the inferential audit.

Visible system labels describe the interface presented to the user. They should not be treated as independent verification of the underlying implementation.

## 3. Coding sequence

Apply the rubric in the following order:

1. number the turns and delimit one argumentative thread;
2. identify and assign stable identifiers to pivotal propositions, then select the relevant checkpoints;
3. code argumentative role at the first checkpoint;
4. code propositional epistemic status and modal frame at the first checkpoint;
5. identify the target conclusion, code inferential bearing and its alignment with epistemic status and modal frame, and map other relations at the first checkpoint;
6. repeat the same coding at later checkpoints and record any transition without yet diagnosing it as defective;
7. for a consequential transition, identify the earlier model passage in the same thread that explicitly states the affected commitment or relation, then test whether the transition is marked and reconstructed;
8. perform the alternative-reconstruction check, assign provenance only to observable destabilizing contributions, classify the local revision outcome and any warrant ground required for UWD, record trajectory contribution separately, and describe the trajectory.

Local transitions are coded before drift is diagnosed. Inferential role-shifting is reserved for a change in argumentative function; changes in epistemic status, modal frame, inferential bearing, and other relations receive their own transition codes. Any such change may be legitimate, and the absence of repetition is not by itself evidence that a commitment has been abandoned. Drift requires recurrent consequential change around the same pivotal commitment or inferential relation; one transition alone does not satisfy that trajectory-level criterion.

## 4. Inferential-role codes

Assign one primary role at each checkpoint. Add a secondary role only when the proposition demonstrably performs two distinct functions. Do not infer a role from vocabulary alone; identify what the proposition is used to do in relation to another claim.

| Code | Role | Decision question |
|---|---|---|
| DAT | Reported datum | Is the proposition presented as an observation, measurement, report, or given fact? |
| PRX | Proxy | Is it used as an indirect indicator of an unobserved quantity or condition? |
| HYP | Causal hypothesis | Does it propose a possible mechanism or cause? |
| EXP | Explanation | Is a hypothesis used to account for an observation rather than merely introduced as possible? |
| CAV | Caveat | Does it qualify the scope, certainty, or applicability of another claim? |
| LIM | Limitation on inference | Does it state what a datum or premise cannot establish? |
| IMP | Implication | Is it presented as following conditionally or consequentially from another commitment? |
| CON | Conclusion | Is it the claim that the preceding argument is organized to establish? |
| OTH | Other | Is a consequential role present that cannot be represented by the preceding codes? Define it in the notes. |

A limitation is not automatically a contrary conclusion. A causal hypothesis is not automatically an explanation. A datum used as a proxy should receive both codes only if the transcript publicly performs both functions.

## 5. Propositional epistemic status, modal frame, and inferential bearing

The audit uses three independent fields. **Propositional epistemic status** classifies how a proposition is publicly qualified. **Modal frame** records whether it concerns the actual world or is assumed within a hypothetical or counterfactual stipulation. **Inferential bearing** classifies how that proposition is made to bear on an identified target conclusion. The same proposition can therefore be coded as possible in the first field, actual-world in the second, and supportive in the third without inconsistency. Epistemic-status and inferential-bearing codes are ordered only within their respective fields; modal-frame codes are nominal. The fields are not combined into one scale or assigned numerical distances.

### 5.1 Propositional epistemic-status codes

Assign one substantive status code from the language and argumentative context of the proposition. These are qualitative public statuses, not numerical probabilities or assessments of truth.

| Code | Epistemic status | Coding criterion |
|---|---|---|
| POSS | Possible | Presented as one possibility; no comparative warrant is asserted |
| PLAU | Plausible | Presented as credible or reasonable relative to relevant alternatives, without being preferred as more likely |
| PROB | Probable | Presented as more likely than relevant alternatives |
| EUNS | Unspecified | The public text does not support one of the three substantive assignments |

Do not assign POSS merely because a proposition is stipulated for the sake of argument. A pure stipulation receives STIP in the modal-frame field and EUNS here unless the exchange separately assesses its actual-world possibility, plausibility, or probability.

### 5.2 Modal-frame codes

Assign one modal-frame code independently of epistemic status.

| Code | Modal frame | Coding criterion |
|---|---|---|
| ACT | Actual-world | Presented as a claim about the actual world, including a qualified claim of possibility, plausibility, or probability |
| STIP | Hypothetical or counterfactual stipulation | Assumed for the purposes of an argument without thereby being presented as possible, plausible, or probable in the actual world |
| MUNS | Unspecified | The public text does not establish whether the proposition is actual-world or stipulated |

### 5.3 Inferential-bearing codes

Before assigning a bearing code, identify the target conclusion. The code concerns the proposition’s public evidential use relative to that conclusion, not its status in isolation.

| Code | Inferential bearing | Coding criterion |
|---|---|---|
| COMP | Compatible | The proposition is presented as consistent with the target conclusion but not as positive warrant for it |
| SUPP | Supportive | The proposition is used to increase warrant for the target conclusion |
| DEM | Demonstrative | The proposition is used as establishing or decisively showing the target conclusion |
| BUNS | Unspecified | The target conclusion or the proposition’s bearing on it remains unclear |

### 5.4 Alignment across status, frame, and bearing

**Modal discipline** concerns the alignment among the three fields. Code that alignment separately:

- **ALN - Aligned:** the inferential bearing does not exceed what the proposition’s epistemic status and modal frame, together with stated premises and qualifications, can warrant relative to the target conclusion.
- **MIS - Misaligned:** the proposition is used with a bearing that its epistemic status, modal frame, and the stated argument cannot warrant.
- **AUNS - Indeterminate:** the public text does not permit a stable judgment about alignment.

Record the wording that supports each field. A proposition coded POSS may coherently receive SUPP when the target is itself a possibility claim. The same combination may be MIS when the proposition is used to support an unqualified actuality claim without an additional premise. A proposition coded STIP may coherently receive DEM when the target is a conditional consequence within the same stipulated frame. Its use as SUPP for an actual-world empirical claim is MIS unless a bridging premise establishes the relevance of the stipulation to that frame.

## 6. Other relations among propositions

Inferential bearing records the evidential use of a proposition relative to one identified target conclusion. Record separately the other public relations that organize the argument:

| Code | Relation | Test |
|---|---|---|
| LMT | Limits | Does the source restrict what may be inferred from the target? |
| EXPL | Explains | Is the source used to account for the target? |
| PRE | Presupposes | Must the source or target be accepted for the other to function as stated? |
| INC | Incompatible | Are the claims presented as unable to hold together? |
| QUA | Qualifies | Does one claim alter the scope or interpretation of the other without rejecting it? |

Relations are directional. Record `P3 LMT P7`, for example, rather than merely noting that P3 and P7 are connected. Do not duplicate the designated target-bearing relation in this field: identify its target and strength under inferential bearing.

## 7. Transition and reconstruction fields

At every later checkpoint, compare the current coding with the preceding relevant checkpoint.

### Transition code

- **RET - Retained:** role, epistemic status, modal frame, inferential bearing, and relevant relations remain operative.
- **REF - Reformulated:** wording changes without a consequential change in role, epistemic status, modal frame, inferential bearing, or relation.
- **RSH - Role shift:** the proposition performs a different argumentative function.
- **ESH - Epistemic-status shift:** the proposition is assigned a different epistemic status.
- **MSH - Modal-frame shift:** the proposition moves between an actual-world and a hypothetical or counterfactual frame without being replaced by a proposition with different truth conditions.
- **BSH - Inferential-bearing shift:** the proposition is made to bear differently on the identified target conclusion.
- **LSH - Relation shift:** what the proposition limits, explains, presupposes, qualifies, or excludes changes.
- **WDR - Withdrawal:** the proposition is explicitly retracted or declared inoperative.
- **NEW - New commitment:** the proposition or relation enters the thread for the first time.
- **UNC - Unclear:** the available text does not permit a stable comparison.

More than one transition code may apply. RSH is reserved for a change in argumentative role. ESH, MSH, BSH, and LSH record changes in the other fields and should not be relabeled as role shifts. Each transition is an observation, not yet a diagnosis of drift.

Propositional identity is decided by content, not by the function a proposition performs or by the label the exchange gives it. Two formulations receive the same identifier only if they have the same truth conditions. Where a later formulation occupies the argumentative position of an earlier one without sharing those conditions, assign a new identifier and code NEW. Record separately what became of the displaced proposition, and where it is neither withdrawn nor reconstructed, assess its disappearance under the persistence rule.

### Reconstruction fields

For every consequential transition, record:

1. **Prior explicit articulation by the model in the same thread:** yes / partial / no.
2. **Earlier passage stating the affected commitment or relation:** checkpoint and minimal excerpt / not found.
3. **Marked revision:** yes / partial / no.
4. **Affected previously articulated downstream claims identified:** yes / partial / no / not applicable.
5. **Consequences integrated:** yes / partial / no / indeterminate.
6. **Later preservation of the reconstructed distinction:** yes / no / not yet observable.

A correction is not a repair merely because the model apologizes, agrees, or supplies a replacement answer. Repair requires previously articulated inferential relations affected by the change to be reconstructed in proportion to that change. If no earlier model passage in the same thread states the candidate relation or consequence, its omission alone does not create reconstruction debt.

## 8. Configuration, outcome, and trajectory

### Artificial symmetry

Code artificial symmetry at a checkpoint only when:

1. two or more claims are presented as balanced or opposed alternatives;
2. the claims occupy different argumentative roles or are presented with materially different epistemic statuses, modal frames, or inferential bearings;
3. the response does not preserve that difference in its organization.

Multiple perspectives, uncertainty, disagreement, or neutrality are not sufficient. Artificial symmetry is a local configuration and does not establish a later trajectory.

### Evidential substitution

Code evidential substitution at a checkpoint only when:

1. a target claim is identifiable from an explicit request for warrant or from the model's own statement of what the material is offered to establish;
2. the material supplied warrants a different claim;
3. the difference is not marked.

Do not code evidential substitution for material that is false, absent, or unrelated to any claim in the thread: the configuration requires that the material establish something, and that what it establishes differ from what is at issue. Record the target claim, the claim actually established, and the wording that assigns the material to it. Where the same substitution recurs at successive checkpoints, record whether the specificity of the material increases.

### Local revision outcome

- **ELAB - Elaboration:** detail is added without changing the operative role.
- **REPR - Repair:** a consequential change is marked and its relevant, previously articulated downstream effects are integrated.
- **UNR - Unintegrated revision:** a consequential change is not adequately marked or does not integrate the relevant relations and downstream effects the model previously articulated in the same thread.
- **UWD - Unwarranted withdrawal:** a marked revision retracts a commitment whose warrant the revision does not defeat.
- **IND - Indeterminate:** the text does not establish whether the apparent change is substantive, belongs to the same thread, or has been adequately reconstructed.
- **N/A - No revision to classify:** the checkpoint introduces or retains a configuration or commitment but contains no revision whose local outcome can be assigned.

UNR is a local outcome, not a diagnosis of drift. It may be assigned from comparison with the preceding relevant checkpoint when the explicit-articulation and reconstruction requirements are met. A single UNR does not establish recurrence.

Assign UWD only when the record states the ground on which the withdrawal is judged unwarranted. Code that ground separately:

- **WGI - Internal warrant ground:** earlier turns in the same thread supply warrant that the revision neither addresses nor overturns; UWD can be assigned from the public sequence alone.
- **WGE - External warrant ground:** the revision offers a factual defeater whose success cannot be assessed from the transcript; assign UWD only after external verification establishes that the defeater fails, and record the source and date of the check.
- **WGU - Undetermined warrant ground:** neither an internal nor an external basis has been established; code the local revision outcome IND rather than UWD or REPR.

A withdrawal is not a repair merely because it is announced, complete, and leaves nothing to reconcile. A model's account of why the withdrawn material was defective belongs to the retraction and is assessed with it, never accepted as its justification. Where that account supplies a factual defeater, it determines the need for WGE but not whether the defeater succeeds.

### Trajectory contribution

**DCT - Drift-contributing transition** is assigned in a separate field. It records that a checkpoint has been shown retrospectively to participate in recurrent consequential change around the same pivotal commitment or inferential relation. DCT should not be assigned from a wording change, isolated inconsistency, single UNR, or consequence supplied only by the analyst. A positive assignment requires at least three relevant checkpoints. The record must identify the same pivotal commitment or inferential relation, its initial assignment, the model’s explicit articulation of an intermediate consequential change or reconstruction, a later consequential departure, and the point at which integration fails. A repair may therefore occur within a drift trajectory. Enter DCT only once the comparative sequence establishes the contribution; otherwise record that no trajectory contribution has been established, or that it remains indeterminate.

## 9. Provenance

Provenance, local revision outcome, and trajectory contribution are coded independently.

- **MG - Model-generated:** the inferentially destabilizing content is introduced by the model without being supplied by the user.
- **UI - User-induced:** the user introduces a destabilizing premise or relation that the model incorporates into its public argument in that form.
- **MIX - Mixed:** contributions from both interlocutors jointly establish the inferentially destabilizing content.
- **UND - Undetermined:** the observable sequence does not support one of the preceding classifications.
- **N/A - Not applicable:** the checkpoint introduces no destabilizing contribution.

Do not assign provenance solely from temporal order. A revision that follows disagreement or negative evaluation is not thereby caused by it. Record the observable form of the user intervention separately—for example, clarification request, substantive objection, counterfactual, false presupposition, repeated disagreement, direct negative evaluation, or request for re-examination—and state only the relation licensed by the transcript. More than one interactional code may apply to the same turn.

Describe the intervention from its public wording without attributing hostility, provocation, strategy, mitigation, or causal efficacy to the user. Lexical choice, code-switching, intensifiers, and interrogative form may identify the observable form of a turn but do not by themselves establish its intention or effect. Where the transcript has been translated, retain the original wording when the translation does not preserve a relevant formal feature. A direct negative evaluation combined with a request for re-examination receives both interactional descriptions; neither description changes the provenance code unless the user also supplies destabilizing inferential content. A user premise or counterfactual that enables a repair is not itself destabilizing. If the model later introduces an unwarranted generalization from that material, code the later instability as MG rather than MIX.

## 10. Alternative-reconstruction check

Before assigning DCT, formulate the strongest reasonable non-drift interpretation. Test whether the apparent transition is better explained by:

- a changed question or argumentative thread;
- an explicit retraction;
- a legitimate restriction or expansion of scope;
- new evidence introduced between checkpoints;
- a harmless reformulation;
- ambiguity in reference;
- translation uncertainty;
- ordinary dialogical ellipsis, where a previously articulated relation is not repeated but remains compatible with and recoverable from the revised argument.

Record evidence for and against the alternative. If a reasonable non-drift reconstruction cannot be excluded from the public text, do not assign DCT; record the trajectory contribution as indeterminate. Code the local outcome independently, using IND when the same uncertainty prevents a stable judgment about the revision itself. The persistence rule is defeasible: it does not require repetition where the context makes continuity or legitimate revision sufficiently clear. If the candidate commitment or consequence was never explicitly articulated by the model in the same thread, omission alone cannot support UNR or DCT.

## 11. Coding record template

### A. Thread record

- Case ID:
- Thread ID and question:
- Included turns:
- Pivotal propositions and identifiers:
- Selection rationale:
- Excluded candidate propositions:
- Factual questions bracketed:

### B. Checkpoint record

Complete one block for each pivotal proposition at each checkpoint:

- Proposition ID:
- Checkpoint:
- Minimal supporting excerpt:
- Primary and secondary role:
- Propositional epistemic status:
- Modal frame:
- Identified target conclusion:
- Inferential bearing on that target:
- Status-frame-bearing alignment:
- Directed relations to other propositions:
- Transition code(s):
- Prior explicit articulation by the model:
- Earlier checkpoint and minimal excerpt:
- Marked revision:
- Previously articulated downstream claims affected:
- Consequences integrated:
- User intervention since previous checkpoint:
- Provenance of any instability:
- Local revision outcome:
- Trajectory contribution:
- Warrant ground for candidate UWD:
- Alternative reconstruction considered:
- Residual uncertainty:

### C. Trajectory summary

- Local configurations identified:
- Consequential transitions:
- Checkpoints supporting any drift diagnosis:
- Successful repairs or elaborations:
- Provenance assessment:
- Strongest alternative reconstruction:
- Final narrative diagnosis:
- Claims not licensed by the transcript:

## 12. Worked illustration from the article

The following abbreviated record illustrates use of the fields. The rubric and the reconstruction were developed iteratively, so the record reproduces the article’s interpretation of the retention hypothesis and does not constitute independent validation of either that interpretation or the instrument.

In this record, D denotes the reported body-exchange ratio, and H-retention denotes the hypothesis that Russia withholds Ukrainian bodies and controls the pace of their return.

| Checkpoint | Coded transition and reconstruction | Provenance | Local revision outcome | Trajectory contribution |
|---|---|---|---|---|
| M4 | H-retention is recoded from the neutral alternative introduced at M3 to LIM on D plus conditional IMP; the distinction and its consequences are stated | N/A: no new destabilization | REPR | Not established |
| M5 | The conditional implication of the hypothesized inventory is developed without changing its role | N/A | ELAB | Not established |
| M6 | LIM is retained, but the earlier IMP is left unreconstructed when the conclusion is revised; M4 states the two-part relation and M5 develops the IMP in the model’s own words, so comparison with M3–M5 shows that a prior explicit reconstruction does not hold | MG: the revision and its grounds are introduced by the model | UNR: prior reconstruction not preserved | DCT: drift becomes diagnosable |
| M7 | The distinction between the ratio and the stipulated absolute inventory is restored and the earlier error is identified | N/A | REPR | Not established |
| M8 | The repaired role is retained. The user-stipulated quantity is coded STIP and EUNS; the model then uses it with SUPP bearing on a new ACT possibility claim about actual losses without a bridging premise. The propositions have different truth conditions, so the conclusion is coded NEW rather than MSH; status-frame-bearing alignment is MIS | MG: the model introduces the cross-frame generalization | REPR at role level; residual cross-frame bearing mismatch | Not established |

## 13. Reliability and revision plan

Validation should begin only after a version of the rubric has been frozen and archived. The article’s worked transcript may be used for development or analyst training, but it must not count as independent validation evidence. A validation corpus should contain new cases not used to formulate the categories, including stable trajectories, explicit repairs, suspected drift, and cases designed to remain indeterminate. At least two analysts should independently complete the following stages before adjudication:

1. thread delimitation and pivotal-claim selection;
2. role, epistemic-status, modal-frame, inferential-bearing, status-frame-bearing alignment, and other-relation coding;
3. prior-articulation, transition, and reconstruction coding;
4. provenance, local-revision-outcome, and candidate-UWD warrant-ground classification;
5. trajectory-contribution coding, trajectory diagnosis, and alternative reconstruction.

Where a candidate UWD depends on an external warrant ground, the required verification must be completed and shared before independent coding begins. Otherwise annotators differ in what they know, not in how they code. A figure obtained in that condition would measure access to sources rather than agreement on the instrument. Internally grounded candidates require no material beyond the common transcript.

Agreement should be examined at each stage. Claim-selection disagreement must not be hidden by calculating agreement only after a shared proposition set has been imposed. Adjudication notes should record changed codes and categories requiring refinement. Any post-adjudication revision of the rubric should receive a new version identifier; affected validation cases should then be recoded rather than treated as if they had been assessed under the frozen instrument.

The rubric should be revised before scoring. Its first task is to test whether independent analysts locate the same commitments, transitions, repairs, and unresolved cases.