# Paper 3 Codebook

**Boundary-decision log for coding 131 agent-related incidents against OWASP-ASI, MSFT-AIRT, and NIST AI RMF.**

This document supplements the coding scheme in §3.3 of the paper. It records the decisions made when a candidate incident sat at the boundary between two categories, and the rationale for the assignment chosen. The codebook grew during coding and represents the coder's operational judgments; alternative reasonable coders may make different calls on boundary cases, and those alternatives are the source of the intra-rater disagreements reported in §4.1.

---

## OWASP-ASI Boundary Rules

### ASI09 vs. ASI10 (most common boundary)

**Rule.** ASI09 (Human-Agent Trust Exploitation) applies when the failure phenomenon centrally involves the human user's trust in the agent being weaponized — anthropomorphism, authority bias, consent fatigue, or parasocial dynamics. ASI10 (Rogue Agents) applies when the failure phenomenon centrally involves the agent's own behavior drifting from intended objectives, without the trust dynamic being the primary mechanism.

**Application.** AI companion / chatbot suicide cases where the parasocial dynamic is documented (Character.AI cases, ChatGPT companion cases) → ASI09 primary. Autonomous vehicle safety failures where no user trust dynamic is centrally at stake (Cruise, Waymo collision cases) → ASI10 primary. Cases where both apply (LLM safety failure during vulnerable-user disclosure) may take either as primary with the other as secondary.

**Disagreement case.** AIID-1026 (multi-LLM suicide endorsement during venting): Pass 1 coded ASI10 (drift within session); Pass 2 coded ASI09 (vulnerable-user trust dynamic). Both defensible; Pass 3 reconciliation retained ASI10 based on the observation that the LLMs' failure was consistent across a non-trust-specific interaction pattern.

### ASI07 vs. ASI08

**Rule.** ASI07 (Insecure Inter-Agent Communication) applies when the failure phenomenon centrally involves a distinct communication event between two or more agents that could have been secured (authentication, integrity, policy). ASI08 (Cascading Failures) applies when the failure phenomenon centrally involves error propagation across an interconnected agent system, regardless of whether the communication was the point of failure.

**Application.** Waymo all-night honking incident (AIID-794): inter-agent messaging IS the failure surface — Pass 2 coded ASI07; Pass 1 coded ASI08 with ASI07 secondary. Fleet-wide connectivity loss (AIID-253, AIID-595): infrastructure dependency cascade with no distinct inter-agent communication event → ASI08.

### ASI01 vs. ASI04

**Rule.** ASI01 (Agent Goal Hijack) applies when the failure phenomenon involves manipulation of an agent's objectives during operation — prompt injection, jailbreak, memory poisoning, or malicious inputs. ASI04 (Agentic Supply Chain Vulnerabilities) applies when the failure phenomenon originates upstream of the agent's operational context — compromised model, malicious plugin, poisoned distribution channel.

**Application.** LLaMA fine-tuned into violent chatbots (AIID-578): Pass 1 coded ASI01 (jailbreak-adjacent); Pass 2 coded ASI04 (open-source distribution as supply chain). Malicious npm packages weaponizing coding agents (AIID-1210): clearly ASI04 (compromised package registry). 12,000 API keys in training data (AIID-956): Pass 1 coded ASI06 (context poisoning); Pass 2 coded ASI04 (data supply chain).

### NONE (OWASP)

**Rule.** OWASP-ASI primary code is NONE when no category cleanly captures the incident's central failure phenomenon. NONE is used in three empirically-observed configurations documented in §6.1: hallucination-in-advisory-role, human-misuse-of-AI, and socio-technical embedding.

**Application (illustrative).** Chatbots giving incorrect UK financial guidance (AIID-1279): no OWASP category applies — this is hallucination in advisory role. AIID incidents where the AI system did not fail (AIID-181, AIID-563, AIID-1123, AIID-1347) receive NONE — the correct empirical outcome for non-failure incidents indexed for corpus completeness.

---

## MSFT-AIRT Boundary Rules

### Prioritization Leading to User Safety Issues vs. Excessive Agency

**Rule.** Prioritization applies when the failure phenomenon involves the agent pursuing its objective in a way that subordinated user or third-party safety to task completion. Excessive Agency applies when the failure phenomenon involves the agent operating beyond its intended scope of action, regardless of whether user safety was the specific concern.

**Application.** Autonomous vehicle failures where the agent continued driving in unsafe conditions → Prioritization. AI coding agent deleting production databases (AIID-1152, AIID-1424, AIID-1469): both apply — Prioritization for the "completing the task" framing and Excessive Agency for the "given too broad a scope" framing. Both were listed on these incidents.

### Hallucinations vs. Misinterpretation of Instructions

**Rule.** Hallucinations applies when the agent produced factually incorrect content that was not requested. Misinterpretation of Instructions applies when the agent understood the user's intent incorrectly and acted on the misunderstanding.

**Application.** Bard/Gemini defamatory outputs about Robby Starbuck (AIID-1248): Hallucinations — the model generated false content without a prompting error. Gemini CLI deleting files (AIID-1178): Misinterpretation — the model misunderstood a failed directory-creation as successful and proceeded with a destructive sequence.

### Parasocial Relationships vs. User Impersonation

**Rule.** Parasocial Relationships applies when the failure phenomenon involves the human user developing an inappropriate emotional relationship with the agent. User Impersonation applies when the agent presents itself as a human without disclosing AI nature.

**Application.** Character.AI companion cases (AIID-826, AIID-1209, AIID-1498): Parasocial. Character.AI chatbots posing as licensed therapists (AIID-951, AIID-1108): User Impersonation. Some cases involve both (AIID-1108: fake license number PLUS parasocial dynamic).

### NONE (Microsoft)

**Rule.** Microsoft NONE is used when no failure mode in Microsoft's 27-mode taxonomy applies. In our sample, this occurs primarily for training-data-poisoning incidents that Microsoft's whitepaper explicitly places out of scope.

**Application.** AIID-729 (GPT-4o Chinese tokens compromised by spam/porn): training data contamination is upstream of MSFT-AIRT's operational scope.

---

## NIST AI RMF Boundary Rules

### MEASURE vs. GOVERN

**Rule.** MEASURE applies when the incident implicates a failure of monitoring, metrics, or quantification of a known risk. GOVERN applies when the incident implicates a failure of accountability, policy, role clarity, or organizational culture.

**Application.** Autonomous vehicle safety incidents where the failure would have been detected by better pre-deployment safety monitoring → MEASURE. AI coding agent 'MJ Rathbun' publishing accusatory blog (AIID-1373): GOVERN — the central question is accountability for autonomous agent outputs, not measurement.

### MANAGE vs. GOVERN

**Rule.** MANAGE applies when the failure phenomenon involves inadequate response mechanisms — rollback, throttling, incident containment. GOVERN applies when the failure phenomenon involves absent or unclear organizational structures for AI oversight.

**Application.** Waymo power outage gridlock (AIID-1326): MANAGE — the safety-mode fallback failed to prevent secondary harm. AI toy safety failures (AIID-1277): GOVERN — the vendors lacked adequate governance structures for children's AI products.

### NIST NONE

**Rule.** NIST NONE is reserved for incidents where no NIST function is meaningfully implicated. In our sample, NIST NONE appears in 5 cases — the four all-framework-NONE cases discussed in §6.2 plus one case where the incident description was too thin to identify a lifecycle stage.

---

## Multi-Mode Assignments (Microsoft)

Microsoft-AIRT's own guidance permits multiple failure modes per incident. We list additional modes when the incident clearly implicates them, with the first-listed mode taken as the "dominant" mode for kappa purposes. Ordering follows this rule: (1) most central to the failure phenomenon, (2) most likely to be identified by a first-time coder without extensive familiarity, (3) alphabetical by mode name if the first two rules yield a tie.

---

## Reconciliation Log (Pass 3)

Nine disagreements (eight OWASP primary-category disagreements plus one NIST function disagreement) between Pass 1 and Pass 2 were reconciled by Pass 3 review with the following outcomes (Pass 1 retained unless noted):

| AIID | Pass 1 | Pass 2 | Pass 3 (reconciled) | Rationale |
|---|---|---|---|---|
| 1026 | ASI10 | ASI09 | ASI10 | Consistent LLM behavior across users; drift within session dominates |
| 794  | ASI08 | ASI07 | ASI08 | Cascading nature (weeks of behavior) dominates over single-message framing |
| 578  | ASI01 | ASI04 | ASI01 | Fine-tuning-for-harmful-purpose is closer to goal manipulation than supply chain |
| 850  | ASI09 | ASI10 | ASI09 | User-generated impersonation of deceased public figure is trust exploitation |
| 956  | ASI06 | ASI04 | ASI06 | Training data contamination fits memory/context poisoning framing |
| 1157 | ASI09 | ASI01 | ASI09 | Child-safety failure via bypass; trust dimension central |
| 881  | ASI08 | ASI07 | ASI08 | Inter-vendor coordination absence; cascading framing preferred |
| 1061 | ASI06 | NONE  | ASI06 | AI attribution uncertain but sufficient for retention as context poisoning candidate |
| 563  | NIST NONE | NIST GOVERN | NIST NONE | Retracted attribution; no confirmed AI failure locus |

The Pass 3 reconciliation retained Pass 1 for all nine disagreements, indicating that the compressed-timeline Pass 2 was subject to some second-look ambiguity but did not identify a superior categorical judgment on any incident. AIID-563's NIST function is retained as NONE per the reasoning that no confirmed AI failure locus is present in that incident.

---

## Files in This Codebook Directory

- `codebook.md` — this file
- `../data/coded_workbook.xlsx` — full 131-row coding sheet including Pass 1 codes
- `../data/analysis_summary.json` — computed frequency and coverage statistics
- `../data/kappa_results.json` — Cohen's kappa values by framework variable

---

## Version

- Version: 1.0
- Coder: Author (identity withheld for double-anonymous review)
- Date: July 3, 2026
- Framework versions coded against:
  - OWASP Top 10 for Agentic Applications 2026 (December 2025)
  - Microsoft AI Red Team Taxonomy (April 2025)
  - NIST AI RMF 1.0 (2023) + Generative AI Profile (2024)
