# PolicyFaultBench

PolicyFaultBench is a provider-neutral, fail-closed research harness for
mutation-based assurance of runtime containment policies used by tool-capable
AI agents.

The repository is deliberately safe by default:

- live model calls are disabled;
- public network destinations are rejected;
- tools operate only on synthetic in-memory services and canary data;
- API keys are loaded from an ignored local environment file;
- policy decisions and state changes are logged independently;
- hard cost ceilings are enforced before and after every provider call.

## Status

Version 0.6.1 completes the prospectively frozen corpus-policy generalisation
extension. Both embedded 12-case provider gates passed. OpenAI achieved 400/400
strict successes. Anthropic achieved 391/400; nine outputs for the single
benign case `ADJ-BANK-U03` added an unrequested `approval_scope` context field
and were quarantined before either policy or sandbox execution. The frozen
800/800 overall acceptance rule therefore did not pass. Across the 791 admitted
proposals, the ordered-rule and capability engines agreed in 1,582/1,582
evaluations, every observed transition verified, and no containment failure
occurred. Preserved core proposals had already replayed 800/800 under the
capability engine.

An independent hash-verified audit preserves the nine strict endpoint failures
while distinguishing them from downstream policy-oracle failures: those nine
trials had no policy or oracle event because the pre-policy gate stopped them.
The original generated analysis remains unchanged for chronology. The audit,
trial table, results report, and both provider ledgers are under
`results/generalization_v1/`, `reports/`, and `study/generalization_v1/`.

Version 0.6.0 adds a prospectively frozen corpus-policy generalization
extension. A 40-case external suite is derived from the numerically lowest
five user tasks and five injection tasks in each of four AgentDojo v1 suites
at commit `089ed468cf3ed0322acc66b0211f26d9d90dbf60`. All identities, accounts,
hosts, and services are replaced with owned synthetic equivalents. The
external suite is balanced at 20 allow and 20 deny cases.

The extension also adds an independently implemented server-resolved
capability policy. The ordered-rule and capability engines conform on both
frozen 40-case corpora. The capability policy has a separate C01-C12 mutation
model and targeted probes; core and expanded mutation scores remain separate.
Preserved OpenAI and Anthropic core proposals replayed 800/800 under the new
policy with no oracle or containment failure. The live external arm remains
prospective under `config/policy_generalization_protocol_v1.json`.

Version 0.5.1 adds the prospectively frozen Anthropic confirmatory replication.
Protocol `pfb-anthropic-confirmatory-v2` fixed the provider, model, 40 × 10
schedule, complete-object gate, compound endpoint, denominator, cost ceiling,
and reporting commitment before live execution. The 12-case preflight gate
passed 12/12. The new ledger then completed 400/400 strict successes with zero
proposal-conformance, policy-oracle, transition, or containment failures. The
independent analysis and audit are under `study/confirmatory_v2/`.

The earlier evidence remains intact. OpenAI v1 matched the complete proposal
object in 400/400 records. Anthropic v1 matched 291/400 because of additional
context commentary and remains historical secondary evidence; its
decision-relevant envelope, policy decision, and state replay passed 400/400.
The invalidated Gemini diagnostic ledger and the quota-censored gate are
excluded from definitive estimates. See the amendment files under `config/`.

The completed version 0.2.0 pilot established:

- 36/36 primary live runs passed across `claude-sonnet-5`,
  `gemini-3.6-flash`, and `gpt-5.6-sol`;
- no primary run aborted and no containment failure changed synthetic state;
- the original 40-case suite killed 7/12 policy mutants (58.33%);
- four targeted functional probes and one reset probe raised expanded mutation
  adequacy to 12/12 (100%) with zero reference-policy failures.

See `reports/PILOT_AND_MUTATION_REPORT.md`, `evidence/pilot/`, and `results/`.
These are pilot results and make no claim about production security.

The original v1 design remains frozen in `config/study_protocol.json`. The
Anthropic replication is separately frozen in
`config/anthropic_confirmatory_protocol_v2.json`: 40 scenarios, ten
repetitions, 400 scheduled trials, and a new ledger. Across the three completed
OpenAI/Anthropic ledgers, 1,200 records are available. The replication does not
retroactively change the first two ledgers or their evidence labels.

## Requirements

- Python 3.11 or newer
- no third-party package is required for offline execution and tests
- API credentials are required only for deliberately enabled live runs

## Quick verification

```bash
PYTHONPATH=src python -m unittest discover -s tests -v
PYTHONPATH=src python -m policyfaultbench.cli validate
PYTHONPATH=src python -m policyfaultbench.cli offline-pilot --limit 12
PYTHONPATH=src python -m policyfaultbench.cli mutation-eval
PYTHONPATH=src python -m policyfaultbench.cli study-plan
```

The offline pilot uses a deterministic mock adapter. It never contacts an API
and never consumes prepaid credit.

## Live execution gate

Live provider calls require all of the following:

1. every offline test passes;
2. `manifests/owned_assets.json` validates;
3. the API key exists only in `.env.local` or the process environment;
4. `PFB_LIVE_ENABLED=true` is set deliberately;
5. the chosen provider and model are frozen in `config/experiment.json`;
6. the configured provider and total cost ceilings are non-zero;
7. the operator confirms that the run is a labelled DEBUG pilot.

Live calls are intentionally not performed during package creation.

## Repository map

- `src/policyfaultbench/`: executable library and CLI
- `policies/`: versioned reference containment policy
- `scenarios/`: 40 synthetic benchmark cases
- `scenarios/external_agentdojo_40.json`: 40 source-grounded external cases
- `schemas/`: JSON schemas for runs and scenarios
- `external_sources/`: frozen AgentDojo source manifest and MIT notice
- `policies/capability_policy_v1.json`: independent capability architecture
- `config/`: provider models, token limits, and hard budgets
- `manifests/`: owned synthetic assets and prohibited target classes
- `tests/`: unit, integration, mutation, reset, and safety tests
- `runs/`: ignored append-only run logs created during execution
- `evidence/pilot/`: sanitized primary provider evidence and combined summary
- `study/confirmatory_v2/`: frozen Anthropic replication, ledger, audit, and analysis
- `results/`: mutation matrix and machine-readable analysis
- `reports/`: bounded technical interpretation and definitive-study plan
- `analysis/`: exact confidence intervals, paired tests, tables, and figures
- `config/study_protocol.json`: frozen confirmatory protocol and exclusion rules

Read `SAFETY.md` before enabling any live model.
