# Protocol Amendment v1.11: SJTU-Self-Deployed-Model Ecological-Validity Supplement

**Status:** frozen supplementary protocol; not part of the confirmatory mutation arm

**Freeze date:** 2026-08-10 (Asia/Shanghai)

## 1. Purpose and scope

This amendment adds a small, separately reported model-trajectory arm using the
Shanghai Jiao Tong University (SJTU) “Zhiyuan No.1” self-deployed model service.
The service runs on a Huawei Ascend 910B NPU cluster. The arm is intended to
test ecological validity: whether naturally generated agent trajectories from
this school deployment reach the already-frozen benchmark tasks and evaluator
blind-spot conditions.

It does **not** change the frozen confirmatory questions, mutation catalog,
benchmark scope, statistical analysis, fingerprints, or conclusions. It cannot
be used to repair, replace, or selectively extend the terminated v1.8 model
arm. All outcomes, including infrastructure failures and task failures, are
retained.

## 2. Provider identity and attribution

- Provider label: Shanghai Jiao Tong University (SJTU) “Zhiyuan No.1”
  self-deployed model service on Huawei Ascend 910B NPU.
- Base URL: `https://models.sjtu.edu.cn/api/v1`.
- Requested model for this supplement: `deepseek-reasoner`.
- Hardware note: the service uses Huawei Ascend 910B NPU rather than NVIDIA
  CUDA GPUs. Small FP8/BF16 or backend-operator rounding differences are a
  platform limitation to disclose, not evidence of a model-behavior mechanism.
- The platform requires the following attribution in the paper and any
  associated interface: “上海交通大学‘致远一号’提供智算服务。”
- The paper will distinguish SJTU's deployment from the model developer
  and any upstream model weights. We will report the requested alias and the
  returned `model` (and `owned_by`/equivalent metadata when supplied) without
  calling the response an independently verified official model weight.
- No API key, authorization header, or private request payload may appear in
  code, logs, screenshots, public artifacts, or the manuscript.

## 3. Research questions

**RQ-S1 (descriptive):** Among six preselected episodes, how often does the
agent served by the school deployment reach the benchmark's official accepted
state?

**RQ-S2 (descriptive):** Among completed episodes, are any harmful persistent
side effects observed outside the requested state?

**RQ-S3 (descriptive):** Do any completed trajectories enter the previously
identified ToolSandbox same-record or cross-namespace evaluator blind spots?

No null-hypothesis test or model ranking will be reported for this arm. Six
episodes are insufficient to support a general capability claim.

## 4. Frozen sample and execution rules

- Exactly six episodes: two tau2, two ToolSandbox, and two AppWorld episodes.
- The episodes are selected before any new school-service generation using the
  first two task indices of each benchmark in the existing frozen v1.8 task
  catalog.
- One alias only: `deepseek-reasoner`. `deepseek-chat`, MiniMax, and Qwen
  aliases are not added unless a new amendment is written before generation.
- Maximum 12 agent turns per episode and at most the existing transport retry
  policy. No reruns, task replacement, prompt changes, model switching, or
  post-outcome sample expansion.
- Because the service documents a 10-request/minute limit, the live runner
  waits at least 6.2 seconds between SJTU model requests and at least 7 seconds
  between episode processes. The fixed Qwen simulator is not throttled by this
  rule.
- The fixed QwenPlus user simulator remains unchanged for tau2 and ToolSandbox.
- A run stops after two consecutive infrastructure failures. The append-only
  log remains resumable, but an incomplete arm is excluded from all claims.
- The local combined agent-plus-simulator cap is 250,000 tokens. This is a
  safety cap, not a claim about the school's service quota.

## 5. Recorded fields and analysis

For every request/episode, record the input hash, prompt hash, requested alias,
returned model, provider hostname, timestamp, response hash, parsing status,
official acceptance, side-effect classification, token usage, latency, retry
count, and error type. Redact credentials before persistence.

The supplement reports counts and confidence intervals only as descriptive
context. It cannot alter the 100/110 harmful-kill or 35/35 benign-preservation
results of the confirmatory arm, and it cannot raise the confirmatory sample
size after observing outcomes.

## 6. Go/No-Go gates

- **Go to the six-episode live arm:** metadata-only probe finds the requested
  alias or the school service documents that `/models` is unavailable while a
  documented OpenAI-compatible endpoint is reachable; the key is supplied via
  hidden input; no secret scan failure occurs.
- **No-Go:** any credential appears in output; the school's service terms
  prohibit this use; the endpoint cannot be authenticated; or the first two
  attempts are infrastructure failures.
- **Reporting gate:** if fewer than six episodes complete, report only an
  operational note and exclude all scientific interpretation.

## 7. Manuscript placement

The arm belongs in a clearly labeled supplementary/appendix subsection titled
“School-self-deployed-model ecological-validity supplement.” It must not be used in
the title, abstract, primary claims, confirmatory tables, or acceptance claim.
The required attribution is included in Acknowledgements/Data Availability and
in the experimental setup note.
