EXP-012: Bedrock Claude Opus 5.5 computer-use adapter, early real-endpoint calls failed, the reported five-call success has no artifact

In one paragraph

bedrock_cua is a separate route for evaluating a frontier model on our environments: Amazon Bedrock's Messages computer-use protocol with Claude Opus 5.5, one computer tool, screenshots in and GUI actions out, every action validated against a closed allowlist before it touches the desktop, every request and response written down. Its own doc is titled "mocked tests only, not yet run on the real endpoint or a desktop", yet the same doc records a few real calls: the first invoke returned a screenshot action; a paired run that replaced older screenshots was rejected with ValidationException: invalid signature in thinking block; and "The earlier history_n=5 runs remain failed runs (endpoint_error) and are not replaced." Those are the real-endpoint results on the branch, all author-claimed and all failures or protocol observations. A separate handoff document dated Oct 25 reports a successful five-call Opus 5.5 desktop interaction on both the QEMU/KVM and OCI paths. We cannot find any artifact for it anywhere. Until one is published it is an unverified report, not a result.

Setup

Pins (`bedrock_cua/config.py`)

MODEL_ID = "us.anthropic.claude-opus-5-5" (the US geo inference profile); freeze refuses any other id, and a test shows global.anthropic.claude-opus-5-5 rejected.
ANTHROPIC_VERSION = "bedrock-2023-05-31", ANTHROPIC_BETA = "computer-use-2025-11-24", TOOL_TYPE = "computer_20251124". Only the computer tool is sent: no bash, no text editor, no tool_choice.
Effort: allowed low|medium|high|xhigh|max, official default medium, frozen at max for both arms with a recorded source note. No thinking parameter is sent ("adaptive, always on"); effort goes in output_config.
HISTORY_MODE = "append_only": every request carries the full earlier conversation unchanged, all screenshots and all assistant blocks, including thinking blocks with their signatures, appended verbatim. No history window, no screenshot pruning, no retries, no block dropping.
Defaults (design values): region us-east-1, max_tokens 16000, max_steps 15, sleep_after_execution 3.0 s, screen 1920x1080. The frozen config is hashed; an episode refuses a config whose hash differs.
API: bedrock-runtime InvokeModel with a plain system prompt that tells the model to use only the computer tool and to stop calling tools when done.

What it refuses

A root principal: sts get-caller-identity runs first and RootCredentialsRefused is raised if the ARN ends with :root or is empty; only the account id and principal kind are recorded.
Credentials in code: "No key, token or environment variable is read or written by this code (a test scans the source)."
Training use: TRAINING_USE = "EXCLUDED: outputs and trajectories of the frontier model must not be used for RL training (approved terms)"; a NO_TRAINING_USE file is written in every artifact directory and the label is on every record and every traj.jsonl line.
Actions outside the allowlist: screenshot, left_click, right_click, middle_click, double_click, triple_click, mouse_move, left_click_drag, scroll (up/down), type, key, hold_key, wait. Zoom, mouse down/up, horizontal scroll and cursor_position are refused explicitly. Each action becomes a flat pyautogui script that must also pass the harness's literal-only validator; the model's text is only ever an inert string literal. A rejected action ends the episode as action_rejected.
Observation PNGs that are not exactly the frozen size (backend_error).
A run without a preflight.json that passed for the same contract hash (added at 71156fa, 20:30 UTC).
Any claim: done_claimed and max_steps are the only non-failure endings; every record carries "claim": None and the label "BEDROCK COMPUTER-USE EPISODE RECORD - not a benchmark result; the checker has not been run".

The episode loop this adapter implements: screenshot in, one or more tool_use actions out, each validated and executed through the slot's guest server, the new screenshot returned as a tool_result, the whole history replayed unchanged on every call. The checker runs outside the loop, never inside it.

Preflight

python -m bedrock_cua preflight makes one bounded real-endpoint conversation of at least six turns in which the model is told to use only screenshot, each answered with a saved 1920x1080 PNG (no desktop, no GUI action). It checks that model, tool, effort, system prompt and max_tokens never change; that every request begins with exactly the previous request's messages, byte-identical; that a signed thinking block was replayed and accepted; and that at least six screenshots were in the last accepted request. One attempt per call, no retry, stops at the first failure. "The mocked tests include a binding stand-in that rejects a rewritten history, but only the real endpoint run proves the contract."

What ran

Per the branch, author-claimed, no artifact:

The first real invoke returned name: computer, input.action: screenshot (BEDROCK.md).
The operator inspected native 1920x1080 guest screenshots on both the VM and OCI arms; "the model's click accuracy at that size is not yet measured" (BEDROCK.md).
A real paired run that replaced older screenshots was rejected: ValidationException, "invalid signature in thinking block" (BEDROCK.md, config.py HISTORY_SOURCE, commit d7181c7). That observation is why history became append-only.
"The earlier history_n=5 runs remain failed runs (endpoint_error) and are not replaced" (BEDROCK.md).
No preflight has been recorded; no preflight.json exists on the branch, and after 20:30 UTC run refuses without one.

Results

No measured result. The real-endpoint outcomes on the branch are: one observed screenshot action on the first call, one ValidationException after a history trim, and earlier runs recorded as endpoint_error failures. No episode reached done_claimed or max_steps on the branch, no checker was run, and no episode.json, traj.jsonl, screenshot, preflight or receipt exists.

Unverified report: a five-call success on both arms

An Oct 25 handoff document, separate from this branch, reports a successful five-call Opus 5.5 desktop interaction on both the QEMU/KVM and OCI paths: the folder rename task, with the checker passing and the slot resetting cleanly. We searched the whole branch checkout for "opus", "handoff", "five call", "5 call" and "renam" and listed every non-code file. No episode record, screenshot, preflight, receipt, history file or log of any Bedrock episode exists on this branch, and no sentence on this branch asserts a successful Opus 5.5 episode, a passed checker or a clean reset. The branch's own statements about real Bedrock calls describe failures and one screenshot action. The branch's run command also requires a passed preflight that does not exist, so a compliant run after 20:30 UTC would have left a preflight.json.

Until an artifact is published, the five-call success is an unverified report and not a result. What would make it a result: the episode.json and traj.jsonl for each arm with the frozen config hash, the per-step screenshots, the preflight.json the run was bound to, the checker's pre and post outputs recorded by the harness (not by the adapter), the reset state-hash equality, and the arm's declared differences. If those exist on another branch or on the operator's machine, they should be committed under an evidence directory and this record updated.

Failures and surprises

Trimming history breaks the thinking-block binding on Bedrock. The AWS page on thinking-block binding requires the earlier prefix to stay unchanged; the adapter observed the rejection and moved to append-only history. This means context grows with every step, and cost and latency with it; the branch does not estimate that.
The adapter's __init__.py and tests say "Nothing in this package has been run against the real endpoint or a real desktop", and BEDROCK.md reports real calls in the same file that is titled "not yet run on the real endpoint". Both are on the branch; the specific statements are what we report, labelled author-claimed.
The exact action parameter set for computer_20251124 is validated defensively from the public reference handler and SDK types; the doc marks the full parameter set and repeat, zoom, left_mouse_down, cursor_position as "Still UNVERIFIED from here".

Limitations

No measured number of any kind: no token usage, latency, steps or outcome on the branch. Usage fields in the record schema are the endpoint's own and stay None when not reported; api_seconds is kept separate from environment_seconds.
Click accuracy at native 1920x1080 is not measured.
The route is for evaluation only; its outputs cannot be used for training under the approved terms, which is why Experiment 2 (EXP-013) uses it for measurement and Experiment 3 (EXP-018) uses an open-weight model.

Reproduce

python -m bedrock_cua freeze CONFIG.json                                  # writes the frozen, hashed config
python -m bedrock_cua preflight --config FROZEN --screenshots DIR_OF_1920x1080_PNGS --out NEW_DIR --turns 6
python -m bedrock_cua run --config FROZEN --preflight NEW_DIR/preflight.json --port N --out DIR

Needs a non-root AWS principal with Bedrock access in us-east-1 and a slot whose guest server answers /screenshot and /execute. The checker and reset are run by the harness around the episode, not by the adapter.

Sources

benchmark/bedrock_cua/BEDROCK.md
benchmark/bedrock_cua/config.py, client.py, preflight.py, actions.py, run.py, __init__.py
benchmark/tests/test_bedrock_cua.py (mocked; "not evidence about the real endpoint")
benchmark/README.md, section "Bedrock Claude Opus 5.5 computer-use route"
Commits 047462b, e05fe6b, d7181c7, 71156fa, 3870385, cf2adc4 (2026-10-25)
The Oct 25 handoff document is not on this branch; its claim is recorded above as unverified.