DrumsBench

Experiments / EXP-003

Positive control probe P on the Studio: six attempts, failed to run

Make the typing and pasteboard detectors fire on studio-a with two Writer instances forced into one session on the tap route (Gate P requires typing_cross above 0 and pb_cross above 0).

failedRan and missed a correctness, isolation, reset or pre-registered performance bar. Kept on the record.
Date
2026-10-23
Arms
macos-sessions
Hardware
Mac Studio M4 Max, 16 cores, 48 GB (studio-a)
OS
macOS 15.7.5 (24G624)
Workload
probes/probe_p_positive_control.sh with probes/positive_control.swift; LibreOffice Writer 26.8.1 and the TextEdit fallback
Code
exp1-sessions-per-studio at f0d01ce
Evidence
results/probes-pilot-2-sessions/probes.md on studio-a (pilot results folder set by DRUMS_RESULTS_DIR; not on the branch)
data/results/exp-003-probe-p-attempts.json
Limitation
The program stopped at its preconditions (window focus, Accessibility) on all six attempts and never typed or set a pasteboard; this is a failure to run, not a detector verdict, and the positive control has still not fired on any machine on its route of record.

In one paragraph

A positive control is a run where we force the thing our detectors are supposed to catch, and check that they catch it. For this experiment that means two LibreOffice Writer instances in one login session, typed into at once and racing on one pasteboard, so that cross-typing and pasteboard crosses must appear. If the detectors see them, a later zero across separate sessions means something. If the detectors never fired, a zero is indistinguishable from a broken detector. Between 2026-10-23 02:58Z and 06:17Z we tried to run this control, probe P, six times on studio-a. All six ended with Gate P: FAIL and exit code 2, the program's "could not run" code. Five attempts on Writer stopped at the no_focus guard: the program could not put Writer in front, and the tap route types only into a front instance. One attempt on the TextEdit fallback stopped at ax_not_trusted: the terminal running it had no Accessibility grant. No typing happened, no pasteboard was set, and the detector halves report 0 of 0 reads. The status is failed: it failed to run at its preconditions, not a detector verdict. As a consequence no isolation zero can be published from this harness yet.

failed

Setup

What probe P does when it runs

probe_p_positive_control.sh starts two Writer instances in one login session, each with its own UserInstallation. The typing half types into both at once with disjoint alphabets (one side a to m, the other n to z); typing_cross counts runs of 3 or more foreign letters and cross_chars every foreign letter. The pasteboard half runs 20 rounds in which one side sets the pasteboard, the other sets it within 100 ms, and the first reads back; pb_cross counts reads that return the other side's string. Gate P passes when typing_cross is above 0 and pb_cross is above 0.

The route of record is tap, CGEvent.post on the session event tap, the same call envd makes. That is a recorded deviation from the design, which named CGEventPostToPid: the pid route had measured typing_cross 0 on the author's laptop, so it cannot serve as the typing control.

POSITIVE_CONTROL_APP=TextEdit is a fallback for a machine without LibreOffice. The probe's own documentation says of it: "That checks the detector, it is not a result."

What ran

Six attempts, 2026-10-23 02:58Z to 06:17Z, each recorded as one verdict row in probes.md.

attemptsapprouteexiterrorwhat the program said
5Writertap2no_focuscould not put LibreOffice Writer in front; the tap route types only into a front instance
1TextEdittap2ax_not_trustedgive the terminal Accessibility (System Settings, Privacy and Security), then run again

For every attempt: typing half fired false, pasteboard half fired false, 0 of 0 pasteboard reads.

Positive control probe P on studio-a: six attempts, none ran to completion failed2026-10-23 · studio-a

MetricValueUnitnNote
probe P attempts6attemptsall Gate P: FAIL, all exit 2; five with app=Writer route=tap, one with app=TextEdit
attempts that ran to completion0 / 6attemptsfive stopped at no_focus (could not put LibreOffice Writer in front; the tap route types only into a front instance); one stopped at ax_not_trusted (the terminal running it lacked Accessibility)
typing half fired0 / 6attemptsnever started on any attempt; typing_cross was not measured
pasteboard half fired0 / 6attemptsnever started on any attempt; 0 of 0 reads; pb_cross was not measured

Host: Mac Studio M4 Max, 16 cores, 48 GB, macOS 15.7.5 (24G624). An institution-managed Mac with MDM and endpoint protection. LibreOffice 26.8.1 installed. Run from the console session over SSH; the terminal had no Accessibility grant; no env session logged in.. Workload: probes/probe_p_positive_control.sh with probes/positive_control.swift: two Writer instances (or the TextEdit fallback) forced into one login session, typed into at once with disjoint alphabets and racing on one pasteboard. Method: Six attempts between 2026-10-23 02:58Z and 06:17Z, each writing one Gate P verdict row to probes.md in a pilot results folder set by DRUMS_RESULTS_DIR. Exit 2 is the program's could-not-run code (no Accessibility, app missing, no window, bad settings); it is neither a detector pass nor a detector fail. Rows below count attempts.. Source: results/probes-pilot-2-sessions/probes.md on studio-a (pilot results folder; not on the branch).

  • A failure to run at its preconditions (window focus, Accessibility), not a detector verdict. Nothing here says anything about isolation.
  • The positive control has not fired on any machine on its route of record (tap). Until it does, no isolation zero may be published from this harness.
  • The pid-route numbers from the author's laptop (pb_cross 20 of 20, typing_cross 0; macOS 27.0, TextEdit, 6 s) are a different route, app, machine and OS and are not a result.
  • The two error classes are no_focus (5 attempts) and ax_not_trusted (1 attempt); both have known fixes that need a person at the machine.

Results

Exit 2 is the program's code for "could not run": no Accessibility, app missing, no window, bad settings. It is neither a detector pass nor a detector fail, and it says nothing about isolation between sessions.

Failures and surprises

Limitations

Reproduce

  1. Complete Day 0 (EXP-002): LibreOffice installed, envd built.
  2. Sit at the Studio in the console session, in a terminal that has Accessibility.
  3. export RUN=results/<date>-<host>-mac; DRUMS_HOST_OK=1 make probes runs A, B, C and P in order, or run probes/probe_p_positive_control.sh alone with DRUMS_HOST_OK=1.
  4. Read results/probes/probes.md for the Gate P line and results/probes/positive_control.jsonl for the counts. A pass needs typing_cross above 0 and pb_cross above 0 on the tap route.
  5. For the pilot, set DRUMS_RESULTS_DIR to a pilot folder so the verdicts stay out of the folder the report reads.

Sources