macOS probes A, B and C, gates, sweep, soak and cost per valid rollout (the Experiment 1 headline)

In one paragraph

This is the experiment the harness in EXP-001 was built for, written down before it runs. Four probes answer the open risks first: can many background sessions be logged in and still painted (A), do keystrokes, pasteboards and modifier keys stay inside their sessions (B), does Chrome keep painting in a background session (C), and do our detectors fire when a leak is forced (P, see EXP-003). A fail in A, B or C stops the macOS arm; a fail in P means the detector is fixed before anything else. Then two gates at N equals 1, a sweep that climbs through N equals 1, 4, 8, 16, 24 and 32 with two 15-minute windows at each N, a 60-minute soak at the largest confirmed N, the isolation tests after every window, and a cost per valid rollout at three host prices. Every threshold below is a design value fixed in advance. None of it has run. No number on this page is a measurement.

Setup

• Machine: studio-a, after Day 0 (EXP-002) is complete, which it is not: 33 environment accounts (env00 to env32) each logged in by hand with Accessibility and Screen Recording granted to envd, task coordinates measured once in env01's session, the verify server running, and the console parked at the login window.
• Policy: a scripted oracle, not a model. The result is about the infrastructure, not about any agent.
• Tasks: the four in EXP-001: writer_save, browser_form, clipboard_cross, visibility, round-robin per environment, environments started out of phase so that idle time is bounded by construction (a design bound, not a measurement).
• Canary: env00 never receives an action and is watched the whole time.
• Sampler: host memory, pressure, swap, per-uid footprint, WindowServer and replayd CPU throughout; watts only if a second sampler runs as root.

What ran

Nothing. The dry-run paths of probes A, B and C are exercised by unit tests with made-up data (drumsctl/tests/test_probe_scripts.py). Every sweep, gate and soak code path runs in tests against a fake envd. Six attempts at probe P failed to run (EXP-003).

Results

None. This section will hold, when the experiment has run:

• the probe verdicts A, B, C and P with their logs, including which Chrome flags probe C needed, if any;
• one row per N in the sweep: confirmed, flaky or failed, with the gate that failed;
• N_max, the largest confirmed N, resting on two separate windows;
• valid rollouts per host-hour at N_max, from the confirmed windows only (the soak's rate printed next to it and not priced);
• memory per environment from the sampler, against the 0-env baseline;
• step p95, reset ready p95 and screenshot p95;
• contamination as "0 of X" only next to a positive control column reading "Y of X with Y above 0"; otherwise the column reads not run and no zero is printed;
• USD per valid rollout at three prices;
• clicks per boot, counted by hand at the keyboard across the 33 accounts, paid again after every reboot;
• or, if no N passes, the fallback line stated in advance: "macOS N_max = 0 on build X, reason Y; Linux on the same Mac gives N." The report prints that line only from a finished sweep. A climb that stops at N equals 1 with isolation_unverified or lost rows means the harness could not verify a check, and is not a capacity result.

The protocol

Probes (Day 1, in order; make probes).

• Probe A, sessions: a person logs env01 to env08 in one at a time; after each login the script records the LoginDone count, WindowServer footprint and CPU, memory pressure, wall seconds, the number of permission prompts (a human count), a background screenshot checked for black, and whether the console still responds. Gate A: at least 4 environment sessions LoginDone with non-black, non-failed screenshots and a responsive console. With only two accounts created, Gate A cannot pass by definition; in the pilot we read its measurements and expand only on an explicit go.
• Probe B, leaks: env01 to env04 at once type a per-user tag into TextEdit for 60 s through the session tap and set their pasteboards to a per-user string, while the console keeps a TextEdit canary and a known pasteboard string; then env01 holds Shift, then Cmd, for 5 s each while env02 types lowercase letters. Gate B: every document holds only its own tag, every pasteboard its own string, canary unchanged, no modifier leak, and no typer or reader failed to run. An ax_not_trusted exit is a permission problem, not an isolation result.
• Probe C, painting: in env01 and env02, Chrome with no flags on the visibility page, Chrome with background-throttling flags off, then Writer typed into the whole time; 60 screenshots about 1 s apart, distinct count, and the server's visible-tick count. Gate C: in both environments at least one Chrome configuration gives at least 55 distinct frames of 60 and a visible tick that grew; if only the flagged Chrome passes, required_flags is recorded and every table says so.
• Probe P, positive control: Gate P as in EXP-003.

Gates at N equals 1 (make gate1, make gate2). Gate 1: every envd healthy with ax_trusted, screen_recording and reset_kills true and a non-black screenshot; TextEdit typing and pasteboard round trips; screenshot p95 under 200 ms over 20 captures. Gate 2: 20 of 20 valid Writer and 20 of 20 valid browser rollouts, reset ready p95 at most 5000 ms, 20 of 20 stale-generation calls refused.

Sweep (make sweep NS=1,4,8,16,24,32). Two 15-minute windows at every N; isolation tests 2, 3, 4, 5 and 7 after each window's rollouts; canaries and sampler throughout. An N is confirmed when every window passed all gates, flaky when some did, failed when none did. The climb stops at the first non-confirmed N. N_max is the largest confirmed N. The N equals 1 baseline for the step gate is the median step p95 over all N equals 1 windows.

Per-window gates (pre-registered exit criteria, all design values).

A missing number fails its gate.

Soak (make soak N=<N_max>). 60 minutes at N_max, then 100 stale-generation trials (isolation test 10; applied_stale must be 0) and a 10-trial pasteboard check (test 2). Fails on any lost trial.

On demand (make positive-control, make isolation-extra). drumsctl's positive control, pasteboard half only on the Studio; isolation tests 8 (files and prefs), 9 (reset during write) and 11 (crash blast radius) at N_max.

Day-plan gates 3 to 6. The N equals 4 row confirmed 2/2 with all four tasks; tests 8, 9 and 11 at N equals 4 exit 0; dialog and crash bystanders within 20%; the sweep table complete with N_max on two windows; the soak with contamination 0, stale applied 0 of 100, and rollouts per host-hour recorded.

Cost (make report). usd_per_valid_rollout = host_usd_per_hour / valid_rollouts_per_host_hour, with the rate measured over the confirmed windows at N_max, at three host prices: an owned Studio amortized over a fixed term plus an energy term (the purchase price is a placeholder until the invoice; watts need the root sampler, else the energy term is left out and the table says so), a cloud Mac on demand (vendor claim with a 24-hour minimum), and a hosted Mac mini priced in euros at the run day's rate (vendor claim; null until the rate is passed in). cost.json copies every input next to its result. Model inference is in no price.

Failures and surprises

None yet. The six failed probe P attempts are in EXP-003; the Day 0 findings that bear on this plan (no CGSession on this build, so the console is parked by hand; envd ad hoc signed, so grants may not survive a rebuild; endpoint protection that may skew reset timings) are in EXP-002.

Limitations

• Everything in the not-claimed list of EXP-001 applies in advance: not a security boundary, not model quality, not automatic session creation, not any app, not full state reset, not long-run stability, not headless unless rerun without a monitor, not a license opinion, not the best Linux, not default Chrome behaviour if flags were required, and zero only next to a positive control that fired.
• The positive control's scope, even when it fires: it shows the typing and pasteboard detectors can fire with interference forced inside one session; it does not show that a leak between sessions, or through another channel, would be caught.
• Numbers will be for one macOS build on one machine, an institution-managed Mac with MDM and endpoint protection whose agents we cannot remove.
• Thresholds were set before any measurement and were not chosen to be attractive. They will not be moved after the fact; a miss is reported as a miss.
• Gate 10, a second person reproducing two windows at N equals 1 and two at N equals 4 on a second Studio within 15% on memory per env, step p95, reset p95 and valid rollouts per host-hour, is part of the plan and has not been attempted.

Reproduce

Steps 1 to 18 of the reproduce table in EXP-001, from make setup-host through make results. The headline steps are make probes, make gate1, make gate2 COORDS=..., make sweep COORDS=..., make soak N=<N_max> COORDS=..., make positive-control COORDS=..., make isolation-extra N=<N_max> COORDS=..., make report EUR_USD=<rate> STUDIO_USD=<invoice>, and make reproduce on a second machine for Gate 10.

Sources

• Repository branch exp1-sessions-per-studio, head f0d01ce: README.md (the headline question, the gates, the reproduce table, the fallback line), probes/README.md and the four probe scripts, drumsctl/sweep.py, drumsctl/gates.py, drumsctl/isolation.py, drumsctl/canary.py, report/report/prices.yaml, report/report/not_claimed.md, docs/isolation-model.md.
• Related: EXP-001, EXP-002, EXP-003, EXP-017.