DrumsBench

Experiments / EXP-002

Studio Day 0: host setup passes 1 and 2, and the 0-env host baseline

Prepare studio-a for the Experiment 1 probes (tools, settings, envd, env accounts, golden tree) and take the 60-second host baseline with no environment session logged in.

preliminaryA real observation that needs more runs, a positive control, or an independent check before it counts.
Date
2026-10-22
Arms
macos-sessions
Hardware
Mac Studio M4 Max, 16 cores, 48 GB (studio-a)
OS
macOS 15.7.5 (24G624)
Workload
host setup scripts under scripts/mac, Gate 0, and the metrics sampler for 60 s
Code
exp1-sessions-per-studio at f0d01ce
Evidence
internal runbook archive (not on the branch): studio-day0/logs/20261022-024429/ (setup, users, accounts, agents, golden, seed, gate0, perms logs; status; run-info)
internal runbook archive (not on the branch): studio-day0/pass1-logs-recovered-from-transcript.txt and pass1-status.txt (pass 1; raw logs lost)
results/2026-10-22-studio-a-mac-pilot2/meta.json and host.jsonl (pilot folder on studio-a; copy in the runbook archive)
data/results/exp-002-host-baseline-0env.json
Limitation
Two of the planned 33 accounts exist, no session has been logged in, envd carries no permission grants, and the 16 GB baseline is a pilot-folder input that includes the machine's management agents; nothing here measures the environment primitive.

In one paragraph

On 2026-10-22, between about 01:15 and 02:55 Eastern, we prepared studio-a, an institution-managed Mac Studio with MDM and endpoint protection, for the Experiment 1 probes. The work ran over SSH in two passes. Pass 1 installed Homebrew, LibreOffice and UTM and set the power and multi-session settings; it failed at the code-signing trust step, which needs an on-screen dialog, and its raw logs were later deleted by mistake. Pass 2 installed envd ad hoc signed, created two of the planned 33 environment accounts (env01 and env02), installed their LaunchAgents without loading them, and seeded the golden tree and its clones. Gate 0 then returned 2, as expected with two accounts, and found that CGSession does not exist on macOS 15.7.5 (measured), contrary to what our earlier documents said. A 60-second host baseline with no environment session logged in recorded about 16.0 GB of 48 GB used, memory pressure normal and swap 0 across 12 samples (measured, pilot folder). The status is preliminary: the setup is incomplete, no session has been logged in, and the baseline is an input for a later report, not a per-environment number.

Setup

What ran

Pass 1 (01:52 to 01:55 Eastern)

setup_host.sh driven through a sudo askpass helper. Measured outcomes, from a transcript recovery because the raw logs were lost:

Pass 2 (02:44 to 02:47 Eastern)

Every step exited 0 except Gate 0, which exited 2 as expected. Measured, from the kept logs:

Gate 0

make gate0 returned 2. It printed: FileVault on; 2 of 33 env users; CGSession missing; no env00. With two accounts this is the expected code; the CGSession line is the finding.

The 0-env host baseline (about 02:47 Eastern)

make meta and make baseline-sample into a pilot run folder, results/2026-10-22-studio-a-mac-pilot2. The sampler ran as the operator, not as root, for about 60 s and wrote 12 samples. No environment session was logged in; the console session held a terminal.

Host baseline with no environment session logged in (pilot folder) preliminary2026-10-22 · studio-a

MetricValueUnitnNote
memory used16.04 (15.99 to 16.07)GB12of 48 GB; mem_used_bytes / 1e9; no environment session logged in
swap used0GB120 in every sample
WindowServer CPU12.9 to 21%12ps decaying average, not instantaneous
memory pressure normal12 / 12samplesmemory_pressure reported normal in every sample

Host: Mac Studio M4 Max, 16 cores, 48 GB, macOS 15.7.5 (24G624). An institution-managed Mac with MDM and endpoint protection whose agents were running throughout. One 1080p display attached. The console session held a terminal. No environment session was logged in.. Workload: idle host: no environment session, no envd running, no task; the metrics sampler only. Method: make meta, then make baseline-sample at about 02:47 Eastern on 2026-10-22, into the pilot run folder results/2026-10-22-studio-a-mac-pilot2. The sampler ran as the operator (not root) for about 60 s and wrote 12 host.jsonl lines. Memory in decimal gigabytes from mem_used_bytes; CPU percentages are ps decaying averages, not instantaneous readings.. Source: results/2026-10-22-studio-a-mac-pilot2/host.jsonl (pilot folder on studio-a; copy in the internal runbook archive); results/2026-10-22-studio-a-mac-pilot2/meta.json (same; its host field is not published).

  • A pilot folder, not the published run folder.
  • The 16 GB includes the institution's MDM and endpoint-protection agents and a console session with a terminal; no environment session existed. It is a baseline input the report subtracts from later windows, not a per-environment number and not an idle cost of the harness.
  • The design's idle-session cost (the footprint of a session's processes after login) was never taken, and the post-login baseline (step 9 of the reproduce table) does not exist.
  • Watts are null: powermetrics needs root and the sampler did not run as root.
  • Total CPU swung between 39.0 and 388.9 percent across the 12 samples (ps decaying averages, with setup work close in time) and is left off this table on purpose.

meta.json recorded build 15.7.5 (24G624), hardware Mac16,9 with 48 GB and 16 CPUs, git sha f0d01ce, and null for the Chromium path, the envd version and the baseline span, because no envd was running and the span is written after the logins.

Results

Failures and surprises

  1. CGSession is missing on macOS 15.7.5 (measured, Gate 0). Our earlier documents said it is present on macOS 15 (author claim). The measured finding wins: that claim is wrong on this build. make park-console and the CGSession line of Gate 0 fail. The console must be parked by hand through the user-switching menu. The probes do not use CGSession. Whether Screen Sharing survives a manual park is untested.
  2. Endpoint protection slows file and process operations (informal measurement). A 15-line script, run once while a test suite was running, measured process start at 5.3 ms on the Studio against 1.7 ms on the laptop, and file create, read and delete at 15.6 ms against 4.0 ms; Python start was about the same on both (17.9 against 23.4 ms). Different machines, different OS versions, one run, under load: this is a note about the environment, not a benchmark. Reset and APFS clone timings may be skewed by it. The suggested mitigation, an exclusion for /usr/local/drums and /Users/env*, was not done.
  3. Code-signing trust needs an on-screen dialog. security add-trusted-cert, both in the user domain and with sudo on the System keychain, failed with "The authorization was denied since no user interaction was possible." The setup script's claim that system-wide trust works over SSH is wrong on 15.7.5. The half-imported identity was deleted. envd is installed ad hoc signed instead, which means TCC grants will not carry across rebuilds the way a stable identity was meant to make them.
  4. Cask loop defect in setup_host.sh. The loop fails when an app already exists outside Homebrew, here the institution-managed Chrome. brew install --cask --adopt was tried and rejected, because it wants to change the group of the managed app. Worked around in the Day 0 driver; the fix (skip a cask whose /Applications app is already present) is recorded for later and not made.
  5. umask bug in the Day 0 driver (internal tooling, not in the repository). The driver set umask 077 for its secret files and the setup scripts inherited it: /usr/local/drums came out mode 0700 and /opt/homebrew lost world access. Repaired by hand: chmod o+rx on 843 directories and o+r on 5,616 files under /opt/homebrew; /usr/local/drums set to 0755 in pass 2. The driver was fixed (077 only around the secret files, 022 for the rest) and tested locally with a fake sudo.
  6. Pass 1 raw logs were deleted by mistake (a remove ran before a move). What survives is a transcript recovery: the setup log with lines over 300 characters cut, and the last 40 lines of the brew log. Rule adopted immediately after, and kept since: every run keeps its raw logs under a timestamped folder, nothing deletes them, secrets are counted and redacted in place, and the folder is archived. Archive, never delete.
  7. FileVault is on. Setup stops unless DRUMS_ALLOW_FILEVAULT=1; a reboot needs a person at the machine for the pre-boot password. "Do not reboot studio-a" is the standing rule.
  8. uv picked Python 3.14.7 first. The suite was stopped and re-run on 3.12.14, pinned by an untracked .python-version.
  9. Two envd binary sizes exist (the release build and the ad-hoc-signed install). The difference is consistent with signing and was not analysed. Neither size is a figure worth publishing.

Limitations

Reproduce

  1. git clone <repo>; uv sync; make test; make build-envd (see EXP-001).
  2. DRUMS_HOST_OK=1 make setup-host. Expect the signing-trust step to need a person at the screen on 15.7.5; make signing-identity redoes that step alone. Expect the cask loop to fail if Chrome is already installed outside Homebrew.
  3. DRUMS_HOST_OK=1 make create-users (we created two accounts, not 33, on purpose).
  4. DRUMS_HOST_OK=1 make install-launchagent.
  5. make golden.
  6. DRUMS_HOST_OK=1 make seed-env-homes.
  7. make gate0. On this build expect CGSession missing.
  8. export RUN=results/<date>-<host>-mac; make meta; make baseline-sample before any env login. Run the sampler as root if watts are wanted.

Sources