DrumsBench

Experiments / EXP-004

Mechanism F: forking the OSWorld guest server from a warm zygote saved 6.2% of memory against a pre-registered 15% bar

Decide, against thresholds fixed before measurement, whether forking the Python guest server from a pre-warmed zygote cuts host memory per session by at least 15% and server-ready median latency by at least 30%, and record what breaks when a live GUI process is forked.

failedRan and missed a correctness, isolation, reset or pre-registered performance bar. Kept on the record.
Date
2026-10-23
Hypothesis
The author's own pre-registration expected (a) to fail because Chromium dominates per-session memory and is never forked.
Arms
l1-fork-prototype
Hardware
4 vCPU, 16 GiB, cgroup v1, no KVM (dev-sandbox)
OS
Ubuntu 24.04, kernel 6.18 (sandbox)
Workload
Drums L0 session with the real OSWorld guest server and Playwright Chromium 141 on a blank data-URL page; 20 sessions per arm (latency), 4 trials of 20 concurrent sessions per arm (memory)
Code
claude/linux-comparison-prep at dce0211
Evidence
docs/data/l1-prototype/bench.json
docs/data/l1-prototype/hygiene.json
docs/data/l1-prototype/hazard.jsonl
docs/l1-shared-runtime-design.md (sections 8 and 9)
docs/mechanism-f-fork-server.md
Limitation
Only the Python guest server was forked; Chromium was exec'd in every arm, so the reachable saving was bounded at about 22% before the experiment started, and the measured 6.2% is on one development sandbox with no incumbent arm.

In one paragraph

We asked whether a classic fork-server trick could make Drums sessions cheaper. A zygote process loads the OSWorld guest server's heavy Python libraries once; each new session is a fork of it, sharing that memory copy-on-write. The thresholds were committed first (commit 84a9241, 2026-10-23 16:27 UTC), the measurement 36 minutes later (dce0211). Measured: host memory per added session went from 231.15 MiB (C1, the fair cold baseline) to 216.83 MiB (F), a 14.32 MiB saving, ratio 0.938, bootstrap 95% interval [13.13, 15.55] MiB. The saving is real in direction but it is 6.2%, and the pre-registered bar was 15%. The server-ready median improved to 0.80 of the baseline where the bar needed 0.70. Two of four bars failed, so the mechanism failed. A separate hazard test showed five distinct ways a fork of a process that already holds X, D-Bus, socket and lock state breaks, which is why Chromium could never be forked this way. "Drums" throughout this record means the L0 prototype: Linux namespaces plus overlayfs plus Xvfb plus an openbox session on the sandbox's own Ubuntu 24.04 root filesystem. No shared-runtime (L1) result exists; every run records drums_level = L0.

Setup

Host. One development sandbox: 4 vCPU, 16 GiB, Linux kernel 6.18, cgroup v1, no KVM. Hostname and provider are not recorded in any evidence file. It is not a VM the team controls and not a cloud host.

What a session is. unshare of the PID, mount, IPC, UTS and network namespaces; an overlayfs full root whose lower layers are a template directory and the sandbox's own /; a private Xvfb at 1280x800; D-Bus, AT-SPI and openbox; the real, unmodified OSWorld guest server (main.py from xlang-ai/OSWorld at 04718ea); Playwright Chromium 141 started with --no-sandbox on a data-URL page. Chromium is started by exec in every arm.

Arms (identical in everything else, run order interleaved):

ArmGuest server start
C0As OSWorld ships it: python main.py, whose debug reloader runs two interpreters
C1A fresh interpreter importing the same module entry F uses, app.run(debug=False); this isolates the fork effect from the entry-point effect
FA fork of the warm zygote; main is imported after the fork because pyautogui connects to X at import time, so the safe fork point is before that import

The zygote pre-imports 16 modules (json, flask, werkzeug, jinja2, lxml, requests, PIL, tkinter, gi, pyatspi, pyscreeze, Xlib, psutil, ctypes). Measured zygote preload 0.221 s, RSS 56152 KiB, PSS 51448 KiB (one process, paid once per host).

Pre-registered thresholds (design values, commit 84a9241):

IDMeasurementBar
aMarginal host memory per added session, slope of (MemTotal minus MemAvailable) at 1, 4, 8, 12, 16, 20 sessionsF slope at most 0.85 x C1 slope, and the bootstrap 95% interval of (C1 minus F) over trials excludes 0
bServer-ready latency, session create to guest /platform answering, 20 per armF p95 below C1 p95 and F median at most 0.70 x C1 median
cApp-ready latency, create to Chromium page title visible over CDPF p95 below C1 p95
dSibling hygiene: distinct PID namespaces and pids, one Xvfb each, no shared listening socket, RNG state differsall hold
eHazard test: fork a running server holding X, D-Bus, socket and lock stateno bar; every breakage reported
fShare bound: how much of per-session memory fork can reach at allno bar; reported

What ran

envd_linux.l1proto.bench ran the three arms: 20 sequential sessions per arm for latency after 2 discarded warm-ups with the arm order rotated, and 4 trials per arm of 20 concurrent sessions for memory with sessions destroyed between trials. envd_linux.l1proto.hygiene checked two simultaneous forked sessions. envd_linux.l1proto.hazard_running_fork forked a stand-in process that holds what the live guest server holds. The raw files are bench.json (12.8 KB), hygiene.json and hazard.jsonl (12 lines) under docs/data/l1-prototype/.

Results

All numbers below are measured unless labelled otherwise.

Mechanism F: fork the OSWorld guest server from a warm zygote (C0, C1, F arms) failed2026-10-23 · dev-sandbox

MetricValueUnitnNote
host memory per added session, C0 (as shipped)280.96 (279.87 to 282.2)MiB4per-trial slopes; n = trials of 20 sessions
host memory per added session, C1 (module entry, cold)231.15 (230.17 to 232.59)MiB4the fair baseline for F
host memory per added session, F (forked)216.83 (215.69 to 218.08)MiB4
C1 minus F14.32 (13.13 to 15.55)MiB4min/max are the bootstrap 95% interval over trials (excludes 0)
saving F vs C16.2%F/C1 = 0.938
pre-registered memory bar (a)15%design value, commit 84a9241: F slope at most 0.85 x C1 slope; FAILED
server-ready latency, C01.53 (p95 1.62)s20
server-ready latency, C11.08 (p95 1.18)s20
server-ready latency, F0.862 (p95 0.93)s20F p50 / C1 p50 = 0.80; bar (b) needed at most 0.70; FAILED on the median, p95 condition held
Chromium-ready latency, C11.43 (p95 1.53)s20
Chromium-ready latency, F1.21 (p95 1.33)s20bar (c) F p95 below C1 p95: PASSED; the gain is the server's, the Chromium part is the same in every arm
create() latency, C12 (p95 2.09)s20
create() latency, F1.74 (p95 1.88)s20
zygote preload time0.221s1paid once per host; zygote RSS 56152 KiB, PSS 51448 KiB
forkable Python memory per session (guest server + init, C1 private)51.7MiB142184 + 10708 KiB private at N = 20, trial 0; the share bound: at most about 22% of 231 MiB even if wholly shared
Chromium private memory per session (10 processes, C1)90.9MiB193100 KiB; exec'd in every arm, never forked
pre-registered bars passed (a, b, c, d)2 / 4barsa memory FAIL, b server-ready median FAIL, c Chromium-ready PASS, d sibling hygiene PASS

Host: 4 vCPU, 16 GiB, cgroup v1, no KVM, Ubuntu 24.04, kernel 6.18 (sandbox). one development sandbox; hostname and provider not recorded; not a VM or cloud host. Workload: Drums L0 session (namespaces + overlayfs + Xvfb + openbox) with the real OSWorld guest server and Playwright Chromium 141 (--no-sandbox) on a blank data-URL page; 20 sessions per arm for latency, 4 trials of 20 concurrent sessions per arm for memory. Method: Memory: slope of (MemTotal - MemAvailable) against concurrent session count at 1, 4, 8, 12, 16, 20 sessions, 4 trials per arm; the 95% interval of (C1 - F) is a bootstrap over trials. Latency: 20 sequential sessions per arm after 2 discarded warm-ups, arm order rotated. C0 = guest server as OSWorld ships it (python main.py, debug reloader); C1 = fresh interpreter with the same module entry F uses; F = guest server forked from a warm zygote. Thresholds fixed in commit 84a9241 before measurement; results in dce0211.. Source: branch claude/linux-comparison-prep: docs/data/l1-prototype/bench.json (memory_summary, latency_summary, breakdown, zygote); branch claude/linux-comparison-prep: docs/data/l1-prototype/hygiene.json; branch claude/linux-comparison-prep: docs/data/l1-prototype/hazard.jsonl; branch claude/linux-comparison-prep: docs/l1-shared-runtime-design.md sections 8 and 9.

  • Only the Python guest server is forked (fork point before import pyautogui); Chromium is exec'd in every arm.
  • One development sandbox, one kernel; intervals are over 4 trials on that host, not over hosts.
  • Drums here is the L0 prototype; no shared-runtime (L1) result exists; drums_level = L0 in every run.
  • No incumbent was measured; nothing here compares Drums with a VM or container baseline.
  • The hazard test forked a stand-in process with live X, D-Bus, socket and lock state, not Chromium itself; the Chromium conclusion is an inference.

Verdict against the bars.

BarMeasuredVerdict
a memoryC1 231.15 MiB, F 216.83 MiB per added session; F/C1 = 0.938; C1 minus F = 14.32 MiB, interval [13.13, 15.55]FAIL: real in direction, 6.2% not 15%
b server readyp50 0.862 s (F) vs 1.077 s (C1) = 0.80, needed at most 0.70; p95 0.930 s vs 1.175 s holdsFAIL on the median condition
c Chromium readyp95 1.332 s (F) vs 1.531 s (C1)PASS, and the whole gain is the server's: the Chromium-specific part is the same 0.35 s in every arm
d hygienedistinct PID, mount and network namespaces; pid sets disjoint; one Xvfb per session (X sockets X90 and X91 each visible only to its own session); ports 5000 and 9222 in each separate network namespace; no zygote file descriptor leaked; siblings' random, os.urandom, uuid4 and numpy global draws all differ (0.9011 vs 0.3356; 0.4918 vs 0.8059)PASS

Reading C0 honestly. C0 (OSWorld's python main.py) is 280.96 MiB per added session and F is 23% below it. But C1, the same fresh-interpreter entry as F without the debug reloader, already accounts for 18 of those points. The fair comparison for the fork mechanism is F against C1: 6.2%.

(f) The share bound. Private memory of one session at N = 20 (from smaps_rollup, trial 0): Chromium's 10 processes hold about 90 MiB private in every arm; the Python guest server plus session init hold 51.7 MiB in C1 (42184 + 10708 KiB) and 37.3 MiB in F (38184 KiB). The 14 MiB difference is exactly the measured slope difference. Even if all of the Python part were shared, the saving could not exceed about 51 MiB of 231 MiB, about 22%. The rest of the per-session cost is Chromium, the other exec'd processes (Xvfb 8 MiB, D-Bus/AT-SPI/openbox 6 MiB), 31.5 MiB of RAM-backed writable layer, and kernel structures. The break-even for the zygote's own 51 MiB is about 4 sessions; at 20 sessions the net host saving is about 235 MiB of roughly 4.6 GiB, about 5% (arithmetic from the measured slopes, not a separate measurement).

(e) What breaks when a live GUI process is forked

warm parent process(GUI + server, initialized)fork()child = one sessionshares the parent's pageswhat the child inherited, and what brokeX displaychild saw the parent's windowX connectionboth hung (parent 12 s, child killed at 15 s)listening socketchild took 100 of 100 accepts, parent 0D-Bus name :1.0duplicated; child's call timed out at 3 slock held at forkstays held in the child, never acquiredmemory per added session (measured, one 4-vCPU sandbox)no fork (C1)231.1 MiBfork (F)216.8 MiB, -6.2%bar: -15% (196.5 MiB)FAIL
Five breakages measured by forking a stand-in process that holds a live X connection with a window, a D-Bus connection, a listening socket and a thread-held lock (hazard.jsonl). Chromium itself was not forked; the inference that it would hit all five is the author's.

Quoting hazard.jsonl:

  1. No isolation from X. windows_visible_to_child: ["PARENT-SESSION-WINDOW"]. The child saw the parent's window because both copies are clients of one X server.
  2. A shared X connection deadlocks both copies. Parent hung_after_s: 12; child hung_and_killed_after_s: 15, partial: "". Replies are consumed by the wrong process.
  3. A shared listening socket is shared. Of 100 connections (3_connections_made: 100), the child accepted 100 and the parent 0. Either process can accept; there is no routing to "its" session.
  4. D-Bus identity is duplicated and the child's connection is dead. Both report unique name :1.0; the parent's call returned ok in 0.0082 s; the child's failed: Timeout was reached after 3.004 s, because the connection's worker thread does not exist in the child.
  5. A lock held by another thread at fork time stays held. 5_lock_held_by_a_thread_at_fork: {acquired_in_child: false}. The child could not acquire it in 2 s. (A first run of this check was invalid: the holder's 30 s sleep had ended before the check; it was re-run with a 600 s hold.)

So fork is safe only before a process opens such state, which is why F forks before import pyautogui and why Chromium, a multi-process browser with live X and D-Bus state, is exec'd in every arm. "Chromium itself was not forked or probed ... That is an inference from the findings, not a Chromium test" (design doc, section 9).

Failures and surprises

Limitations

Reproduce

On a Linux host with root, overlayfs, Xvfb, openbox, the OSWorld guest-server kit and Playwright Chromium:

python -m envd_linux.l1proto.bench --out bench.json --latency-n 20 --warmup 2 --trials 4 --max-n 20
python -m envd_linux.l1proto.hygiene --out hygiene.json
python -m envd_linux.l1proto.hazard_running_fork --out hazard.jsonl
python -m envd_linux.l1proto.report bench.json

Compare memory_summary.C1_minus_F.ci95 and latency_summary.*.server_ready_s against the bars in design-doc section 8.

Sources