In one paragraph
Before any matched run against an incumbent can be trusted, the driver that would run it has to work on real sessions. This record is that smoke test and nothing more. The I3 first-check driver ran the public task e0df059f in four blocks (post_setup, no_reuse, no_reuse, post_setup) at concurrency 1 on real Drums L0 sessions: 12 of 12 rollouts valid, no faults, no interference, canary clean in every block. Measured arm wall times were 23.39 s (post_setup, including a first rollout that does the setup and the save, 3.65 s of work across the two blocks) and 27.80 s (no_reuse). The summary labels itself engineering_validation_only, no_headline_claim, no_training_claim, resource_parity_claimed: false, and names the raw ratio between arms NOT_A_CLAIM. We print no cost per valid and no valid per hour: the price in pins.json is a placeholder of 3600 USD per hour, "so cost == seconds; not a price". A second pair of runs added per-session cgroups in the incumbent's shape. The first attempt is INVALID for CPU and memory (it read cgroup v2 files on a v1 host, so zeros mean "not measured"); the corrected v1 run read the limits back from the kernel (cpu.cfs_quota_us 200000 over 100000, memory.limit_in_bytes 4294967296, pids.max 4096) and saw a per-session kernel peak of about 94 MiB on this trivial task, 12 of 12 valid again. "Drums" throughout means the L0 prototype: namespaces, overlayfs on the sandbox's own Ubuntu 24.04 root, Xvfb, openbox; no shared-runtime (L1) result exists, and no incumbent ran.
Setup
Host. One development sandbox: 4 vCPU, 16 GiB, kernel 6.18, cgroup v1 (hybrid), no KVM; hostname and provider not recorded. preflight_probe.json on the same host: go_incumbent: false (no vmx/svm, no VirtualBox), go_drums: true, go_drums_restore: false (a private ip was not on PATH for this probe), go_joint_run_on_this_host: false; three runtime probes passed (namespaces, overlay redirect_dir, v1 cgroup limits); four gates untested.
The driver. compare/drums_compare/first_check.py: alternating ABBA blocks, concurrency 1, the interference canary (a marker written in rollout k must be absent in k+1), external-preparation accounting (pool construction before the driver clock, reported as external_prep_s), host-wide memory and CPU telemetry, and optional per-environment cgroup probes. It computes no verdict.
Arms.
| Label | What it is |
|---|---|
drums_no_reuse | a Drums L0 session destroyed and recreated per rollout |
drums_post_setup | the same session type using OSWorld's public snapshot API, implemented as a CRIU checkpoint after the first setup and restore for later rollouts |
Pool DrumsSessionPool(4, eager_reset=False); task sequence e0df059f x 3 per block; composition recorded in composition_manifest_at_smoke.json (Ubuntu 24.04 noble, Chromium 141.0.7390.37, LibreOffice 24.2.7.2, xvfb 2:21.1.12, openbox 3.6.1, Python 3.12.3, OSWorld guest server from 04718ea unmodified).
Three variants.
| Folder | Commit | What changed |
|---|---|---|
first_check_smoke/ | 3c87319 (11:20 UTC) | host-wide memory and CPU only; per_environment_resource_capture: "MISSING..." |
first_check_smoke_limits/ | 4bc9afc (11:29 UTC) | first per-session attempt with 2 CPU / 4 GiB session cgroups; probe read v2 files on a v1 host: INVALID for CPU and memory |
first_check_smoke_v1/ | d6a9e8a (11:40 UTC) | corrected probe (cgroup_telemetry.py supports v1 memory, limits, pids and per-pid CPU) |
What ran
Twelve rollouts per variant, 36 in all, starting at about 11:16 UTC on 2026-10-24 (t_start 1792840766). Each block file holds three rows with reset_s, wall_s, score, valid, canary, and a summary with the placeholder prices labelled "driver smoke placeholder, not a price". smoke_stdout.log ends with the RESULT line; smoke_stderr.log holds a fitz deprecation warning and a missing proxy-file notice. The full test suite was run on the committed tree afterwards (full_suite12.log, 232 passed, see EXP-024).
Results
First-check driver smoke on real Drums L0 sessions (drums_post_setup vs drums_no_reuse) and per-session cgroup limits read back from the kernel preliminary2026-10-24 · dev-sandbox
| Metric | Value | Unit | n | Note |
|---|---|---|---|---|
| original smoke: drums_post_setup valid rollouts | 6 / 6 | rollouts | 2 blocks, 0 faults, 0 interference, canary clean | |
| original smoke: drums_no_reuse valid rollouts | 6 / 6 | rollouts | 2 blocks, 0 faults, 0 interference, canary clean | |
| original smoke: drums_post_setup arm wall time (6 rollouts) | 23.39 | s | includes the first rollout's setup and save; raw ratio to the other arm is labelled NOT_A_CLAIM in the file | |
| original smoke: drums_no_reuse arm wall time (6 rollouts) | 27.81 | s | ||
| original smoke: drums_post_setup first-rollout work (setup and save, both blocks) | 3.65 | s | 2 | per block 1.791 and 1.858 s |
| original smoke: external preparation (pool construction before the driver clock) | 0.45 | s | the pool creates sessions lazily, so this understates preparation | |
| original smoke: drums_post_setup per-rollout revert reset | 0.79 to 0.858 | s | 4 | block means 0.824 and 0.815 s |
| original smoke: drums_no_reuse per-rollout reset after the first | 2.35 to 2.46 | s | 4 | first rollout of each block 0.815 / 0.814 s on a fresh environment |
| limits variant (first per-session attempt): valid rollouts | 12 / 12 | rollouts | INVALID for CPU and memory: v2 files read on a v1 host (cpu_s 0.0, mem_peak 0.0 mean 'not measured'); only pids.max 4096 and procs_max 9 are real | |
| v1 smoke (corrected): valid rollouts, both arms | 12 / 12 | rollouts | ||
| v1 smoke: drums_post_setup arm wall time | 23.99 | s | first_rollout_work 3.709 s | |
| v1 smoke: drums_no_reuse arm wall time | 29.36 | s | ||
| v1 smoke: session CPU quota read back from the kernel | 2 | vCPU | cpu.cfs_quota_us 200000 / cpu.cfs_period_us 100000 | |
| v1 smoke: session memory limit read back from the kernel | 4 | GiB | memory.limit_in_bytes 4294967296; memsw the same (no swap) | |
| v1 smoke: session pids.max read back from the kernel | 4,096 | pids | ||
| v1 smoke: per-session peak memory (kernel), drums_post_setup blocks | 93.82 to 93.84 | MiB | 2 | about 2% of the 4 GiB limit on this trivial task |
| v1 smoke: per-session peak memory (kernel), drums_no_reuse blocks | 94.62 to 94.94 | MiB | 2 | |
| v1 smoke: per-session CPU, sampled lower bound, drums_post_setup blocks | 1.05 to 1.97 | s | 2 | sum of utime+stime of pids seen in the cgroup; cpuacct not in the cgroup on v1 |
| v1 smoke: per-session CPU, sampled lower bound, drums_no_reuse blocks | 2.96 to 4.02 | s | 2 | |
| v1 smoke: processes per session, maximum seen | 9 to 10 | processes | empty_samples 0 in every block |
Host: 4 vCPU, 16 GiB, cgroup v1, no KVM, Ubuntu 24.04, kernel 6.18 (sandbox). one development sandbox; hostname and provider not recorded; not a VM or cloud host. Workload: Public OSWorld task e0df059f three times per block, 2 blocks per arm in ABBA order, concurrency 1, through the real DesktopEnv; drums_no_reuse = a Drums L0 session destroyed and recreated per rollout; drums_post_setup = the same session type with OSWorld's public snapshot API implemented as a CRIU checkpoint after the first setup and restore for later rollouts. Both arms are Drums; no incumbent ran.. Method: compare/drums_compare/first_check.py driver (engineering validation only; computes no verdict). Original smoke 3c87319 with host-wide memory and CPU only; first per-session attempt 4bc9afc read cgroup v2 files on a v1 host (INVALID for CPU and memory); corrected v1 smoke d6a9e8a applies 2 CPU / 4 GiB / 4096 pids per session and reads the limits back from the kernel, sampling at 0.2 s. Prices are a placeholder 3600 USD/hour so cost equals wall seconds; no cost or valid-per-hour figure is published here.. Source: branch claude/linux-comparison-prep: docs/data/l1-criu/first_check_smoke/summary.json (arms.*.rollouts, valid, faults, wall_s, first_rollout_work_s, external_prep_s); branch claude/linux-comparison-prep: docs/data/l1-criu/first_check_smoke/block0_drums_post_setup.json, block1_drums_post_setup.json, block0_drums_no_reuse.json, block1_drums_no_reuse.json (rows[].reset_s, summary.reuse); branch claude/linux-comparison-prep: docs/data/l1-criu/first_check_smoke/pins.json, preflight_probe.json; branch claude/linux-comparison-prep: docs/data/l1-criu/first_check_smoke_limits/README.md, summary.json; branch claude/linux-comparison-prep: docs/data/l1-criu/first_check_smoke_v1/summary.json (arms.*.wall_s, per_environment_blocks[].sessions.*.limits_as_read_by_kernel, mem_peak_mib_kernel, cpu_s_lower_bound, procs_max, empty_samples); branch claude/linux-comparison-prep: docs/incumbent-preflight-plan.md sections 8b and 8c.
- NOT a comparison: both arms are Drums L0 sessions; the incumbent adapter has run against fakes only.
- Prices are a placeholder (3600 USD/hour, 'not a price'); no cost per valid or valid per hour is published.
- Host-wide peak_delta_per_env_mib values (106.3, 67.3, -27.4, 18.2) are host noise and are not reported as per-session figures.
- The limits variant is invalid for CPU and memory; the v1 smoke's CPU is a sampled lower bound.
- resource_parity_claimed is false in every summary; per-session capture exists on the Drums arm only.
- After the two limit-enabled smokes one empty session cgroup directory per run was left behind (open finding, not diagnosed).
- Drums here is the L0 prototype; no shared-runtime (L1) result exists.
Original smoke (measured, first_check_smoke/summary.json)
drums_post_setup | drums_no_reuse | |
|---|---|---|
| rollouts / valid / faults | 6 / 6 / none | 6 / 6 / none |
| arm wall time | 23.394 s | 27.805 s |
| first-rollout work (setup and save) | 3.648 s over two blocks (1.791 and 1.858 s) | 0 |
| per-rollout reset after the first | 0.858, 0.790 (block 0); 0.811, 0.815 (block 1) s | 2.451, 2.455; 2.403, 2.349 s |
| first rollout of each block | setup+save | 0.815 and 0.814 s on a fresh environment, no revert |
| interference / canary | 0 / clean x 6 | 0 / clean x 6 |
| external preparation (pool build, outside the driver clock) | 0.450 s | same |
| host CPU busy per block | 5.07, 5.09 s | 6.33, 6.14 s |
The host-wide peak_delta_per_env_mib values (106.3, 67.3 for post_setup; -27.4, 18.2 for no_reuse) are host noise; one is negative. They have no per-session meaning and we do not report them as results. The file's raw_ratio_first_over_second_with_prep_NOT_A_CLAIM is 1.185 and is, as named, not a claim.
Per-session cgroup limits (measured, first_check_smoke_v1/summary.json)
| post_setup block 0 | post_setup block 1 | no_reuse block 0 | no_reuse block 1 | |
|---|---|---|---|---|
| limits as read by the kernel | cpu.cfs_quota_us 200000 / cpu.cfs_period_us 100000 (2 CPU), memory.limit_in_bytes 4294967296, memory.memsw.limit_in_bytes 4294967296, pids.max 4096 | same | same | same |
| peak session memory, kernel | 93.84 MiB | 93.82 MiB | 94.62 MiB | 94.94 MiB |
| peak session memory, sampled | 92.35 | 85.93 | 89.02 | 92.32 |
| CPU, sampled lower bound | 1.05 s | 1.97 s | 2.96 s | 4.02 s |
| processes, max seen / empty samples | 9 / 0 | 9 / 0 | 10 / 0 | 9 / 0 |
12 of 12 valid; arm wall times 23.987 s (post_setup, first-rollout work 3.709 s) and 29.357 s (no_reuse). About 94 MiB peak is roughly 2% of the 4 GiB limit on this trivial task: the limits work on this host; nothing here is limit pressure. resource_parity_requires in the file: "the incumbent VM's settings verified (vCPU, RAM) AND the Drums cgroup limits verified as applied to every live session, plus per-environment capture on BOTH arms; none of that holds".
The invalid limits variant (kept, first_check_smoke_limits/)
12 of 12 valid with session cgroups at 2 CPU / 4 GiB, no fault. But per_environment reads cpu_s_lower_bound 0.0, mem_peak_mib_sampled 0.0, mem_peak_mib_kernel: "MISSING (no memory.peak)", cpu.max null, memory.max null: the probe at 4bc9afc read only cgroup v2 files and this host mounts v1 (separate cpu, memory, pids directories). The README: "These zeros are 'not measured', not 'used nothing'." Only pids.max 4096, procs_max 9 and the empty_samples counts are real there.
Failures and surprises
- The first per-session capture was invalid for the two numbers it was built to produce, because of a v1/v2 mismatch; it is kept as raw data with a README rather than overwritten.
- Working trees of the smokes (306 files, 5.7 MB, a synthetic Chromium first-run profile with random per-profile identifiers) were accidentally committed in
54566c4andd6a9e8aand removed at the tip in886a362; they remain in history.first_check_smoke_EVIDENCE.md: "the exposure is synthetic-profile bytes, not credentials". - After the two limit-enabled smokes one empty session cgroup directory per run was left behind in the cpu, memory and pids hierarchies, with no process in it; the suite runs left none. Recorded as an open finding, not diagnosed (
suite_runs/README.md). - The pool creates sessions lazily, so
external_prep_sof 0.45 s understates preparation. go_drums_restorewas false in the preflight probe for a trivial reason (ipnot on PATH for CRIU's network restore), while the smoke's restores ran with a private copy; the preflight is read-only and did not start a session.
Limitations
- Both arms are Drums. The incumbent adapter has run against fakes only; no VM, container or incumbent provider is in any file here.
- Placeholder price; no cost, cost per valid or valid per hour is published, and none should be computed from these files.
- Host-wide memory deltas are noise; per-session capture exists on the Drums arm only, with CPU as a sampled lower bound (cpuacct is not in the cgroup on v1).
- Twelve rollouts of one trivial task per variant; the first-rollout work is a large share, so per-rollout figures are not steady state.
- The cgroup v2 limits path was not exercised on a real v2 kernel in this sandbox; see EXP-026 for what happened with v2 on a real host.
- No L1 result exists.
Reproduce
DRUMS_OSW_PYTHON=<venv python> DRUMS_OSW_SRC=<OSWorld checkout> DRUMS_CRIU_HOME=<private criu dir> \
python -m envd_linux.l1proto.first_check_smoke --out <dir> [--limits]
python -m envd_linux.preflight probe --out preflight_probe.json
python -m envd_linux.composition --out composition_manifest.json
Needs root, overlayfs, Xvfb, nft, the OSWorld kit and CRIU 4.2. Treat every cost_* and valid_per_hour* field as a placeholder until pins.json carries a real price.
Sources
docs/data/l1-criu/first_check_smoke/summary.json,pins.json,preflight_probe.json,composition_manifest_at_smoke.json,block0_drums_post_setup.json,block0_drums_no_reuse.json,block1_drums_post_setup.json,block1_drums_no_reuse.json,smoke_stdout.log,smoke_stderr.log,composition.stderr,preflight_probe.stderr(3c87319, 2026-10-24 11:20:58 UTC)docs/data/l1-criu/first_check_smoke_limits/README.md,summary.json,pins.json, block files, logs (4bc9afc, 11:29:28 UTC)docs/data/l1-criu/first_check_smoke_v1/summary.json,pins.json, block files, logs (d6a9e8a, 11:40:12 UTC)docs/data/l1-criu/first_check_smoke_EVIDENCE.md(886a362),docs/data/l1-criu/drums_composition_manifest.jsondocs/data/suite_runs/README.md(open finding on leftover cgroup directories)docs/incumbent-preflight-plan.mdsections 8b and 8c;docs/matched-comparison-protocol.mdcompare/drums_compare/first_check.py;envd-linux/envd_linux/l1proto/first_check_smoke.py,envd_linux/cgroup_telemetry.py,envd_linux/limits.py,envd_linux/preflight.py,envd_linux/composition.py- Branch
claude/linux-comparison-prep; result filedata/results/exp-023-first-check-smoke.jsonon this site