In one paragraph
With one five-line installer fix (fs.pipe-user-pages-soft = 0), the Drums release sustained 512 desktops on T1-fixed host A (1,201 rollouts, 100% valid, 144 GiB of memory left), on the fresh host of the v0.2.0 integration re-run (1,151 of 1,152 valid) and on host B of the Firecracker study (1,171 of 1,172 valid). On the same machine type tuned KVM sustained 128 on every host it ran on (at 256 only 139 to 143 VMs fit in memory), and tuned Firecracker sustained 16 on one host and 32 on the other. Rungs double, so 512 is the last sustained doubling, not a ceiling. The same pre-registered study gave Drums 1.38x tuned KVM's valid rollouts per dollar on host A and 1.34x on fresh host B, both CIs excluding 1. What does not hold yet: T1-fixed host B stopped at 128 (one quarantined environment produced 10,535 instant failures, counted against Drums), and on the Firecracker study's pre-registered host A one create failed at rung 8 and Drums has no result. So 512 is measured on three hosts, and no single study shows it on both of its hosts.
Setup
All hosts r8i.16xlarge on-demand (spot had no capacity), Ubuntu 24.04, kernel 7.0.0-1014-aws, gp3 600 GiB at 16k IOPS. T1-fixed: drumsd sota/t1-pipefix a8d118b (rc1 bd43483 plus a 5-line installer change), drums-lab cb3145d, drums-criu b4fbfddf, hardened profile on, 4 vCPU and 4 GiB per environment; tuned KVM as in EXP-031. Integration: the v0.2.0 release tarball, the frozen T1-fixed procedure on one fresh host. Firecracker study: Drums v0.2.0 against Firecracker v1.18.0 (one microVM per desktop, OSWorld image, 20 s snapshot, plus a quiescent-snapshot variant), harness bench-r2/firecracker frozen at a4de57c, us-east-1f. Sustained step: at least 95% valid, 0 contamination, MemAvailable at least 3 GiB, step p95 at most twice the first step's.
What ran
T1-fixed: Drums ladder then KVM ladder on host A, the same on fresh host B. Integration (2026-11-11): the same procedure on the v0.2.0 tarball. Firecracker study (2026-11-12): both arms' ladders on hosts A and B, plus a same-host fork and reset workload.
Results
Density and per-dollar, T1-fixed: Drums release with the pipe fix against tuned KVM, hosts A and B preliminary2026-11-10 · hosts A and B
| Metric | Value | Unit | n | Note |
|---|---|---|---|---|
| Max sustained desktops, Drums, host A | 512 | desktops | 512 rung: 1,201 rollouts, valid share 1.0000, 6,194 valid/h, min MemAvailable 143.8 GiB | |
| Max sustained desktops, Drums, host B | 128 | desktops | 256 not sustained: one env quarantined, 10,535 instant reset failures on its slot, counted against Drums | |
| Max sustained desktops, tuned KVM, hosts A and B | 128 | desktops | at 256 only 143 VMs fit in memory, on both hosts | |
| Drums / tuned KVM, valid rollouts per USD, host A | 1.38 | x | 55 | 95% CI [1.32, 1.44]; Drums better |
| Drums / tuned KVM, valid rollouts per USD, host B (replication) | 1.34 | x | 55 | 95% CI [1.28, 1.41]; Drums better; inside +-20% |
| Memory per desktop at N 128, Drums vs tuned KVM, host A | 0.708 to 2.68 | GiB | Drums 0.708 [0.688, 0.741], tuned KVM 2.676 [2.631, 3.015] | |
| Reset p50 / p99 at N 128, Drums, host A | 1.79 | s | p99 20.379 s (tuned KVM 2.630 / 7.371 s) | |
| Invalid rollout rate, Drums and tuned KVM | 0 | share | Drums 0.0000 [0.0000, 0.0036], KVM 0.0000 [0.0000, 0.0055] |
Host: r8i.16xlarge (64 vCPU, 512 GiB, nested virtualization on), on-demand, us-east-1a, gp3 600 GiB 16k IOPS, Ubuntu 24.04, kernel 7.0.0-1014-aws. B fresh. Workload: Same 55 tasks, scripted mock policy (no model), seed 20261108; ladder doubling from 8 to 512. Method: Pre-registered (frozen 2026-11-09T22:04:28Z, sha256 1958a81c...): changes only the Drums build (RC1 plus fs.pipe-user-pages-soft = 0). Sustained step: at least 95% valid, 0 contamination, MemAvailable >= 3 GiB, step p95 <= 2x the first step's. Max desktops claim rule: the same direction on both hosts. valid/USD: paired bootstrap over tasks, 10,000 resamples.. Source: research archive: runbooks/sota/t1/20261109T2203Z-fixed/REPORT.md; research archive: runbooks/sota/t1/20261109T2203Z-fixed/summary/tables.md.
- Max desktops: 512 on A, 128 on B, so by the frozen rule there is NO density claim from this study alone.
- Steps double, so '128' means 128 to 255 desktops and '512' is the last sustained doubling, not a ceiling.
- Drums is hardened while the KVM agent has root in its VM; each is its product's default. No model in the loop.
- Disclosed before the freeze: an exploratory run had already shown this build reaching 256 desktops.
Density on two more hosts: the v0.2.0 integration re-run (vs tuned KVM) and the Firecracker study (vs tuned Firecracker) preliminary2026-11-12 · integration host; Firecracker host B
| Metric | Value | Unit | n | Note |
|---|---|---|---|---|
| Integration host: Drums 512 rung | 1151 / 1152 | rollouts | valid share 0.9991, 6,145 valid/h, min MemAvailable 171.1 GiB: sustained | |
| Integration host: tuned KVM max sustained | 128 | desktops | at 256 only 139 slots came up | |
| Integration host: Drums / tuned KVM valid per USD | 1.4 | x | 55 | 95% CI [1.33, 1.47]; agrees with T1-fixed within +-20% |
| Firecracker host B: Drums v0.2.0 512 rung | 1171 / 1172 | rollouts | valid share 0.9991, 6,152 valid/h: sustained; operating N 256 (7,052 valid/h) | |
| Firecracker host B: tuned Firecracker max sustained (20 s snapshot) | 16 | desktops | 32 failed on B; on host A 32 sustained, 64 failed: 16 to 32 across the two hosts | |
| Firecracker host A: Drums result | 0 | rollouts | none: one of eight creates failed at rung 8 (a slot service-plan read/write race) and the frozen rule stopped the ladder | |
| Firecracker host B: Drums reset at its operating N 256 | 12.56 | s | p99 48.58 s; at equal N 16: Drums 0.46 / 0.79 s vs Firecracker 1.52 / 3.31 s | |
| Firecracker host B: per-task valid/USD, Drums / Firecracker (20 s arm) | 7.6 | x | 45 | 95% CI [7.19, 8.01]; NO CLAIM by the frozen power rule (Firecracker under 200 rollouts at its operating N) |
Host: r8i.16xlarge (64 vCPU, 512 GiB, nested virtualization on), on-demand (one fresh host per job), Ubuntu 24.04, kernel 7.0.0-1014-aws. Firecracker study in us-east-1f; Firecracker without jailer, Drums hardened. Workload: Same 55-task scripted suite and ladder as T1; Firecracker v1.18.0 one microVM per desktop, OSWorld image, 20 s snapshot (and a quiescent-snapshot variant). Method: Integration: the frozen T1-fixed procedure re-run on the v0.2.0 tarball on a fresh host, no new freeze. Firecracker: PLAN frozen 2026-11-12T04:18:13Z (sha256 b7b15c96...), harness commit a4de57c, same sustained-step rule.. Source: research archive: runbooks/sota/integrate/20261111T1725Z/REPORT.md; research archive: runbooks/sota/integrate/20261111T1725Z/raw/t1/summary/tables.md; research archive: runbooks/sota/round2/firecracker/20261112T0226Z/REPORT.md; research archive: runbooks/sota/round2/firecracker/20261112T0226Z/summary-B/tables.md.
- 512 was sustained on three separate hosts across three jobs (T1-fixed host A, the integration host, Firecracker host B), but never on both hosts of one pre-registered study. T1-fixed host B stopped at 128 and Firecracker host A has no Drums result.
- The integration re-run has no freeze of its own and one host.
- The Firecracker host uses nested virtualization: the VM arm pays its exit cost and Drums does not; no .metal check. Firecracker ran without jailer.
- At its own operating point Drums' reset is slow (12.56 s p50 at 256 desktops); report it next to the 512.
Failures and surprises
- T1-fixed host B, 256 desktops: environment s251 was quarantined ("gate cgroup failed"), the harness kept hammering the dead slot, and its 10,535 instant reset failures failed the step. Needed fixes: drumsd keeps the full gate detail and replaces a quarantined environment; the harness retires a slot after repeated instant failures (a pre-registered rule change).
- Firecracker host A: one of eight creates failed at rung 8 (
service-plan.json: EOF while parsing), a write and read race in the slot service plan, which erased the pre-registered Drums result. - The Firecracker study's 80 USD cap was exceeded (about 89 USD) when an exploratory script hung while the operator's laptop slept.
Limitations
No model in the loop. One instance type. Drums at its own operating point resets slowly (12.56 s p50, 48.58 s p99 at 256 desktops on the Firecracker host). The Firecracker host uses nested virtualization, so the VM arms pay its exit cost and Drums does not; Firecracker ran without jailer while Drums was hardened. The Firecracker per-dollar ratio is "no claim" under the frozen power rule. Desktop capacity is not task throughput and not a cost saving by itself.
Reproduce
T1-fixed: REPRODUCE.md in its folder, drumsd a8d118b. Firecracker: REPRODUCE.md in its folder, harness a4de57c.
Sources
Pre-registrations, freezes, reports, summary tables and raw checksums under Evidence. Research archive, internal for now.