DrumsBench

Research notes

How shared runtimes, resets and verifiers work, what others have built, and what is still open.

Plain English first, then the detail. These notes change as the evidence changes; each carries its date.

These notes explain the ideas the experiments test: what a shared runtime is, how an RL rollout loop uses it, why checkpoint and restore is the hard part, what one macOS login session owns, who has built what before us, and how verifiers and contamination checks go wrong. Every number in them is labelled (measured, author-reported, author-claimed, vendor claim, design value, estimate) and every "checked on" date is kept. Where we were wrong before, the correction is in the note, dated. Read what Drums is first.

  1. First head-to-head benchmark: Drums against virtual machines and containersSame server, same tasks, same agent. Drums uses about 9 times less memory per copy and fits about 8 times more desktops on one server, but it produced fewer correct finishes per dollar than tuned virtual machines and Firecracker, because it could not save a few apps.2026-10-28benchmarkresults
  2. What Drums is, and what it is notThe hypothesis as written on 2026-10-15, the narrowed form of 2026-10-23, what a valid rollout is, and the list of things we are not claiming.2026-10-25thesisdefinitions
  3. The shared-runtime modelOne booted base runtime, private mutable state per attempt, copy-on-write, the reset and branch paths, and the exact list of state that must be scoped for training correctness.2026-10-25architectureisolationcopy-on-write
  4. The RL rollout loop and why cost per valid verified rollout is the metricPolicy, environment, observation, action, verifier, trainer. Valid, failed-but-valid and invalid rollouts. The branch-from-a-prepared-state experiment, with the prior art named.2026-10-25rlmetricsverifiers
  5. Checkpoint and restore, and what breaksCRIU basics for newcomers, the three Linux failures we hit (a lost POSIX lock, runc's empty network namespace, GNOME trees and mount namespaces), and the macOS equivalent that has no process checkpoint at all.2026-10-25criulinuxmacosrestore-fidelity
  6. Desktop isolation on macOS, one login session per environmentWhat one macOS GUI session owns and what it shares, why sessions and not per-process input, the two-VM cap and the license clauses in plain words, the CGSession finding, and the laptop measurements that are not Studio results.2026-10-25macosisolationlicense
  7. Prior art, what we got wrong, and vendor productsThe map of who runs computer-use environments and how, with author-reported numbers and licence status as of the checked-on dates; the corrections we made to our own beliefs; and the hosted products.2026-10-25prior-artlicencescorrections
  8. Verifiers, contamination, and why zero needs a fired positive controlWhat a verifier is, where verifiers go wrong in public benchmarks, how our verify server and canary work, the eleven isolation tests of the macOS harness, and why a zero contamination count means nothing until the detector has been shown to fire.2026-10-25verifiersisolationmethodology
  9. Open questionsThe engineering questions nobody has answered, the eight falsification questions we re-answer weekly with their current answers, and the sentence that closes every version of this page.2026-10-25open-questionsfalsification
  10. GlossaryThe words this site uses, defined the way the records use them.2026-10-25definitions