In one paragraph
The full write-up is the research note First training run; this record puts it in the experiment log with its result file. Pre-registered, one look: held-out success went from 0.5417 to 1.0000, a paired difference of +0.4583, 95% CI [0.3750, 0.5458] over 32 unseen variants, both evaluations complete. The run made only 6 optimizer steps in 8 hours because the policy saturated the 64 training variants by step 5. The run doubled as an audit of our own product and found 37 issues, the largest being that one 128-desktop run took the control-plane database, the console and every gateway down for about 3.5 hours. D2 (EXP-037) replicated the result on the fixed product.
Setup
prime-rl 167d4c3: FSDP2 full-parameter trainer on 6 GPUs, vLLM TP2 on 2 GPUs, GRPO groups of 8, batch 128, lr 3e-6, no reference KL. Desktops: one Drums Cloud worker installed from the public drumsd v0.2.1 release, reached through the gateway. Agent: 1280 x 720 screenshots, 5 kept in context, 15 turns, 1,536 completion tokens, thinking off. Task family from drums-verifiers 0.2.1 plus branch templates/d1 at 42e1e60; the checker compares the target tree with the oracle's, and 4 near-miss solutions per variant must score 0.
What ran
Template choice by a step-0 scan, a one-step smoke, the freeze at 13:09:28Z, then training on Modal for W = 480 minutes of trained time, then offline evaluation of step 0 and step 6 on the held-out (K = 6) and training (K = 2) variants.
Results
D1: eight hours of full-parameter GRPO on Qwen3.5-27B, every desktop through Drums Cloud, held-out success verified2026-11-16 · Modal GPUs + Drums Cloud worker
| Metric | Value | Unit | n | Note |
|---|---|---|---|---|
| Held-out success, step 0 to step 6 (last within W) | 0.542 to 1 | success rate | 32 | +0.4583, 95% CI [0.3750, 0.5458]; criterion met |
| Held-out valid episodes, step 0 / step 6 | 189 / 192 | episodes | step 6: 192 of 192; step 0 had 3 no_app_crash invalid | |
| Optimizer steps in 8 h of trained time | 6 | steps | saturated by step 5: 93% then 98% of groups discarded as zero-advantage | |
| GPU and host spend (estimate from timestamps) | 504 | USD | about 84 GPU-hours on Modal plus the host; an estimate, not a bill |
Host: Modal 8 x H200 141 GB (6 trainer, 2 vLLM TP2); Drums worker c8i.24xlarge (96 vCPU, 192 GiB) on drumsd v0.2.1, worker Ubuntu 24.04, kernel 7.0.0-1014-aws. one 50-minute stint on 8 x B200 before a preemption. Workload: d1_files_collect (one self-made drums-verifiers template): 64 training variants (seed 611161), 32 held-out (seed 611162), zero overlap; 15 turns, 1280 x 720, 5 screenshots in context. Model: Qwen/Qwen3.5-27B rev fc05daec, full-parameter GRPO, prime-rl 167d4c3. Method: Pre-registered (frozen 2026-11-16T13:09:28Z, sha256 6819b607...). Held-out K = 6, step 0 vs last checkpoint within W = 480 min trained; paired variant bootstrap, 10,000 resamples, seed 20261116; met when the lower bound is above 0. One look.. Source: research archive: runbooks/rl-l2/20261116T0426Z/REPORT.md; research archive: runbooks/rl-l2/20261116T0426Z/analysis/verdict-heldout-FINAL.json; research archive: runbooks/rl-l2/20261116T0426Z/analysis/verdict-train.json.
- One template, one app, one run: a working loop and a within-template transfer, not a general capability claim.
- The first step-0 evaluation was incomplete (a gateway restart) and was repeated once in full, as pre-registered.
- The run took the control-plane database, the console and every gateway down for about 3.5 h; it continued by talking to drumsd directly. Six operator actions were needed.
Failures and surprises
- The first step-0 held-out evaluation was incomplete (a gateway restart errored 47 episodes) and was repeated once in full, as pre-registered; the repeat is used.
- The control-plane outage; the run continued by talking to drumsd directly. Six operator actions in all.
- No B200 capacity for 3 hours; the run used H200, with one 50-minute B200 stint before a preemption.
Limitations
One template, one app, one run. Saturation means more steps would not have taught more on this pool. Spend about 504 USD is an estimate from timestamps, not a bill.
Reproduce
The pre-registration lists every pin; the analysis is analysis/d1_analysis.py on the evaluation records.
Sources
Pre-registration, report, run log, audit and verdict files under Evidence; the frozen pre-registration is also published on this site.