This page is the evidence behind the blog post Our first real training run on Drums. The pre-registration was frozen before the launch; its sha256 is below and the frozen text is linked unchanged. The run log and the 37-finding audit of our own product are internal for now.
Setup
| item | value |
|---|---|
| model | Qwen/Qwen3.5-27B, revision fc05daec18b0a78c049392ed2e771dde82bdf654, bf16 |
| trainer | prime-rl commit 167d4c3, full-parameter GRPO, FSDP2 on 6 GPUs, vLLM (TP2) on 2 GPUs, lr 3e-6, groups of 8, 128 samples per step, temperature 0.8 |
| GPUs | Modal, 8 x H200 (one 50-minute stint on 8 x B200 before a preemption; resumed from the last checkpoint) |
| budget | W = 480 min of trained time (trainer wall from the start of its loop; evaluation waits do not count) |
| task family | d1_files_collect from drums-verifiers 0.2.2: collect every file whose name starts with a prefix from a folder tree into a new folder, renaming each copy; the checker reads the files |
| variants | 64 training (seed 611161), 32 held-out (seed 611162), zero overlap on instruction, file set and parameters |
| desktops | one Drums Cloud worker host (c8i.24xlarge), up to 128 desktops in flight for training, 48 for evaluation |
| agent | 1280 x 720 screenshots, 5 kept in context, 15 turns maximum, 1,536 completion tokens, thinking off |
Pre-registered criterion
Offline evaluation of the start weights and of the last checkpoint written within W on the 32 held-out variants, K = 6 episodes each (192). Per variant, success = share of valid episodes the checker scored 1.0. Paired variant bootstrap of last minus step 0 (10,000 resamples, seed 20261116, percentile 95% interval); met when the lower bound is above 0. Completeness: at least 90% of the 192 episodes valid and every variant valid once; an incomplete evaluation is repeated once in full and the repeat is used. One look.
Frozen 2026-11-16T13:09:28Z, sha256 6819b60727060129b8923d49b987e9861472c11c92f872e31a80c7461a4a8e24: PREREGISTRATION-frozen.md.
Result
| evaluation | step 0 | step 6 (last) |
|---|---|---|
| held-out, 32 variants, K = 6 | 102 of 189 valid; mean over variants 0.5417 | 192 of 192; 1.0000 |
| training, 64 variants, K = 2 | 74 of 125 valid; 0.6016 | 128 of 128; 1.0000 |
Held-out paired difference over all 32 variants: +0.4583, 95% CI [0.3750, 0.5458]. Criterion met. Training variants: +0.3984, CI [0.3047, 0.4922].
At step 0, 30 of the 32 held-out variants were mixed (sometimes solved, sometimes not), 1 was never solved and 1 always solved.
Episode shape under evaluation (median): step 0 used 15 turns (the cap), 13 actions and 161 s; step 6 used 8 turns, 7 actions and 62 s.
The training curve
| step | batch reward | wall for the batch | episodes discarded (whole group same outcome) |
|---|---|---|---|
| 1 | 0.641 | 8 min | few |
| 2 | 0.648 | 14 min | few |
| 3 | 0.758 | 15 min | some |
| 4 | 0.844 | 16 min | 59% (184 of 312) |
| 5 | 0.859 | 1 h 24 min | 93% (1,808 of 1,936) |
| 6 | 0.875 | 3 h 37 min | 98% (7,744 of 7,872) |
GRPO learns from groups whose members disagree. By step 5 almost every group of 8 attempts on a training variant all succeeded, so the trainer threw them away and waited for the rare mixed group. Eight hours bought six optimizer steps; the last one cost about 7,900 desktop episodes. About 9,000 rollouts in all, 0 failed on the host.
What happened along the way
- The first attempt through our cloud gateway hit a hard-coded rate limit and then a spend cap; the second overloaded the small control-plane database behind the gateway, which took our console and every gateway down for about 3.5 hours. The run continued with the trainer talking to the host directly. The data path will stop calling the control plane; that is a design change we have written up and started.
- The GPU provider preempted the container during step 3; the run resumed from the step-2 checkpoint on a different GPU type. Trained time was the budget.
- We restarted the host's gateway during the first step-0 evaluation; 47 episodes errored and the evaluation was incomplete (166 valid, three variants without a valid episode). Per the frozen rule it was repeated once in full and the repeat is the number above. The incomplete run read 0.48; its records are published too.
What this shows and does not show
Shows: the loop works end to end for a client's shape of job, and the gain transfers to unseen variants of the same template. Does not show: anything comparative (no other environment ran), transfer across apps or templates, or robustness across seeds (one run). The 1.00 is on a template the policy had saturated.
Files
- Records, one line per scored episode: step0-heldout-k6r (used), step0-heldout-k6 (incomplete, not used), last-heldout-k6, step0-train-k2, last-train-k2; each has a
.result.jsonnext to it. - Verdicts: verdict-heldout-FINAL.json, verdict-train.json; the script: d1_analysis.py.
- Power simulation from the step-0 scan: d1_power.py, power-d1_files_collect.json (0.64 at +0.10, 0.93 at +0.15).
- Training config: train-d1.toml. Cost: ledger.md (about 500 USD at list prices, estimated from timestamps).
Reproduce: python3 d1_analysis.py --step0 step0-heldout-k6r.jsonl --last last-heldout-k6.jsonl prints the verdict JSON above.