In one paragraph
The idea: fork a running desktop with CRIU at a decision point, train the shared prefix once, and get more learning per GPU-hour. It worked as engineering: 1,460 forks at about 6.5 s each with isolation and state gates passing, twice the updates per trained hour (28 against 16) and 59% more kept trajectories per GPU-second. It did not work as learning. No checkpoint reached the control's 0.8634 within the control's budget, so the time ratio is censored at at least 1.035x (worse). At equal trained time the branch arm scored 0.654 against 0.863, a difference of -0.209 (CI [-0.339, -0.074]). Children share a 6-turn prefix and 63% of forks gave every child the same reward, so most continuation advantages were zero. Branching stays an internal mechanism; the claim stays "cheaper rollouts", not "cheaper learning".
Setup
Control: the L1 Phase A plain-GRPO Drums arm (EXP-029), same GPU and environment instances, same prices (21.96048 USD/h), same evaluator scripts. Branch: trunks in slots 0 to 31, fork children in 32 to 127, in-loop evaluation in 128 to 143; fork capture on a 48 GiB tmpfs. Tree advantage: trunk segments trained once with weight C; child continuations get (r - V_m) / sigma; zero-advantage items are not trained.
What ran
One timed run, resumed once: the environment disk filled during step 21 (D8), the disk was grown and training resumed from step 20 with the void interval (6,870.4 s) removed from the clock, a rule fixed before any evaluation. Seven band evaluations (steps 4 to 28).
Results
rl-branch: forking a running desktop at turn 6 for tree GRPO did not make training cheaper failed2026-11-10 · G1 + env host
| Metric | Value | Unit | n | Note |
|---|---|---|---|---|
| Checkpoints whose band evaluation reached 0.8634 within 28,811 s | 0 / 7 | checkpoints | censored: target not reached within the control's budget | |
| GPU hours and dollars, branch / control (lower bound) | 1.04 | x | 1 | at least 1.035x (175.75 vs 169.78 USD); lower bound, no CI |
| Equal trained time: band success, branch step 28 minus control step 16 | -0.209 | success rate | 18 | 0.654 vs 0.863, 95% CI [-0.339, -0.074]; branch worse |
| Held-out (K = 10), branch step 28 minus control | -0.092 | success rate | 22 | 0.445 vs 0.538, 95% CI [-0.195, -0.003] |
| Updates in the budget, control vs branch | 16 to 28 | updates | control 16, branch 28 | |
| Kept trajectories per GPU-second, control vs branch | 0.092 to 0.145 | per s | +59% for branch: a throughput gain without a learning gain | |
| Forks: request time | 6.5 | s | 1460 | p90 7.6 s; 1,460 forked trees; 100 children failed |
| Forks with at least two children whose children all got the same reward | 911 / 1446 | forks | 0.630 unanimous: zero continuation advantage |
Host: GPU p4d.24xlarge spot (8 x A100 40 GB); env c8i.24xlarge (fork capture on a 48 GiB tmpfs), Ubuntu 24.04, kernel 7.0.0-1014-aws. same instances as the L1 control (EXP-029). Workload: Same tasks, evaluator and budget (W = 480 min trained) as L1 Phase A; tree GRPO with 2 trunks x 4 forked children per group, fork after turn 6. Model: ScaleCUA-qwen3.5-osworld-sft rev ff2508f6 (9B). Method: Pre-registered 2026-11-08T07:56:20Z (sha256 e095e7b2...). Primary: trained time at the first complete band K = 9 evaluation >= 0.8634, censored at 28,811 s; ratio = t_branch / 27,832 s. Prices 21.96048 USD/h on both arms. Paired task bootstrap, 10,000 resamples, seed 20261104.. Source: research archive: runbooks/rl-branch/20261108T0656Z/REPORT.md; research archive: runbooks/rl-branch/20261108T0656Z/analysis/crossing/learn_phaseb_crossing.json; research archive: runbooks/rl-branch/20261108T0656Z/analysis/branch/branch_analysis.json; research archive: runbooks/rl-branch/20261108T0656Z/analysis/util/util.json.
- One pair, run four days apart. The run is in two segments (disk-full resume, D8; the void interval of 6,870.4 s is removed from the clock).
- The control's 27,832 s is an upper bound. Checkpoint-to-checkpoint noise looks like about +-0.1 (the step 24 dip).
- Only one setting was tested (2 trunks, 4 children, fork at turn 6). This does not show that tree rollouts cannot help with other settings.
Failures and surprises
- D8: environment disk full (slot state grew from 85 to 131 GB); steps 22 to 71 skipped every batch until the resume.
- D9: the evaluation launcher did not forward its run prefix; two evaluations landed in the control's folder and were moved unchanged.
- A run-log note that step 28 was outside the censor was wrong and was corrected before any analysis.
Limitations
One pair, four days apart, in two segments. The control's 27,832 s is an upper bound. The step 24 dip suggests checkpoint noise of about 0.1 at N = 18 tasks. More trunks, later forks or value-targeted forks were not tested. Study spend about 483 USD (estimate).
Reproduce
REPRODUCE.md in the study folder; analysis from raw/ only with analysis/branch_crossing.py, branch_analysis.py and util_analysis.py (commands in the report). A full rerun is about 15 to 17 h of both hosts, about 350 USD on spot.
Sources
Pre-registration, report, run log and analysis outputs under Evidence. Research archive, internal for now.