BEHAVIOR-1K project recap, Sep 30 to Nov 1

Every number comes from the handoff (docs/HANDOFF_DISTILL.md on the notes branch), the issues and PRs of KantaS12/Behavior1k-Challenge-Tests, or the world model notes. The full write-up with more tables is docs/PROJECT_RECAP.md on the notes branch. All times are HST.

Before the group's 2026 stack existed, we ran pt50 on sorting_vegetables to learn what limits it. One flow step collapses (q 0.017 against 0.492 at 10 steps). WarmPrior keeps q at 1 to 2 steps (0.123 against 0.038 at 1 step, p 0.03), but every model we fine-tuned from pt50 scored about half of pt50. The audit traced that to our learning rate, 2.5e-5, ten times Comet's SFT rate, after a zero-step round trip proved the pipeline itself bit-identical. A prompt probe showed pt50 barely uses its prompt (subtask text raises action error by 19%, nonsense by 18%). The stage head, RFT pipeline and 100-task fine-tune were built but never run at scale before the Oct 3 restart.

In the v3.9.3 batched evaluator the first observation is often a stale spawn render, the same in every env. pt50 then closes its left gripper on nothing and never drives, which explained why turning_on_radio scored 0. Replaying the stale request closed the gripper in 32 of 32 chunks and the fresh one in 0 of 32. Stale batches were 0 of 20 with 1 env and 5 of 6 with 4, because each extra env adds 3 cameras and pushes the first fresh frame past the reset's 4 render updates. Details and videos are on the radio stale frame page.

SnapFlow self-distills pt50's flow sampler to one step. The 30k-step student halves pt50's 1-step error offline (0.00144 against 0.00278 MSE to the 10-step teacher), but in sim the 1-step student loses 0.233 Q against pt50 at 10 steps (CI -0.408 to -0.058) and every arm spends about 2.7 s per request, since the VLM prefix dominates. See the SnapFlow sim page.

A timer planner with the team's follower scored a total Q of 1.0 on 5 tasks against 9.33 for pt50, and the team's own run with a privileged phase detector got Q 0.017 against pt50's 0.019 on the new 50. Even perfect step switching does not help, so the follower finishing steps is the limit. See planner against pt50 and the 99-task submission.

After the deadline the goal became better training data. A scripted recoverer with privileged sim state restores the state where pt50 stalled, finishes the next phase and records it. A round counts clean recordings, which pass the phase detector and the harm gate, on development cases and on a frozen set of 10 held-out cases from 8 tasks never run or diagnosed. Replays are on the recovery replays page.

cuRobo was the only change that moved the held-out yield. The dev fixes that lifted dev from 2 to 4 of 5 did not transfer, and the v6n drop came from run-to-run variance in IK stances, not a code bug. The recoverer is now a part-time data source.

Three of the top four verified leaderboard rows use a world action model, which starts from a video model and predicts video and actions together. Flex-pi was picked as the best fit in the WAM comparison. Its LoRA pilot (5 tasks, 8,000 updates, 10.6 L40 GPU-hours) learned the grippers but not the arms.

The diagnosis on Nov 1 found no time or delta bug. The predicted arm displacement correlates only 0.04 with the demo, because q01 and q99 action normalization squeezed arm deltas to a std of 0.09 in [-1, 1]. The config now uses z-score actions, and the retrain, pilot6 (job 15295531, about 10.5 L40 GPU-hours), was submitted on Nov 1 with a Slurm start estimate of about 8:30 PM HST Nov 2. MolmoAct2 has its data converted (10,000 episodes, 119 million frames), and its smoke test, job 15293671, waits for an H200 with a Slurm estimate of about 1:25 PM HST Nov 5.

• Ten episodes per arm resolve only about 0.2 Q. Pair arms by instance and report CIs.
• Capture raw inputs first. The stale frame stayed hidden until the first two requests were saved.
• A converged checkpoint needs a small fine-tune learning rate, or it drifts without learning.
• Fewer flow steps buy no speed on BEHAVIOR, since the VLM prefix dominates each call.
• pt50 barely listens to its prompt, so text planners cannot steer it.
• Tune on dev, report on a frozen held-out set. Only a real motion planner moved the held-out yield.
• Compare offline predictions against hold and repeat-last baselines before spending sim time.

• Flex-pi pilot6 retrain with z-score actions, job 15295531, pending. Its pass bar is arms and torso beating hold-still.
• MolmoAct2 smoke test, job 15293671, pending for an H200.
• Recoverer v6o with two seeds (about 3.6 GPU-hours) paused, PRs 115 to 124 open.
• Winners are announced Nov 4. No WAM team has published its method yet.