BEHAVIOR-1K 2026 challenge, issues 74 and 83, recovery failures as of Oct 23
Before collecting more recovery data, this page shows why the scripted recoverers fail. On the 44 pilot cases from pt50's near misses, 110 of the latest 132 attempts failed. They are grouped below by cause and by case, so seeds that failed the same way share one clip. 35 of the failed attempts end with the robot holding something other than its target, mostly the basket, gift box or shoe pt50 was carrying, which comes up in several causes. Please mark each group and add a note. The marks stay in this browser, and Copy all feedback turns them into one block to paste back to me.
Attempts count each case and seed once, using its latest run (first pass, stall-fix rerun or the v2 run on Oct 22 night). Attempts that a later run passed are not counted.
| cause | attempts | cases | tasks |
|---|---|---|---|
| Target not visible after looking | 30 | 10 | collecting_childrens_toys, preparing_lunch_box, putting_up_Christmas_decorations_inside, sorting_vegetables |
| Finished, but rejected for knocking other things | 17 | 6 | assembling_gift_baskets, preparing_lunch_box, putting_up_Christmas_decorations_inside |
| Arm stuck a few centimetres short | 16 | 6 | collecting_childrens_toys, preparing_lunch_box, sorting_vegetables |
| Placed, but the sim does not count it as inside | 10 | 5 | putting_shoes_on_rack |
| Ran out of time carrying the object | 9 | 3 | putting_shoes_on_rack, sorting_vegetables |
| Other timeouts | 12 | 8 | assembling_gift_baskets, preparing_lunch_box, putting_shoes_on_rack, sorting_vegetables |
| Robot tipped over | 5 | 4 | putting_shoes_on_rack, putting_up_Christmas_decorations_inside, sorting_vegetables |
| No grasp fits the gripper | 6 | 3 | putting_up_Christmas_decorations_inside, sorting_vegetables |
| Out of reach for the arm | 5 | 3 | collecting_childrens_toys, preparing_lunch_box, sorting_vegetables |
| all | 110 | 48 case and cause groups |
30 attempts on 10 cases. The recoverer looks for the target twice and gives up when no camera sees enough of it (0.01% of the head image or 0.1% of a wrist image). Every attempt here stays below that in all three cameras, before and after both looks, and most read exactly 0. The look turns the base to face the target (its heading error goes to zero) and leans the torso, then looks again leaning further, but it never drives closer. In 8 of these 10 cases the target starts 3 to 6.3 m from the hand, and in the other two it is 1.5 m away. In three of the four sorting_vegetables cases the robot is also still holding the wicker basket pt50 picked up, which fills the wrist cameras, and in putting_up_Christmas_decorations_inside 202 it holds a gift box. The bottle of tea in preparing_lunch_box is inside the closed fridge, so the next phase should have been to open the fridge. Every seed of a case took exactly the same number of steps, so the seed noise never changes the outcome.
What might fix it. Drive toward the target before looking, like the issue 76 look-around search, which drives to working distance and found the target 38 of 40 times. Also put down what the robot holds first, and label a closed container as an open phase.
The phase is to pick up the dice from the bed. The log says not visible after looking. The target starts 1.5 m from the nearest hand. What the clip shows. The robot turns to face the dice on the bed, 1.5 m away, but all three cameras still show the bookcase. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 18.7 s, 18.7 s, 18.7 s. The clip is seed 1.
The phase is to pick up the bottle of tea from the fridge. The log says not visible after looking. The target starts 3.8 m from the nearest hand. What the clip shows. The bottle of tea is inside the closed fridge. Seeds 0, 1 and 2 failed this way (first pass), lasting 18.2 s, 18.2 s, 18.2 s. The clip is seed 1.
The phase is to pick up the bottle of tea from the fridge. The log says not visible after looking. The target starts 3.9 m from the nearest hand. What the clip shows. The bottle of tea is inside the closed fridge. Seeds 0, 1 and 2 failed this way (first pass), lasting 18.2 s, 18.2 s, 18.2 s. The clip is seed 1.
The phase is to pick up the pillar candle from the sofa. The log says not visible after looking. The target starts 6.3 m from the nearest hand. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 17.7 s, 17.7 s, 17.7 s. The clip is seed 1.
The phase is to pick up the candy cane from the sofa. The log says not visible after looking. The target starts 4.5 m from the nearest hand. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 14.5 s, 14.5 s, 14.5 s. The clip is seed 1.
The phase is to place the pillar candle on the breakfast table. The log says not visible after looking. The target starts 1.5 m from the nearest hand. What the clip shows. The label asks for a place, but the robot holds nothing and the candle rests in the wicker basket. Seeds 0, 1 and 2 failed this way (first pass), lasting 18.7 s, 18.7 s, 18.7 s. The clip is seed 1.
The phase is to pick up the broccoli from the wicker basket. The log says not visible after looking. The target starts 4.4 m from the nearest hand. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 15.5 s, 15.5 s, 15.5 s. The clip is seed 1.
The phase is to pick up the sweet corn from the wicker basket. The log says not visible after looking. The target starts 3.7 m from the nearest hand. What the clip shows. The wicker basket in the right hand fills both wrist views, and the head camera shows the kitchen with no sweet corn in it. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 16.6 s, 16.6 s, 16.6 s. The clip is seed 1.
The phase is to pick up the sweet corn from the wicker basket. The log says not visible after looking. The target starts 3.0 m from the nearest hand. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 13.4 s, 13.4 s, 13.4 s. The clip is seed 1.
The phase is to pick up the broccoli from the wicker basket. The log says not visible after looking. The target starts 3.7 m from the nearest hand. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 15.0 s, 15.0 s, 15.0 s. The clip is seed 1.
17 attempts on 6 cases. The recoverer finished, and in 16 of the 17 attempts the phase detector confirmed the pick, but the disturbance gate threw the recording away because something else moved too far while it worked. In putting_up_Christmas_decorations_inside 205 seed 0 the detector could not be applied at all. The gate rejects an attempt when the object the target rests on moves more than 2 cm or anything else moves more than 10 cm. In assembling_gift_baskets the coffee table itself moves 0.2 to 0.8 m and everything on it slides with it.
What might fix it. Some of these may be fine to keep. preparing_lunch_box 201 only moved the cookie, the sandwich and the packing box 11 to 18 cm. Your call on where the line should be.
The phase is to pick up the pillar candle from the coffee table. The log says no failure reason recorded (the oracle ran out of time or reported done). Moved the most: butter_cookie_84 0.33 m, swiss_cheese_80 0.29 m, butter_cookie_81 0.27 m, bow_73 0.27 m. Seeds 0, 1 and 2 failed this way (first pass), lasting 24.6 s, 23.0 s, 21.7 s. The clip is seed 1.
The phase is to pick up the pillar candle from the coffee table. The log says no failure reason recorded (the oracle ran out of time or reported done). Moved the most: butter_cookie_84 0.99 m, bow_75 0.96 m, butter_cookie_82 0.95 m, bow_74 0.93 m. What the clip shows. The coffee table and everything on it move 0.4 to 1.0 m. Seeds 0, 1 and 2 failed this way (first pass), lasting 28.5 s, 25.6 s, 24.3 s. The clip is seed 1.
The phase is to pick up the half apple from the chopping board. The log says no failure reason recorded (the oracle ran out of time or reported done). Moved the most: chocolate_chip_cookie_207 0.18 m, club_sandwich_209 0.14 m, packing_box_210 0.14 m. Seeds 0, 1 and 2 failed this way (first pass), lasting 15.0 s, 12.3 s, 12.5 s. The clip is seed 2.
The phase is to pick up the pillar candle from the sofa. The log says no failure reason recorded (the oracle ran out of time or reported done). Moved the most: pillar_candle_222 0.59 m. Seeds 0, 1 and 2 failed this way (first pass, stall-fix rerun), lasting 17.3 s, 9.3 s, 10.7 s. The clip is seed 2.
The phase is to pick up the pillar candle from the sofa. The log says no failure reason recorded (the oracle ran out of time or reported done). Moved the most: candy_cane_226 1.28 m, pillar_candle_222 1.26 m. Seeds 0, 1 and 2 failed this way (first pass), lasting 11.1 s, 11.6 s, 12.3 s. The clip is seed 1.
The phase is to pick up the pillar candle from the sofa. The log says no failure reason recorded (the oracle ran out of time or reported done). Moved the most: candy_cane_226 0.19 m, wreath_227 0.17 m. Seeds 0 and 1 failed this way (first pass), lasting 30.0 s, 36.5 s. The clip is seed 1.
16 attempts on 6 cases. The hand gets within 5 to 14 cm of the grasp and stops closing in. The stall rule then backs off and tries a new base pose. In the v2 runs it does four repositions, replans from scratch and does four more before giving up. Most of these are on the chopping board in preparing_lunch_box and the bed in collecting_childrens_toys, where the log says there is no free side to stand at (side_plan_none, base_blocked), so the robot keeps reaching from the same bad spot.
What might fix it. A better base placement at the counter and the bed. These attempts already use a side grasp, including the board game.
The phase is to pick up the board game from the bed. The log says reach plateau in approach: goal error stuck at 0.07 m for 3 s after N reposition(s); reach plateau in descend: goal error stuck at 0.14 m for 3 s after N reposition(s); reach plateau in approach: goal error stuck at 0.11 m for 3 s after N reposition(s). Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 68.3 s, 82.2 s, 66.9 s. The clip is seed 0.
The phase is to pick up the board game from the bed. The log says reach plateau in approach: goal error stuck at 0.06 m for 3 s after N reposition(s); reach plateau in approach: goal error stuck at 0.07 m for 3 s after N reposition(s); reach plateau in descend: goal error stuck at 0.08 m for 3 s after N reposition(s). Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 65.9 s, 56.1 s, 68.5 s. The clip is seed 0.
The phase is to pick up the chocolate chip cookie from the chopping board. The log says reach plateau in descend: goal error stuck at 0.07 m for 3 s after N reposition(s); reach plateau in descend: goal error stuck at 0.11 m for 3 s after N reposition(s); reach plateau in approach: goal error stuck at 0.12 m for 3 s after N reposition(s). Moved the most: half_apple_213 0.20 m, chopping_board_211 0.15 m. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 55.1 s, 48.4 s, 44.4 s. The clip is seed 1.
The phase is to pick up the club sandwich from the chopping board. The log says reach plateau in descend: goal error stuck at 0.12 m for 3 s after N reposition(s); reach plateau in descend: goal error stuck at 0.11 m for 3 s after N reposition(s). Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 70.1 s, 58.8 s, 63.5 s. The clip is seed 2.
The phase is to pick up the chocolate chip cookie from the chopping board. The log says reach plateau in approach: goal error stuck at 0.06 m for 3 s after N reposition(s); reach plateau in approach: goal error stuck at 0.09 m for 3 s after N reposition(s). Moved the most: club_sandwich_209 0.33 m, chopping_board_211 0.18 m. Seeds 0, 1 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 49.2 s, 48.8 s, 47.4 s. The clip is seed 1.
The phase is to pick up the broccoli from the wicker basket. The log says reach plateau in descend: goal error stuck at 0.05 m for 3 s after N reposition(s). Moved the most: leek_207 1.84 m, leek_208 1.76 m, vidalia_onion_216 1.75 m, vidalia_onion_215 1.70 m. Seed 1 failed this way (first pass), lasting 59.9 s. The clip is seed 1.
10 attempts on 5 cases. The place routine let go of the shoe and backed off, but the detector still says the phase is in progress. The labelled destination is the hall tree. In 9 of the 10 attempts the shoe ends about 0.6 m up on the bench seat, which the sim does not count as inside, and in one it falls to the floor. The team's original place routine failed all 21 earlier shoe attempts too, each at the 75 s timeout.
What might fix it. Treat the hall tree as unsupported for now, or aim the place at the low shelf from the side.
The phase is to place the sandal in the hall tree. The log says no failure reason recorded (the oracle ran out of time or reported done). What the clip shows. The sandal is set down on the bench seat. Seeds 0, 1 and 2 failed this way (first pass), lasting 25.6 s, 30.7 s, 22.2 s. The clip is seed 0.
The phase is to place the sandal in the hall tree. The log says no failure reason recorded (the oracle ran out of time or reported done). Seeds 0, 1 and 2 failed this way (first pass), lasting 12.6 s, 16.9 s, 14.1 s. The clip is seed 2.
The phase is to place the gym shoe in the hall tree. The log says no failure reason recorded (the oracle ran out of time or reported done). Seeds 0 and 2 failed this way (first pass), lasting 17.7 s, 17.9 s. The clip is seed 2.
The phase is to place the sandal in the hall tree. The log says no failure reason recorded (the oracle ran out of time or reported done). Seed 0 failed this way (first pass), lasting 21.5 s. The clip is seed 0.
The phase is to place the sandal in the hall tree. The log says no failure reason recorded (the oracle ran out of time or reported done). Seed 1 failed this way (first pass), lasting 64.6 s. The clip is seed 1.
9 attempts on 3 cases. The object is in the hand, but the robot never gets it to the destination in the 75 s budget. For the shoes the hall tree is still 2.7 to 4.6 m away when time runs out, and after the carry stalls no base plan is found. In sorting_vegetables 200 the robot carries the whole wicker basket with both hands and the vegetables spill out.
What might fix it. Base navigation while carrying, and whether a basket held by pt50 should be put down first.
The phase is to place the gym shoe in the hall tree. The log says timeout in carry, goal error 2.81 m; timeout in carry, goal error 2.72 m; timeout in carry, goal error 2.76 m. Seeds 0, 1 and 2 failed this way (first pass), lasting 75.0 s, 75.0 s, 75.0 s. The clip is seed 1.
The phase is to place the gym shoe in the hall tree. The log says timeout in carry, goal error 4.61 m; timeout in carry, goal error 4.45 m; timeout in carry, goal error 4.62 m. Seeds 0, 1 and 2 failed this way (first pass), lasting 75.0 s, 75.0 s, 75.0 s. The clip is seed 1.
The phase is to place the wicker basket on the floors. The log says timeout in carry, goal error 0.91 m; timeout in carry, goal error 0.75 m; timeout in carry, goal error 0.63 m. Moved the most: leek_208 1.66 m, broccoli_210 1.65 m, broccoli_209 1.56 m, vidalia_onion_215 1.51 m. What the clip shows. The robot carries the whole basket with both hands and the vegetables spill. Seeds 0, 1 and 2 failed this way (first pass), lasting 75.0 s, 75.0 s, 75.0 s. The clip is seed 1.
12 attempts on 8 cases. These hit the time cap in the approach, base or lower stage, or before the stall rule existed (no reason recorded). 6 of the 12 are putting_shoes_on_rack, 5 are on the cluttered chopping board or next to a basket held in the other hand, and 1 is assembling_gift_baskets.
The phase is to pick up the chocolate chip cookie from the chopping board. The log says timeout in approach, goal error 0.10 m. Moved the most: chopping_board_211 0.11 m. Seeds 0 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 90.0 s, 90.0 s. The clip is seed 2.
The phase is to place the sandal in the hall tree. The log says no failure reason recorded (the oracle ran out of time or reported done). Seeds 1 and 2 failed this way (first pass), lasting 75.0 s, 75.0 s. The clip is seed 2.
The phase is to pick up the sandal from the floors. The log says timeout in base, goal error nan m; timeout in approach, goal error 0.13 m. Moved the most: gym_shoe_78 0.72 m, sandal_76 0.40 m. Seeds 0 and 2 failed this way (recoverer v2 on Oct 22 night), lasting 110.0 s, 110.0 s. The clip is seed 2.
The phase is to pick up the broccoli from the wicker basket. The log says timeout in base, goal error nan m; timeout in approach, goal error 0.25 m. Moved the most: vidalia_onion_215 0.82 m, vidalia_onion_216 0.70 m. Seeds 1 and 2 failed this way (first pass), lasting 40.0 s, 40.0 s. The clip is seed 2.
The phase is to place the pillar candle in the wicker basket. The log says no failure reason recorded (the oracle ran out of time or reported done). Seed 0 failed this way (first pass), lasting 75.0 s. The clip is seed 0.
The phase is to place the sandal in the hall tree. The log says no failure reason recorded (the oracle ran out of time or reported done). Seed 0 failed this way (first pass), lasting 75.0 s. The clip is seed 0.
The phase is to place the gym shoe in the hall tree. The log says timeout in lower, goal error 0.20 m. Seed 1 failed this way (first pass), lasting 75.0 s. The clip is seed 1.
The phase is to pick up the broccoli from the wicker basket. The log says timeout in base, goal error nan m. Moved the most: vidalia_onion_214 1.29 m, bok_choy_218 1.14 m, broccoli_210 1.00 m, bok_choy_217 0.93 m. Seed 2 failed this way (first pass), lasting 40.0 s. The clip is seed 2.
5 attempts on 4 cases. The base tilted more than 0.3 rad from where it started and the attempt was stopped. In three of the four cases the robot still holds something pt50 picked up (a gift box, a gym shoe or the wicker basket).
What might fix it. Check base clearance before driving, and put down held objects first.
The phase is to pick up the pillar candle from the sofa. The log says robot tipped (0.35 rad); robot tipped (0.31 rad). Seeds 0 and 2 failed this way (first pass), lasting 14.8 s, 14.8 s. The clip is seed 2.
The phase is to pick up the sandal from the floors. The log says robot tipped (0.32 rad). Moved the most: sandal_76 0.49 m, gym_shoe_78 0.46 m. Seed 1 failed this way (recoverer v2 on Oct 22 night), lasting 66.0 s. The clip is seed 1.
The phase is to place the sandal in the hall tree. The log says robot tipped (0.36 rad). Seed 2 failed this way (first pass), lasting 30.0 s. The clip is seed 2.
The phase is to pick up the broccoli from the wicker basket. The log says robot tipped (0.31 rad). Moved the most: leek_207 1.80 m, vidalia_onion_215 1.76 m, vidalia_onion_214 1.09 m, broccoli_210 0.18 m. Seed 0 failed this way (first pass), lasting 30.6 s. The clip is seed 0.
6 attempts on 3 cases. The gripper opens 11 cm. The gift box is about 20 cm on every side, so no grasp fits. The broccoli is 9 cm across, close to the opening, and sits in the wicker basket the robot holds in its other hand.
What might fix it. The gift box needs a two-handed or push strategy. For the broccoli, put the basket down first.
The phase is to pick up the gift box from the floors. The log says no grasp fits the gripper (box [0.205, 0.196, 0.187] m, opening 0.11 m). Seeds 0, 1 and 2 failed this way (first pass), lasting 4.8 s, 4.8 s, 4.8 s. The clip is seed 1.
The phase is to pick up the broccoli from the wicker basket. The log says no grasp fits the gripper (box [0.091, 0.09, 0.061] m, opening 0.11 m). Moved the most: leek_207 1.28 m, vidalia_onion_214 1.27 m, vidalia_onion_215 1.19 m. Seeds 0 and 2 failed this way (first pass), lasting 12.7 s, 7.9 s. The clip is seed 0.
The phase is to pick up the broccoli from the wicker basket. The log says no grasp fits the gripper (box [0.091, 0.09, 0.061] m, opening 0.11 m). Moved the most: vidalia_onion_215 0.66 m, vidalia_onion_216 0.64 m. Seed 0 failed this way (first pass), lasting 26.2 s. The clip is seed 0.
5 attempts on 3 cases. The inverse kinematics cannot reach the pre-grasp within the joint limits, missing by 2 to 11 cm, from every base pose the recoverer tried.
What might fix it. Same as the plateau group, a better base pose.
The phase is to pick up the toy train from the desk. The log says out of reach (inverse kinematics within the joint limits misses the pre-grasp by 0.05 m, the grasp by 0.05 m); out of reach (inverse kinematics within the joint limits misses the pre-grasp by 0.04 m, the grasp by 0.02 m). Seeds 0, 1 and 2 failed this way (first pass), lasting 11.9 s, 12.4 s, 12.4 s. The clip is seed 1.
The phase is to pick up the chocolate chip cookie from the chopping board. The log says out of reach (inverse kinematics within the joint limits misses the pre-grasp by 0.06 m, the grasp by 0.04 m). Moved the most: half_apple_212 0.50 m, club_sandwich_209 0.38 m, half_apple_213 0.30 m, chopping_board_211 0.28 m. Seed 1 failed this way (recoverer v2 on Oct 22 night), lasting 77.7 s. The clip is seed 1.
The phase is to pick up the broccoli from the wicker basket. The log says out of reach (inverse kinematics within the joint limits misses the pre-grasp by 0.11 m, the grasp by 0.11 m). Moved the most: vidalia_onion_215 1.76 m, vidalia_onion_214 1.72 m, leek_207 1.56 m. Seed 1 failed this way (first pass), lasting 22.9 s. The clip is seed 1.
These four recordings are the recover-and-continue chains, which keep going after the recovered pick. In all four the recoverer picked the toy (the dice in case 202, the toy train in case 204 alt) and the detector confirmed it. pt50 then took over to place it in the bookcase. In the three dice recordings it got stuck going back and forth at the shelves for 75 to 111 s until the stall rule handed over, and in the toy train recording it ran to its 90 s cap. The place recoverer then used its whole 75 s without the detector confirming the place. In the dice recordings its log shows the carry stalling, a base move and a drop spot on a shelf 0.4 to 0.7 m up. On dice seed 1 it got as far as letting go and backing off, and the place was still not confirmed. So only the pick in each recording can be used. An earlier toy train test recording is left out, because that collector version marked every later phase finished after one step. The clips start at the hand-over to pt50 and run at 2x speed.
The same recoverers on the team's new-task harvest cases, with the team's follower policy instead of pt50 after the hand-over.
Both recoverers tip the robot next to the desk, v2 after 1.1 s and the v1 oracle after 16.2 s with a 0.31 rad tilt. The v1 log shows the base tilting on each of its three repositions against the desk.
Recoverer v2 started with a top grasp, switched to a handle grasp and then a side grasp, stalled 0.33 to 0.45 m from its goal and timed out 0.55 m away. The v1 oracle kept the top grasp and picked it in 12.5 s, so this one is already solved by using v1 for picks where v2 fails.
After the recoverer put the plate in the sink, the follower picked up the tupperware on its own. On the next phase it stood in front of the closed fridge and went back and forth for 60 s. The fridge was never opened, because the phase list this run used has no "open the fridge" step, and the collector has no recoverer for a "place in" phase. The issue 82 phase engine now keeps an open phase current, but this run used the older tracker.
The panels are, from left to right, the left wrist camera, the right wrist camera and the head camera. Clips load only when played. Pilot clips come from attempt.mp4 under recov50/runs/{pilot,rerun_plateau,a_cont}, chain clips from the LeRobot recordings, and the counts from each attempt's result.json. The work is tracked in issue 74 and issue 83.