Active search, before and after | Kanta Saito

The old recoverer failed these cases with "not visible after looking": it only turned the base toward the target and leaned. The new search (PR #87) drives to stances near where the target should be, then the start spot, then room centres, and at each spot turns through 8 headings, checks the head camera at two torso leans and points each free arm's wrist camera forward, out and down. Each case shows the old attempt on the left and the new one on the right, seed 0. So far 7 of 10 searches found the target and 1 of 10 cases pass. The two preparing_lunch_box cases have the bottle of tea inside the closed fridge, which needs an open phase first. These runs use privileged sim state and are not challenge-legal.

Results

The clips

case	phase	before	after	found	seen by
collecting_childrens_toys 200	pick up the dice from the bed	not visible after looking	pass	yes	head
preparing_lunch_box 203 alt	pick up the bottle of tea from the fridge	not visible after looking	inside the closed fridge: needs an open phase first	no	-
preparing_lunch_box 207 alt	pick up the bottle of tea from the fridge	not visible after looking	inside the closed fridge: needs an open phase first	no	-
putting_up_Christmas_decorations_inside 201	pick up the pillar candle from the sofa	not visible after looking	out of reach (inverse kinematics within the joint limits misses the pre-grasp by 1.20 m, the grasp by 1.20 m)	yes	head
putting_up_Christmas_decorations_inside 202	pick up the candy cane from the sofa	not visible after looking	reach plateau in approach: goal error stuck at 0.09 m for 3 s after 2 reposition(s)	yes	left wrist
putting_up_Christmas_decorations_inside 204	place the pillar candle on the breakfast table	not visible after looking	timeout in base, goal error nan m	yes	head
sorting_vegetables 201	pick up the broccoli from the wicker basket	not visible after looking	not found after searching 2 spots	no	-
sorting_vegetables 204	pick up the sweet corn from the wicker basket	not visible after looking	reach plateau in approach: goal error stuck at 0.11 m for 3 s after 2 reposition(s)	yes	head
sorting_vegetables 206	pick up the sweet corn from the wicker basket	not visible after looking	finished, but rejected: disturbed 1 other object(s): sweet_corn_212 0.031 m	yes	head
sorting_vegetables 207	pick up the broccoli from the wicker basket	not visible after looking	reach plateau in approach: goal error stuck at 0.06 m for 3 s after 2 reposition(s)	yes	head

collecting_childrens_toys 200, pick dice_269

now passes

The phase is to pick up the dice from the bed. Before: not visible after looking. After: pass. Spots: support bed_ivdnny_0 (already there). It found the target at view 2 after 8 s, with the head camera (torso lean down). The drive toward it was there. It ended 0.8 m from the target. Search time 9 s.

before, recoverer v2 run, seed 0, real time, 19 s

1. Plan. Where to look: the bed_ivdnny_0 named in the phase ("pick up the dice from the bed"). At each spot: 8 headings, the head at two torso leans, and the wrist cameras of free arms pointed forward, out, down. Time limit 120 s.

Spot 1: support bed_ivdnny_0 at (21.2, 6.8), drive already there

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 9 s

3. The recovery. The pick or place after the search.

after, the recovery, real time, 8 s

Is the new behaviour better?

preparing_lunch_box 203 alt, pick bottle_of_tea_208

still fails

The phase is to pick up the bottle of tea from the fridge. Before: not visible after looking. After: inside the closed fridge: needs an open phase first. The search stopped at once, because the target is inside the closed fridge and needs an open phase.

before, first pass, seed 0, real time, 18 s

No new video: the attempt stopped before moving.

Is the new behaviour better?

preparing_lunch_box 207 alt, pick bottle_of_tea_208

still fails

before, first pass, seed 0, real time, 18 s

No new video: the attempt stopped before moving.

Is the new behaviour better?

putting_up_Christmas_decorations_inside 201, pick pillar_candle_223

found, then failed

The phase is to pick up the pillar candle from the sofa. Before: not visible after looking. After: out of reach (inverse kinematics within the joint limits misses the pre-grasp by 1.20 m, the grasp by 1.20 m). Spots: support sofa_hiphpn_0 (drive blocked). It found the target at view 36 after 74 s, with the head camera (torso lean down). The drive toward it was blocked. It ended 3.0 m from the target. Search time 81 s.

before, recoverer v2 run, seed 0, real time, 18 s

1. Plan. Where to look: the sofa_hiphpn_0 named in the phase ("pick up the pillar candle from the sofa"). At each spot: 8 headings, the head at two torso leans, and the wrist cameras of free arms pointed forward, out, down. Time limit 120 s.

Spot 1: support sofa_hiphpn_0 at (13.6, 0.5), drive drive blocked

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 81 s

3. The recovery. The pick or place after the search.

after, the recovery, real time, 1 s

Is the new behaviour better?

putting_up_Christmas_decorations_inside 202, pick candy_cane_226

found, then failed

The phase is to pick up the candy cane from the sofa. Before: not visible after looking. After: reach plateau in approach: goal error stuck at 0.09 m for 3 s after 2 reposition(s). Spots: support sofa_lugrhk_1 (reached). It found the target at view 34 after 75 s, with the left wrist camera, with the left arm pointed left (torso lean down). It ended 1.0 m from the target. Search time 80 s.

before, recoverer v2 run, seed 0, real time, 14 s

1. Plan. Where to look: the sofa_lugrhk_1 named in the phase ("pick up the candy cane from the sofa"). At each spot: 8 headings, the head at two torso leans, and the wrist cameras of free arms pointed forward, out, down. Time limit 120 s.

Spot 1: support sofa_lugrhk_1 at (11.9, 1.7), drive reached

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 80 s

3. The recovery. The pick or place after the search.

after, the recovery, real time, 36 s

Is the new behaviour better?

putting_up_Christmas_decorations_inside 204, place pillar_candle_223

found, then failed

The phase is to place the pillar candle on the breakfast table. Before: not visible after looking. After: timeout in base, goal error nan m. Spots: support wicker_basket_218 (drive blocked); support wicker_basket_218 (reached). It found the target at view 41 after 86 s, with the head camera (torso lean up). It ended 0.9 m from the target. Search time 86 s.

before, first pass, seed 0, real time, 19 s

1. Plan. Where to look: the wicker_basket_218 named in the phase ("place the pillar candle on the breakfast table"). At each spot: 8 headings, the head at two torso leans, and the wrist cameras of free arms pointed forward, out, down. Time limit 120 s.

Spot 1: support wicker_basket_218 at (11.9, 1.6), drive drive blocked
Spot 2: support wicker_basket_218 at (12.8, 2.9), drive reached

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 86 s

3. The recovery. The pick or place after the search.

after, the recovery, real time, 75 s

Is the new behaviour better?

sorting_vegetables 201, pick broccoli_210

still fails

The phase is to pick up the broccoli from the wicker basket. Before: not visible after looking. After: not found after searching 2 spots. Spots: support wicker_basket_220 (drive blocked); support wicker_basket_220 (drive blocked). It did not find the target in 57 views. Search time 127 s.

before, recoverer v2 run, seed 0, real time, 16 s

1. Plan. Where to look: the wicker_basket_220 named in the phase ("pick up the broccoli from the wicker basket"). At each spot: 8 headings, the head at two torso leans, and the wrist cameras of free arms pointed forward, out, down. Time limit 120 s.

Spot 1: support wicker_basket_220 at (9.0, 2.6), drive drive blocked
Spot 2: support wicker_basket_220 at (7.7, 1.4), drive drive blocked

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 127 s

3. The recovery. The pick or place after the search.

The recovery stopped at once.

Is the new behaviour better?

sorting_vegetables 204, pick sweet_corn_212

found, then failed

The phase is to pick up the sweet corn from the wicker basket. Before: not visible after looking. After: reach plateau in approach: goal error stuck at 0.11 m for 3 s after 2 reposition(s). Spots: support wicker_basket_221 (reached). It found the target at view 1 after 13 s, with the head camera (torso lean up). The drive toward it was there. It ended 0.8 m from the target. Search time 15 s.

before, recoverer v2 run, seed 0, real time, 17 s

1. Plan. Where to look: the wicker_basket_221 named in the phase ("pick up the sweet corn from the wicker basket"). At each spot: 8 headings, the head at two torso leans, and the wrist cameras of free arms pointed forward, out, down. Time limit 120 s.

Spot 1: support wicker_basket_221 at (9.1, 1.4), drive reached

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 15 s

3. The recovery. The pick or place after the search.

after, the recovery, real time, 32 s

Is the new behaviour better?

sorting_vegetables 206, pick sweet_corn_213

found, then failed

The phase is to pick up the sweet corn from the wicker basket. Before: not visible after looking. After: finished, but rejected: disturbed 1 other object(s): sweet_corn_212 0.031 m. Spots: support wicker_basket_221 (reached). It found the target at view 1 after 8 s, with the head camera (torso lean up). It ended 0.9 m from the target. Search time 9 s.

before, recoverer v2 run, seed 0, real time, 13 s

Spot 1: support wicker_basket_221 at (4.9, 1.1), drive reached

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 9 s

3. The recovery. The pick or place after the search.

after, the recovery, real time, 11 s

Is the new behaviour better?

sorting_vegetables 207, pick broccoli_209

found, then failed

The phase is to pick up the broccoli from the wicker basket. Before: not visible after looking. After: reach plateau in approach: goal error stuck at 0.06 m for 3 s after 2 reposition(s). Spots: support wicker_basket_220 (tilted); support wicker_basket_220 (drive blocked). It found the target at view 42 after 88 s, with the head camera (torso lean down). It ended 0.6 m from the target. Search time 90 s.

before, recoverer v2 run, seed 0, real time, 15 s

Spot 1: support wicker_basket_220 at (9.1, 0.1), drive tilted
Spot 2: support wicker_basket_220 at (9.8, -1.6), drive drive blocked

A top-down map of the plan comes with the next run.

2. What it does. The search, from the restore until it sees the target and drives up.

after, the search, real time, 91 s

3. The recovery. The pick or place after the search.

after, the recovery, real time, 39 s

Is the new behaviour better?

Clips are the attempt videos at real time. Left wrist, right wrist and head camera from left to right. Results come from result.json under recov50/runs/search_smoke and the earlier runs. See also why the scripted recoveries fail and issue 74.