Scenario Lab

Initial state

You are facing the dining room table from the doorway. Late afternoon light comes through a sliding glass door to your left, casting long shadows across the floor. The overhead light is off. A round wooden table with four chairs is centered in the room. Three of the four chairs are pushed in; the far-left chair is pulled out at an angle, its legs partially blocking a path around the table. On the table surface you can see two woven placemats: the near one is clear, the far one has a streak of orange crayon wax ground into its surface — the crayon itself is lying at the edge of the mat, partially rolled off. A child's plastic cup with a yellow lid sits on the near placemat, upright but empty. On the floor to your right, a stainless steel dog bowl has migrated roughly a meter from its usual corner spot and is now sitting slightly under the nearest chair, containing about three inches of water. A medium-sized mixed-breed dog is lying near the sliding glass door, watching you but not moving. Faint sound of a TV from the adjacent room.

Goal state

The dining table surface is clear of crayons and child items. The crayon-marked placemat has been taken to the kitchen counter or a designated drop zone for adult attention (it cannot be fully cleaned by the robot). The dog bowl is returned to the corner it came from and is not tipped. The pulled-out chair is pushed back in so all four chairs are flush with the table. The plastic cup is either placed in the kitchen sink area or confirmed already handled. Floor under and around the table is unobstructed.

Objects involved

Name	Descriptor	Role
dining table	round, medium-sized, dark-stained wood	obstacle
woven placemat with crayon mark	beige woven fabric, orange wax streak across surface	target
orange crayon	short, blunt-tipped, soft wax, partially unwrapped	target
plastic children's cup	clear plastic with yellow snap-on lid, small	target
clean woven placemat	beige woven fabric, no markings	distractor
stainless steel dog bowl	medium, brushed stainless, partially full of water	target
dining chair (pulled out)	wooden, ladder-back, pulled out at angle from table	obstacle
three remaining dining chairs	wooden, ladder-back, pushed flush with table	obstacle
mixed-breed dog	medium-sized, lying near sliding glass door, calm	distractor

Expected actions

1. Approach the dining table along the right side, navigating around the displaced dog bowl 5s
2. Pick up the orange crayon from the edge of the far placemat using right gripper 6s
3. Place the crayon on top of the crayon-marked placemat to consolidate items 3s
4. Slide the crayon-marked placemat with crayon on top toward the near table edge to bring it within reach 4s
5. Pick up the placemat with both grippers, keeping it level so the crayon does not fall off 6s
6. Navigate toward kitchen doorway while holding the placemat flat 8s
7. Set down the placemat with crayon on the kitchen counter or designated drop zone 5s
8. Return to the dining room and approach the plastic cup on the near placemat 7s
9. Pick up the plastic cup with right gripper 4s
10. Navigate to kitchen and place the plastic cup inside the sink 8s
11. Return to dining room and navigate around the table to the displaced dog bowl under the chair 7s
12. Reach under the chair with both grippers and pick up the dog bowl with care to keep it level and minimize water spillage 8s
13. Slowly retreat from under the chair while keeping the dog bowl balanced 5s
14. Navigate to the corner of the room and set down the dog bowl in the original corner spot 7s
15. Return to the pulled-out chair and push it in flush with the table edge 7s
16. Visually confirm table surface is clear, all chairs are pushed in, and dog bowl is in corner 5s

Narration script

00:00 I am in the dining room doorway. The overhead light is off and afternoon sun is cutting across the floor from the left. I can see the table with two placemats, one of them has orange crayon smeared into it. The dog is resting near the sliding door — not in my path.

00:06 I approach along the right side of the table, navigating around a stainless steel dog bowl sitting partially under the nearest chair. The bowl has water in it — I need to handle that carefully later.

00:11 I pick up the orange crayon resting at the edge of the far placemat with my right gripper. It is a short, soft wax crayon — slightly tacky.

00:17 I place the crayon on top of the marked placemat to keep the items together. Then I slide the placemat toward me so I can get both grippers underneath it without overreaching.

00:24 I pick up the placemat with both grippers, keeping it roughly level. The crayon shifts slightly but stays on the mat. I will hold it flat during transit.

00:30 I navigate toward the kitchen doorway, moving slowly to keep the placemat balanced. The crayon is resting in the center of the mat — it is stable.

00:38 I set the placemat down on the kitchen counter with the crayon still on top. This is a drop zone for adult attention — the wax stain is not something I can clean.

00:43 I return to the dining room and approach the near placemat where the child's plastic cup is sitting upright and empty.

00:50 I pick up the plastic cup with my right gripper. The yellow lid is snapped on — the cup feels light and fully empty.

00:54 I navigate to the kitchen and place the cup inside the sink. It lands on its side but that is fine — it is empty.

01:02 I return to the dining room and navigate around to the dog bowl under the near chair. The chair leg is partially in the way — I angle my approach from the side to give my arms clearance.

01:09 I reach under the chair with both grippers and grip the dog bowl from opposite sides. There is water inside — I keep it level. The bowl is stainless and slightly cold.

01:17 I slowly retreat away from under the chair, keeping the bowl balanced. A small amount of water shifts but nothing spills over the rim.

01:22 I navigate to the corner of the room — the original spot for the bowl — and set it down gently. The water settles. The bowl is stable and upright.

01:29 I move to the pulled-out chair and push it in toward the table until it is flush with the other three chairs.

01:36 I do a final visual scan: table surface is clear, both placemats have been handled, all four chairs are pushed in, the dog bowl is back in the corner. The dog has not moved. Task is complete.

Edge cases

The dog gets up and walks toward the robot while it is carrying the full water bowl, requiring a pause and slight re-route to avoid the animal.
The crayon has rolled fully off the placemat and onto the floor under the table, requiring the robot to locate and retrieve it from a partially occluded position before consolidating with the mat.
The chair is pushed far enough out that the dog bowl has drifted to a position where reaching under requires the robot to partially reposition a chair leg, adding an object-moving sub-step.
A second crayon of a different color is found stuck under the edge of the placemat, not visible in the initial view — requiring the operator to notice and include it during pickup.
The plastic cup actually has a small residual amount of liquid inside that is not visible until picked up and tilted, requiring the operator to carry it more carefully to avoid spilling.