Scenario Lab

Initial state

You are positioned at the threshold of the dining room, facing a six-seat rectangular table. Late afternoon light comes through half-drawn horizontal blinds, casting alternating bright and shadowed bars across the table surface. An overhead ceiling fan is spinning slowly; its rhythmic ticking is audible. The table is occupied primarily by the aftermath of a child's Advent calendar project: a large cardboard Advent calendar with 24 numbered flap doors lies flat in the center, several doors torn fully off and scattered across the table. Small foil chocolate wrappers — some crinkled flat, some still loosely ball-shaped — are distributed across the tabletop and two have fallen onto the seat of the nearest chair. A half-full juice box with its straw still in it sits at the far left corner of the table, slightly tipped but not yet spilling. A felt-tip marker with its cap off is resting across the spine of the calendar. Three of the numbered cardboard flaps that were torn off have slid partially under a cloth placemats at the far end. At the right edge of your vision you can see the kitchen pass-through; a family member is audibly moving around in the kitchen but is not visible. The lighting is uneven — the shadowed bars from the blinds make some of the small foil balls hard to distinguish from the table's dark wood grain.

Goal state

Table surface is fully cleared of all Advent calendar debris, loose foil wrappers, the felt-tip marker (with cap replaced), and the juice box. All cardboard pieces — including the main calendar sheet and torn flaps from under the placemat — are stacked or consolidated and placed off the table (on a nearby sideboard or chair seat if present, or set aside on the floor against the wall). Foil wrappers are collected together (at minimum consolidated into one hand for disposal; placing in a nearby trash or loose pile off the table both acceptable). Juice box is upright and placed off the table without spilling. Marker cap is replaced before the marker is set aside. Chair that had foil wrappers on its seat is clear enough that a person could sit down without sitting on debris.

Objects involved

Name	Descriptor	Role
Advent calendar sheet	large flat cardboard, roughly 60x40cm, festive printed surface, multiple torn-open numbered flaps	target
torn cardboard flaps	small irregular cardboard pieces, printed with numbers 1-24, partially under placemat	target
foil chocolate wrappers	small, silver-gold foil, some crinkled flat, some loosely spherical, difficult to see in low-contrast zones	target
felt-tip marker	medium black barrel, cap separated, resting across cardboard spine	target
marker cap	small black cylindrical cap, loose on table surface	target
juice box	small rectangular cardboard carton, approximately half-full, red fruit juice, white straw inserted, slightly tipped	target
cloth placemat	rectangular woven fabric, dark red, concealing cardboard flaps underneath	obstacle
dining table	six-seat rectangular table, dark wood grain surface, lit unevenly by horizontal blind shadows	obstacle
ceiling fan	white four-blade fan, spinning slowly, creates rhythmic visual and auditory distraction overhead	distractor
sideboard	low wooden credenza along the wall, clear surface, used as staging area for removed items	tool
small trash bin	short cylindrical bin in corner near sideboard, partially visible	tool

Expected actions

1. approach the dining table from the threshold, navigating to the near side 5s
2. pick up the felt-tip marker cap from wherever it rolled or rests near the marker 5s
3. push or twist the cap back onto the tip of the felt-tip marker to reseal it 4s
4. set the capped marker aside on the sideboard or off the table surface 4s
5. pick up the juice box from the far left corner using a gentle lateral grip, keeping it upright 6s
6. place the juice box upright on the sideboard or floor away from the table edge 4s
7. slide the placemat at the far end to expose the partially occluded cardboard flaps underneath 5s
8. pick up the loose cardboard flaps from under the placemat and from the table surface 8s
9. lift the main Advent calendar cardboard sheet from the table center 5s
10. stack or consolidate all cardboard pieces together and place the stack on the sideboard or against the wall 6s
11. pick up loose foil wrappers from the table surface, gathering them across multiple grabs 12s
12. navigate around to the near chair to pick up the foil wrappers from the chair seat 7s
13. consolidate all foil wrappers and place or drop them into the nearest trash receptacle or off the table in a collected pile 6s
14. visually scan the table surface in the shadowed blind-stripe zones to confirm no foil remnants remain 5s
15. retreat slightly to perform a wide-angle visual check that the full table surface and both surveyed chairs are clear 5s

Narration script

00:00 I am at the entrance to the dining room. The table is covered in Advent calendar debris — cardboard, foil wrappers, a juice box tipping at the far corner. The ceiling fan is ticking overhead and the light from the blinds is striped across the table, making some of the small foil balls hard to see against the dark wood.

00:06 I approach the near side of the table. I can see the felt-tip marker resting across the calendar spine — the cap is off, probably somewhere nearby on the table surface.

00:12 I locate the marker cap a few centimeters to the right of the marker. I pick up the cap with my right gripper.

00:17 I push the cap firmly onto the tip of the marker, feeling it seat. I pick up the capped marker and set it on the sideboard behind me to the left.

00:25 I turn back toward the table and navigate to the far left corner. The juice box is partial, straw still inserted, and leaning maybe ten degrees off vertical. I approach slowly and grip the box from both lateral sides to stabilize it before lifting.

00:33 I lift the juice box upright and carry it carefully to the sideboard, placing it upright next to the marker. No spill.

00:38 I move to the far end of the table. I can see the edge of at least two cardboard flaps peeking out from under the cloth placemat. I slide the placemat laterally toward the wall to expose what is underneath — three torn flaps are revealed, partially overlapping.

00:46 I pick up the exposed flaps one by one, then gather the additional loose flaps from the main table surface. I have a stack of cardboard pieces in my left gripper.

00:54 I lift the main Advent calendar sheet from the center of the table. It is large and slightly floppy — I support it from underneath with my right arm while gripping the edge with my left. I consolidate the torn flaps on top of it.

01:02 I carry the full cardboard stack to the sideboard and set it down flat. The cardboard portion of the table is now clear.

01:08 Now for the foil wrappers. Several are easy to see in the bright stripe zones, but I need to slow down in the shadowed bands — two small crinkled balls are nearly invisible against the dark grain. I sweep across the table surface methodically, picking up wrappers in clusters with my right gripper.

01:20 I navigate around to the near chair on my left. Two foil wrappers are on the seat. I pick them both up.

01:27 I consolidate all foil wrappers and drop them into the small trash bin I can see in the corner near the sideboard.

01:31 I retreat a half-meter and perform a slow visual scan of the full table surface, paying attention to the shadowed strips from the blinds. The table appears clear. Both checked chair seats are clear.

Edge cases

One or two foil wrappers have slipped off the far table edge entirely and are on the floor beneath — the robot's chest-height cameras may not detect them without a deliberate downward-angle approach or navigating to the far side of the table.
The juice box, when lifted, has a slow drip from a small puncture near the base where a child pressed it — requires deciding whether to tip it upright faster or continue slow stabilized carry, leaving a small wet spot on the table that may need a secondary wipe.
The placemat, when slid aside, reveals not three but five cardboard flaps, plus a half-eaten foil-wrapped chocolate that was missed initially — adds an unplanned object and a stickiness risk to the gripper if the foil tears during pickup.