Smoke RescueOpenCV 5 / System and results
API checking

Two robot dogs. One burning house. No map.

Search through smoke.
Inspect every decision.

Smoke Rescue sends two quadruped robots into a burning house they have never seen. Each robot runs the same onboard brain: it explores room by room, finds people with OpenCV 5 colour and thermal detection, and builds a map it shares with its partner over a lossy radio. OpenCV 5 ORB place recognition corrects the shared map when one robot sees a place the other already mapped. Terra, a GPT-5.6 agent, reads each robot's report a few times per mission and suggests the next room to search; the brain checks every suggestion and keeps moving while Terra thinks. When the search is done, both robots walk out along their own trail.

Smoke Rescue architecture. Top: on each robot, sensors feed a C# reflex brain and OpenCV 5, Terra suggests rooms that the brain checks, and both robots build one shared map over a lossy radio. Bottom: on AWS, the judge website on Amplify calls a Lambda mission API, which launches an ECS Fargate task that runs the robot brain, stores frames in S3, tracks runs in DynamoDB and asks GPT-5.6 Terra for room suggestions.
Deployed and running on 27 October 2026. Live missions run the brain in a headless simulator on Fargate. It has no rendered camera images, so the website draws the streamed robot poses, maps and finds in 3D. The three filmed houses are recordings of the same brain in Unity HDRP.

Measured results

What the robots did

Filmed house (Unity)People foundFalse findsBoth robots outMission time
Three-bed ranch5 of 50Yes, at 365 s and 375 s6:16
L-shaped ranch4 of 50Yes, at 369 s and 413 s6:53
One-door house4 of 50Yes, along their own trail, at 454 s and 463 s7:44

On 39 real floor plans the brain had never seen (HouseExpo, headless simulator, 25 October 2026), two robots sharing one map found 99% of people and one robot found 92% (p = 0.002). Map merging measured robot B's start tilt as 2.42 degrees against a true 2.39 degrees, with no duplicated people.

Run a live mission on the judge website

01 / Recorded evidence

Watch an agent test run

Loading the run catalog.

Walkthroughs show recorded robot views and decisions. Use fullscreen to inspect the camera panels. Simulation timestamps in the trace may differ from video playback time. The trace below provides searchable decision text.

02 / Evidence to action

Follow the agent's decisions

Each step groups the triggering event, executed tools and resulting orders. A missing verdict means the log did not record one. Model summaries are claims by the model; check the tool errors and evidence IDs.

Choose a run to load decisions.

03 / Offline evaluation

With and without agents

Choose a run to load metrics.

Values come from brain.compare_runs. Agent counters come from the run summary and can differ from the executed calls in the trace. Search area is a coverage proxy within 2.5 m of the true paths, sampled on a 0.5 m grid. It is not verified visible floor area. A survivor match uses a 1.5 m threshold. Compare duration and scenario before drawing conclusions; a short test does not establish a rescue benefit.