feat(teleop): add PICO WebXR body tracking

[TomCC7]

Contribution path

• Stacked on chore(teleop): rename Quest stack to WebXR #3676

Problem

The renamed WebXR teleop stack has no way to request PICO body-tracking permission, decode the browser's joint poses, or inspect tracking health on hardware.

Solution

• Add a device-neutral BodyTrackingSnapshot output and strict JSON decoder.
• Keep body tracking off by default, with explicit optional and required WebXR session modes.
• Capture XRFrame.body poses in bounded-floor space when available and fall back to local-floor.
• Add a PICO API-test blueprint and live monitor without coupling body data to controller or viewer poses.
• Accept PICO's six-button WebXR controller packets, where the platform-reserved menu button is omitted.

How to Test

Run the hardware-facing API test:

dimos run demo-pico-body-tracking

Automated checks run locally:

CI=1 uv run pytest dimos/teleop/webxr dimos/robot/test_all_blueprints_generation.py dimos/robot/test_all_blueprints.py -q
uv run ruff check dimos/teleop/webxr dimos/robot/all_blueprints.py
uv run ruff format --check dimos/teleop/webxr dimos/robot/all_blueprints.py
uv run --group lint mypy --disable-error-code=misc dimos/teleop/webxr
node --check dimos/teleop/webxr/web/static/teleop.js
node --check dimos/teleop/webxr/web/static/webxr_body.mjs

Result: 150 passed, 3 skipped, 40 deselected.

Physical PICO validation remains to be completed with the command above.

AI assistance

OpenAI Codex with GPT-5 implemented the change, resolved the parent rebase conflicts, and ran the checks. The author should review and understand the final diff before merge.

Checklist

• I have read and approved the CLA.

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
4943	1	4942	75

View the top 1 failed test(s) by shortest run time

dimos.e2e_tests.test_dimsim_path_replaning::test_path_replanning

Stack Traces | 221s run time

lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x72e50addbc50>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x72e50af87920>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x72e50ad6d400>
direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x72e50ad67860>
spawn_wall_on_pose = <function spawn_wall_on_pose.<locals>.spawn at 0x72e50af87100>

    @pytest.mark.self_hosted_large
    def test_path_replanning(
        lcm_spy, start_blueprint, dim_sim, direct_cmd_vel_explorer, spawn_wall_on_pose
    ) -> None:
        start_blueprint(
            "--dimsim-scene=empty",
            "run",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        # robot spawns at (3, 2)
    
        # side wall
        dim_sim.add_wall(2, -2.5, 12, -2.5)
        # other side wall
        dim_sim.add_wall(2, 3.5, 12, 3.5)
        # back wall (behind robot)
        dim_sim.add_wall(2, -2.5, 2, 3.5)
        # forward wall (far end)
        dim_sim.add_wall(12, -2.5, 12, 3.5)
        # dividing wall at x=7 with doors at y=[-1.5,-0.5] and y=[1.5,2.5]
        dim_sim.add_wall(7, -2.5, 7, -1.5)
        dim_sim.add_wall(7, -0.5, 7, 1.5)
        dim_sim.add_wall(7, 2.5, 7, 3.5)
    
        direct_cmd_vel_explorer.linear_speed = 0.8
        direct_cmd_vel_explorer.follow_points([(10, 2), (2.5, 2), (3, 2)])
    
        # When the robot comes within 1.5 m of the left door's centre, drop a wall
        # in the opening so the planner has to bail out and route through the
        # right door at y=-1 instead.
        spawn_wall_on_pose(
            point=(7, 2),
            threshold=1.5,
            wall=(7, 1.5, 7, 2.5),
        )
    
        dim_sim.publish_goal(10.913, 0.588)
    
>       lcm_spy.wait_until_odom_position(10.913, 0.588, threshold=1, timeout=120)

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x72e50ad6d400>
direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x72e50ad67860>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x72e50addbc50>
spawn_wall_on_pose = <function spawn_wall_on_pose.<locals>.spawn at 0x72e50af87100>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x72e50af87920>

dimos/e2e_tests/test_dimsim_path_replaning.py:60: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/lcm_spy.py:167: in wait_until_odom_position
    self.wait_for_message_result(
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x72e50afa0720>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x72e50addbc50>
        threshold  = 1
        timeout    = 120
        x          = 10.913
        y          = 0.588
dimos/e2e_tests/lcm_spy.py:153: in wait_for_message_result
    wait_until(
        event      = <threading.Event at 0x72e51094a930: unset>
        fail_message = 'Failed to get to position x=10.913, y=0.588'
        listener   = <function LcmSpy.wait_for_message_result.<locals>.listener at 0x72e50afa07c0>
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x72e50afa0720>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x72e50addbc50>
        timeout    = 120
        topic      = '/odom#geometry_msgs.PoseStamped'
        type       = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

predicate = <bound method Event.is_set of <threading.Event at 0x72e51094a930: unset>>

    def wait_until(
        predicate: Callable[[], bool],
        *,
        timeout: float,
        interval: float = 0.1,
        message: str | None = None,
    ) -> None:
        """Poll ``predicate`` until it returns truthy or ``timeout`` elapses."""
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            if predicate():
                return
            time.sleep(interval)
>       raise TimeoutError(message or f"Timed out after {timeout}s waiting for condition")
E       TimeoutError: Failed to get to position x=10.913, y=0.588

deadline   = 4258881.50540193
interval   = 0.1
message    = 'Failed to get to position x=10.913, y=0.588'
predicate  = <bound method Event.is_set of <threading.Event at 0x72e51094a930: unset>>
timeout    = 120

.../utils/testing/waiting.py:35: TimeoutError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[greptile-apps]

Greptile Summary

This change adds configurable PICO WebXR body tracking, validates incoming body-joint snapshots, reports tracking health, and registers a dedicated hardware-test blueprint. The body-only WebXR flow was exercised with a bounded-floor reference space and its emitted snapshot was accepted by the server model.

Confidence Score: 5/5

No blocking failure remains.

No accepted blocking findings remain after exercising the body-only WebXR snapshot contract.

T-Rex Logs

What T-Rex did

• • Checked PR-base body capture availability to establish baseline for the body-only flow.
• • Executed the authored focused body-only WebXR contract harness and confirmed it emitted a hips pose with bounded-floor that the server model accepted exactly, with no security issue reproduced.
• • Updated the WebXR JavaScript syntax check to align with the new body-only flow.
• • Noted a focused repository test-suite environment failure during the test run.

_{Ran code and verified through T-Rex}

_{Reviews (5): Last reviewed commit: "test(webxr): isolate body monitor timing" | Re-trigger Greptile}

[TomCC7]

Addressed the Greptile P1 in 82d7bc6. WebXRControllerState now requires the six button slots it indexes while keeping the seventh menu button optional. A regression test sends a real five-button LCM Joy packet through WebXRTeleopModule and verifies the packet is rejected without escaping the callback, stale controller state is cleared, and zero velocity is published. Local verification: 54 WebXR tests passed; Ruff and mypy passed.

[TomCC7]

@greptileai please re-review the latest commit 82d7bc6. The prior P1 about short controller packets has been fixed and regression-tested.