Ivan/feat/planning-trajectory - #3352
Draft
leshy wants to merge 210 commits into
Draft
Conversation
Vendor the motion2 referee (worlds, gold oracle, judge, scoring, rust candidate crate) from ivan/feat/body_obstacle @ d7c1b7c88 into dimos/navigation/motion/autoresearch/ as its canonical home, made fully standalone: the package runs on numpy + scipy + pydantic alone, with relative imports throughout, so a plain directory copy works anywhere. Two substitutions close the dependency knots: - types.py replaces dimos.msgs (Vector3/Quaternion/Pose/PoseStamped/ Path/PointCloud2/SolidPrimitive) with minimal classes whose numeric formulas are copied op-for-op from the originals. - geometry.py vendors the pure-geometry slice of motion/obstacle.py (CollisionShape, DistanceField, scored_clearance, station_poses, turn_mask, sweep_samples, near_field_diff, ...) with BodyAwareAvoidanceConfig reparented to a plain-pydantic AvoidanceConfig. Re-vendor deliberately if obstacle.py's scoring math ever changes (the source branch is not merged here). Dropped relative to motion2: planners/core.py and the core-keyed test suites (they need the production motion stack, which is not on this branch); default --planner is now target. sim2d.py is renamed sim.py. The pyo3 module name dimos_motion2_target and python.rs plan() signature are frozen interface and unchanged. Verified against the pinned referee at d7c1b7c88: 56-world battery (--gen 40 --seed 0), per-world scored/truth/veto/flicker/consist columns BIT-IDENTICAL for both --planner target (gold 0.9624, consistency 0.8041, dq 0) and --planner gold (gold 1.0, dq 0, no self-veto). Speed pillar differs only by machine load, as expected.
…ssion test - __main__.py is now the canonical entry (python -m ...autoresearch): it pins OPENBLAS/OMP/MKL/NUMEXPR/VECLIB pools to 1 thread BEFORE numpy loads. avoid_ms is time.process_time(), which sums CPU across all threads, and OpenBLAS's spinning pool was being charged to the candidate whenever a spin window overlapped a timed plan() call — measured on this battery, pinning takes identical-code score spread from 0.334 to 0.0069. - --jobs N: battery fan-out over worker processes, each pinned to its own core, each measuring its own per-process CPU time — so parallel workers cannot contaminate each other's timings (threads in one process would). The parent warms both pickle caches serially, then spawns workers cache-read-only. 56-world battery: ~20 s at --jobs 8 vs ~2.5 min serial; gold/consistency bit-identical to serial. - scenarios.py cache contract: AUTORESEARCH_CACHE_DIR redirects both pickle caches (shared cache dir for lab worktrees), AUTORESEARCH_CACHE_RO suppresses writes, and all cache writes are atomic (tmp + os.replace) so a crash can never leave a torn pickle. - --json: one JSON document with per-world records (score + verdict fields, non-finite floats nulled) plus the summary — machine-readable for eval harnesses. Default output stays byte-compatible. - --build: build + install the rust candidate first (maturin develop --release), so "score the latest planner" is one command. - test_gold.py: for every curated world plus generated seeds 0/28/30, --planner gold must never be vetoed by its own judge and scores ~1.0 against itself — the regression test for the densify_states truncation bug fixed at d7c1b7c88 (gen028 self-veto). - README.md: quick-start for build/score/view/tests, the timing model, and vendoring provenance. Verified: serial battery per-world columns remain bit-identical to the d7c1b7c88 baseline; --jobs 8 matches serial on all deterministic columns; 19 gold tests pass; --view writes a valid sim2d.rrd.
Replace the brute-force seed planner with the autoresearch lab's best candidate: the exp_0008 + exp_0009 + exp_0010 merge with the two-tier yaw publication gate (lab worktree exp_0016, base commit d911a39; the merged planner.rs was uncommitted on the lab branch due to an evo bookkeeping accident, hence the worktree provenance). rustfmt applied; two clippy allows added on test fns; no functional edits. What it adds over the seed, cumulatively earned across the run: lazy demand-driven distance field + per-(yaw,cell) clearance, CSR point buckets, precomputed gait tables, 16-byte A* heap nodes, goal-anchored shortcut smoothing, start-pose-invariant working area + exact grid index arithmetic, a demand-driven backward 2D Dijkstra differential heuristic, ceil-based fine yaw publication, and a two-tier yaw gate that publishes coarse yaw only where measured chord clearance affords the swept-box window. Battery (56 worlds, --gen 40 --seed 0) vs the seed planner it replaces: referee 104.4 -> 106.7/111, gold 0.9624 -> 0.9629, consistency 0.8041 -> 0.9473, speed pillar 0.12 -> 0.89 (at the 20 ms budget), dq 0, gen028's rotation-cylinder veto cleared. gold/consistency match the lab's recorded exp_0016 numbers exactly. Also: crate tests from the lab land as tests/invariants.rs — behavioral black-box properties (bit-determinism, no cross-call memoization, sealed-world refusal, thin walls never hopped, open-world routing). They are developer aids, deliberately NOT wired into any harness gate: the benchmark treats the crate as a black box and gates behavior through the referee. lib.rs's header now states the CPU-time threading rule and the judge-constant coupling of the yaw gate.
…loops
python -m dimos.navigation.motion.autoresearch.export <dest> scaffolds a
complete, self-contained research lab: the referee copied as referee/
(relative imports make the package name-independent), the current rust
planner seeded as candidate/, the bench/ gate harness, generated locks,
warmed caches, a uv venv, and a git repo -- ready for /evo:discover.
gates.json carries the benchmark command, the recommended gates with
epoch-3 thresholds, and the seeded baseline, so a lab agent registers
them mechanically.
The harness is the proven one from the source lab (autoresearch-planner
run_0000), adapted for standalone operation:
- trust chain unchanged in shape: tracked bench/trust.lock -> .evo/
{bench_guard.py, harness.lock, referee.lock} -> every harness byte.
- referee pinning goes content-hash: referee.lock pins every referee/
file by sha256 (+ stray-file ban), the live co_code of judge/
score_world/summarize/se2_path, and the scoring constants by value --
the old lab's git-head pin has no meaning without a dimos checkout.
- check_rules.py keeps the full anti-cheat ban list (fs/env/net/pyo3-
reach/subprocess/cross-call state; threading deliberately permitted
under CPU-time scoring) with frozen-surface hashes generated at
export into bench/frozen.json.
- fitness weights move to bench/fitness.py with the retune procedure
in its docstring (they were retuned twice in the source lab).
- bench/run drops DIMOS_REPO: lab-root walk-up (EVO_LAB_ROOT override),
the lab venv, PYTHONPATH staging of the .so, baked-in BLAS pinning,
and AUTORESEARCH_CACHE_DIR pointing at the shared .evo/cache.
- caches: the gold cache ships from the source package (plain dict
pickle, key-compatible; warmth probed at export), world caches are
REBUILT under the lab import name (they pickle Scenario module paths).
- policy: no cargo-test gates. candidate/tests/invariants.rs ships as
optional crate tests; behavior is gated through the referee.
Templates are byte-copied, lintable files under export/templates/
(excluded from repo ruff/mypy -- lab conventions, not repo code); all
export-varying data lives in generated files.
Self-test: fast tier (9 tests, ~1 s, CI-safe) exports with build/venv/
warm skipped and proves the tree, standalone import, the trust chain,
and four tamper scenarios (referee edit, stray file beside the judge,
harness edit, banned construct in the candidate). Full smoke behind
AUTORESEARCH_FULL_EXPORT=1.
Verified end-to-end on a real export: uv sync, gold cache warm (3 ms
probe), locks green, cargo build, 2-world battery fitness 25.0/26,
./eval curated 108.13/111 dq 0, ext_invariants + parity gates pass --
with no dimos checkout involved anywhere in the lab.
referee.lock pins live co_code hashes and CPython bytecode changes across minor versions, so a lab uv-synced under a different python would fail ext_invariants for what is really an interpreter mismatch. Write .python-version from the exporting interpreter so uv provisions the same minor everywhere (surfaced deploying to a python-3.13 host).
Standalone flat-ground MuJoCo Go2 for the sim-to-real work. Deliberately does
not build on unitree/mujoco_connection or the existing sim stack -- those are
reference only.
model.py menagerie unitree_go2/scene.xml; Unitree<->MuJoCo motor
permutation resolved by name, not by assumed index order
replay.py re-runs the on-board law tau = kp(q_des-q) + kd(dq_des-dq) +
tau_ff against simulated joint state, scores vs recorded lowstate
__main__.py CLI
Timestep matches the recordings exactly (0.002 s = 500 Hz), so it is one
command per sim step with no resampling. Actuators are torque with gear=1,
so ctrl is N.m directly.
Ships two recordings with the exact networks that produced them, as the
ml-trajectory-research LFS archive:
unitree_himloco01 freewalk (HIMLoco) <-> freewalk_mcf.bin
unitree_v11_gait_height01 v11-final <-> v11_final.bin
FINDINGS.md records why joint-level replay does not work on this data: across
every recording, lowcmd/policy_lowcmd carry q_des=0, kd=0, kp=1.0, tau_ff=40.0
constant over all twelve joints -- template defaults the publisher never fills.
lowstate.dq and .tau_est are identically zero and ~25% of lowstate.q rows are
outside physical joint range. Verified four ways (dimos codec, hand CDR parse,
both channels, three files), so the harness is kept but comparison should be
made at body-pose level against vive_pose until the publisher is fixed.
Wall-clock paced replay in mujoco.viewer.launch_passive, --speed to scale. Same control loop as the headless path.
…dependency
policy.py reads the "FREE" v1 blob directly -- speed-banded HIMLoco experts
plus shared normalization -- and runs the HIM forward pass in numpy. Verified
against MNN on all three bands: max |ours - MNN| = 1.1e-4, well under the
2e-3 fp32 floor the exporter validates at.
walk.py drives that policy on the flat menagerie scene, commanded either by a
constant or by a recording's control_log (zero-order held), so a run is
directly comparable to its vive_pose. Sanity: vx=0.5 -> 0.536 achieved,
upright; turn-in-place stays put.
--policy <bin> drive from control_log instead of replaying lowcmd
--view interactive MuJoCo window
Two values the blob does not carry are pinned here as constants with the cfg
keys they came from: CONTROL_DT (cfg "dt") and TORQUE_LIMITS
(cfg "torque_limits").
MNN is added as a dep so the port can be re-checked against the original, but
nothing on the runtime path imports it.
Drop the cd, the $D shorthand and the direnv exec prefix -- direnv is already loaded, so the commands should be copy-pasteable as-is.
… the sim
A translucent mocap box follows the recorded pose while the policy walks, so
sim and hardware can be compared by eye before any error metric exists.
Two Vive conventions were read off the data, not assumed (test_vive.py pins
the reasoning):
quaternion is wxyz -- under that reading the tracker z axis holds
0.997 +/- 0.003 alignment with world z over a run;
xyzw gives 0.725 +/- 0.279, impossible for a walker
frame is z-up -- 0.09 m of z range against 1.0 and 1.6 m in x and y
Where the tracker sits relative to base_link is still unknown. It is mounted
roughly 15 cm above the body, so the default offset is (0, 0, -0.15) with
--tracker-z to adjust; the in-plane component is not modelled. The track is
anchored at t=0, which cancels the unknown room origin and mounting rotation
but not the lever arm -- that shows up as soon as the body turns, which is
exactly what makes a wrong offset visible.
…the sim
The ghost looked mirrored because it was. Two mounting facts, both read off
the data:
tracker is inverted R[2,2] = -0.997 (himloco01), -0.996 (v11) while the
robot is upright throughout -- so a base offset that
should point down in world is +z in tracker frame
yaw is ~94 deg robot forward lands along the tracker's +y, not +x
The yaw is fitted sim-free, two independent ways that agree: circular mean of
travel direction under a pure +vx command gives 93.6 deg (concentration 0.88,
n=2347), and maximizing cos(body velocity, commanded direction) gives 94.0 at
0.840 on a clean unimodal curve.
Fitting it against the simulator does not work, and the failure is worth
recording: the policy rollout diverges from the real robot within a second or
two, so a sim-vs-ghost displacement score on 2 s windows spans only 0.79-1.09 m
against ~1 m of travel, and its argmin lands at 285 deg -- about 180 deg wrong.
The mount is a property of the recording and has to be fitted from it.
Also corrects the ghost box, which was 2x too wide: the trunk collision from
the official URDF is 0.3762 x 0.0935 x 0.114 at origin 0 0 0, so `base` sits at
the trunk's geometric centre, level with the hips. menagerie inherits that body
one-to-one, so MuJoCo qpos[0:3] is exactly the URDF `base` frame -- there is no
mapping to do, and no `base_link` (the root link is called `base`).
…te claim The simulator always begins standing: walk() resets to the menagerie `home` keyframe and overwrites the leg joints with the policy's default_pose. Nothing in a recording can change that, because there are no joint angles in one -- so the first seconds of himloco01, where the tracker climbs from 0.166 m to a steady 0.229-0.245 m as the robot stands up, cannot be compared against it. --start anchors the ghost and the command schedule past that. Corrects an earlier claim in FINDINGS that ~75% of lowstate.q rows were usable. That came from a |q| < 4 rad filter, which is far too loose. Against the real mechanical limits (calf is [-2.72, -0.84], always negative) the recorded calf angles are in range in 0.0% of rows on himloco01 and 0.1% on v11, with mean +0.007 and a +/-14 rad span. lowstate.q is not joint angles. Also fixes a variable shadowed between an index and a rotation matrix in base_track, caught by mypy.
… fixed
Trajectory error cannot be the objective. Perturbing only the initial joint
angles and replaying the same commands diverges by 136 cm at 12 s for a 3 deg
perturbation and 58 cm for a 17 deg one -- non-monotonic, i.e. chaotic. Against
that floor the sim-vs-real windowed error is 1.5x at 0.5-2 s and 0.88x at 10 s,
so a long-horizon trajectory score carries no information about physics.
metrics.py computes what does survive: speed, speed gain, yaw rate gain, body
height mean/std, gait frequency, plus chaos_spread() to measure each
statistic's own noise across perturbed rollouts. Nothing should be fitted
against a statistic whose sim-real difference does not clearly exceed it.
First results over 40 s of himloco01: speed (0.410 sim vs 0.389 real) and speed
gain (0.780 vs 0.759) agree to within the noise -- the policy's translational
response transfers. height_std is the one clear discriminator so far, the
simulated body bobbing 50% more than the real one at 3x the noise floor.
Two estimator bugs, both mine, both recorded in FINDINGS:
sample-window filtering differentiating a 253 Hz recording and a 50 Hz
rollout with the same 25-sample window smooths them
by 0.1 s and 0.5 s respectively; with Vive dt jitter
as large as the interval itself this reported the
robot walking at 3.87 m/s
mean of ratios mean(achieved/commanded) explodes near zero command
and cancels across sign flips, reporting the
simulator turning backwards; now a least-squares
slope through the origin
vive.read_vive_pose now clocks off the payload's t_host: monotonic, worst gap
15 ms against 92 ms for log_time, while the payload's own ts goes backwards at
91 points.
Still not trustworthy and marked as such: yaw_rate_gain (correct on a constant
command, wrong against the fast-alternating recorded schedule -- needs lag
compensation) and gait_hz (locking onto drift at 0.6 Hz instead of the ~2 Hz
gait). height_mean is not comparable at all while the tracker offset is a guess.
It died with "IndexError: index -1 is out of bounds for axis 0 with size 0" from deep inside the summary print, because a start beyond the last command makes the duration negative and the rollout empty. Validate up front and name the actual span, which is the number the user needs.
…imators --start was not reaching the commands. A patch adding the offset to cmd_at() silently failed to apply while the same offset did land on the ghost lookup, so the simulator ran the first seconds of a recording and was scored against a ghost six seconds later. Every --start number reported before this was wrong. The tell: applied commands had vx=0 where the schedule said vx=0.422. Pinned by test_start_offset_reaches_the_command_schedule. gait_hz was locking onto the robot drifting around the room, reporting 0.58 Hz. Now high-passed by subtracting a 1 s moving average, Hann-windowed, and searched from 1.0 Hz. No ground truth exists to calibrate against -- HIMLoco free_walk has no clocked gait, and the only explicit rate in the fleet is 1.5 Hz on an experimental trot-clock policy -- so this is plausibility, not calibration. (50 Hz is the control rate, himloco.rs:185, confirming walk.CONTROL_DT.) Command gains now find the policy-to-body lag by cross-correlation and regress at that lag, reporting it too. The lag is itself the strongest discriminator found so far. With commands aligned the picture is coherent and matches the viewer: the simulated gait runs at 3.30 Hz against 1.75 Hz real, bobs 33% higher, and answers a turn command in 0.07 s against 0.49 s while turning less per unit command. Translation still transfers -- speed 0.441 vs 0.389 and speed_gain 0.887 vs 0.839, both within about twice the chaos noise. Fitting targets, by SNR: yaw_lag 14.0, yaw_rate_gain 6.7, gait_hz 3.4, height_std 2.8.
… sweep gait_hz is window-dependent: the same configuration reads 1.54, 1.52, 3.30 and 1.51 Hz on 15/25/40/45 s windows. The 3.30 was an outlier -- a harmonic winning one FFT -- and it is what the previous commit's "simulated gait is twice as fast" headline rested on. Read across windows sim sits near 1.5 Hz and the real robot wanders 1.1-1.75; they are not clearly different. gait_hz is now marked unusable as a fitting target until estimated by autocorrelation or a median across sub-windows. Stable and discriminating, still: height_std, yaw_rate_gain, yaw_lag, with speed and speed_gain agreeing between sim and hardware. First sweep of leg-joint physics against the real targets shows no single parameter closes the gap -- raising armature pulls gait_hz toward real but pushes height_std away and collapses speed past 0.1; damping costs speed for little gain. The menagerie defaults are the best row tested. That is the honest case for a multi-objective search, and also evidence that whatever looks wrong in the viewer is not simple leg-joint under-damping.
…cs search gait_hz now takes the first peak of the height autocorrelation instead of an FFT peak. It reads 3.33/3.23/3.33/3.33 Hz on 15/25/40/45 s windows against 1.52-1.69 real, where the FFT swung between fundamental and harmonic. That un-retracts the earlier "simulated gait is twice as fast" finding: the claim was right, the estimator was not. evaluate.py scores one configuration -- runs the policy under the recorded commands, summarizes both sides, and divides each statistic by its own noise floor so a term cannot be won by driving something the simulator cannot resolve. search.py drives it with Optuna's CMA-ES sampler, the right choice for a continuous, low-dimensional, noisy objective (TPE would suit categorical or conditional spaces, which this is not). Measuring the noise floor once and reusing it takes a trial from 7.3 s to 1.6 s, so 200 trials is about five minutes. It also keeps losses comparable: recomputed per trial, a trial could win by getting noisier. 30 trials say joint friction is the missing physics. frictionloss 0.2 -> 1.23 (6x) with armature and damping essentially unmoved takes gait_hz from 3.333 to 1.562 against 1.695 real (SNR 5.2 -> 0.4) and height_std from 0.033 to 0.026 against 0.023 (SNR 4.9 -> 1.5). Loss 4.97 -> 4.27. Consistent with real gearboxes; menagerie's 0.2 N*m models a much freer joint than the robot has. yaw_lag does not move and cannot: 0.04-0.06 s in sim against 0.46 s on hardware is command transport and filtering, not a leg-joint property. It is the largest remaining loss term and needs an explicit command delay, otherwise the next search will drag every other parameter around trying to compensate for it.
Ivan's alternative -- that the real robot is simply heavier and slower to swing round -- tests cleanly and comes out negative. Tripling the trunk's rotational inertia moves sim yaw_lag not at all (0.06 -> 0.06 against 0.46 real), while a 0.5 s command delay takes it to 0.55. The policy is a closed loop at 50 Hz: it feels a heavier body on the first tick and compensates, but cannot compensate for a command it has not received. The measured cross-correlation is a sharp single peak (0.88 at 0.50 s, against 0.66 at zero lag) -- the signature of a shift rather than a filter. Inertia is not useless, x3 improves total loss 4.97 -> 4.43; it just does not explain this. walk() gains command_delay, and the search space gains command_delay plus trunk mass/inertia scaling so the two hypotheses compete on the data. 80 trials: loss 4.97 -> 2.39, with yaw_lag SNR 10.0 -> 1.0, height_std 4.9 -> 1.0, yaw_rate_gain 4.3 -> 1.7. The search settled on command_delay 0.451 s with no knowledge of the cross-correlation that independently measured 0.46-0.50 s, and left trunk_inertia_scale at 1.039 -- unchanged, rejecting the inertia hypothesis on its own. Caveats recorded in FINDINGS: frictionloss fell from 1.23 to 0.307 once delay was available, so the earlier friction figure was partly standing in for the missing delay. And gait_hz is now the worst term (SNR 4.9) where the physics-only search had driven it to 0.4 by leaning on friction, with speed_lag regressing 0.7 -> 3.0. The two optima disagree, so the scalar loss is trading gait accuracy against the rest; next step is weighting or a multi-objective study with a Pareto front.
…e groups NSGA-II over gait / translation / rotation, each the RMS of its statistics' SNR. Seven separate objectives would leave nearly every trial non-dominated; these three are the ones that actually trade. 150 trials, 24 non-dominated. corr(gait, rotation) = -0.42 across the front: they genuinely pull against each other, which is what the scalar loss was averaging away. The physics-only search sat at the gait end (gait 0.4, rotation 10) and the delay search near the rotation end (rotation ~1, gait 4.9) -- both were real points on this curve reported as if they were the answer. Most of the space is smeared across its full range, as expected when the optimum is a curve rather than a point. Two parameters stand out: armature wants ~7x the menagerie default (median 0.071 against 0.01, where the scalar search had said 0.0158), and command_delay's median of 0.369 s spans the 0.46 s measured independently by cross-correlation. Best balanced point, minimizing the worst objective rather than the sum: gait 2.41, translation 2.05, rotation 2.71 -- every term under 2.8 noise floors against a baseline where yaw_lag alone was 10. The read on sim-to-real: no point in this six-parameter space matches gait and rotation together, which points at a missing mechanism (actuator bandwidth, contact behaviour) rather than a mis-tuned constant. The front is also the test for whether adding one helps -- a better model should collapse the curve toward the origin rather than slide along it. Two tests earned their place while writing this: the objective groups must partition the comparable statistics, or a statistic goes silently unoptimized; and log scaling is only allowed where a range spans a decade, which caught trunk_mass_scale set to log over 0.6-2.0.
…e lag A MuJoCo motor delivers the requested torque on the same step; a real BLDC through a gearbox reaches it over a few milliseconds of current-loop bandwidth. actuator_step() applies that lag, tau=0 reproducing the ideal actuator exactly. The Pareto front said no scalar on an existing term could match gait and rotation together, which pointed at a missing mechanism rather than a mis-tuned constant. This is the first candidate. A quick probe is promising: 5 ms takes gait_hz from 3.33 to 1.52 against 1.70 real and height_std from 0.033 to 0.029 against 0.023, loss 4.97 -> 4.53. Past ~30 ms it destabilises the gait entirely, which is the expected shape for a bandwidth limit. Added to the search space as actuator_tau over 0-50 ms. Whether it is a real improvement is a question for the front: a better model should collapse the curve toward the origin rather than slide along it.
Same 150 trials, same objectives, only actuator_tau added: min distance to origin 3.82 -> 2.87 best sum 6.01 -> 4.49 best worst-objective 2.71 -> 2.20 best gait 1.05 -> 0.67 best translation 0.44 -> 0.22 best rotation 0.81 -> 0.94 The curve moved toward the origin rather than sliding along it -- every aggregate improved together and two of three objectives got a better attainable minimum. That was the agreed test for a real model improvement against a new way to trade one error for another. The strongest evidence is what the search did not choose. actuator_tau is searched over 0-50 ms, so an ideal actuator is available for free, and not one of the twenty Pareto-optimal trials sits at zero: min 0.0094, median 0.0219, max 0.0453 s. A spurious knob would leave some optimal points at the identity value. Ten to forty-five milliseconds is also the right order for a BLDC current loop through a gearbox. Best balanced point is now gait 0.67, translation 2.20, rotation 2.06 against 2.41 / 2.05 / 2.71 before -- gait inside one noise floor, nothing above 2.2, where the untouched baseline had yaw_lag alone at 10. Rotation remains the worst term and did not move. Delay and actuator lag both model when torque arrives; nothing yet models what the foot does once it lands. The menagerie feet use condim=1, i.e. frictionless point contacts that generate no tangential force at all, which for a turning quadruped is the obvious next suspect.
FINDINGS had grown to 767 lines across sixteen appended sections, several of which documented intermediate states that later work superseded -- a gait_hz retraction that was itself retracted, a friction figure that changed once command delay existed. Readable as a diary, not as a place to find out where the project stands. Rewritten to 200 lines organised by what a reader actually needs: current status and best configuration up front, then what the recordings contain, what is calibrated, why the judge is distributional, and the two mechanisms found. The bulk of the new document is proposals. Foot contact is the standing hypothesis for the remaining rotation error -- the menagerie feet use condim=1, frictionless point contacts that generate no tangential force, which is a strange way to model a robot that turns by shearing its feet against the ground. Then pinning the tracker translation (a ruler beats a weakly conditioned fit), validating on the held-out v11 recording, and what to capture next time. Superseded narrative is dropped, but the estimator mistakes are kept as a short "traps" list. Every one of them produced a plausible, finite, wrong number, and they are the most reusable thing here. README now points at it rather than duplicating.
Three changes, each killing a wrong number the judge was fitting: - Shared epoch: vive t_host and control_log log_time are the same clock, but each stream was zeroed at its own first message, pairing every real pose with a command 0.31 s in its future (4.4 s on v11). The fitted "command delay" of 0.317 s was this artifact to within 4 ms; genuine epoch-corrected turn latency is ~0.17 s. - Sensor-space height: inverting the guessed tracker offset put 11.4 mm of lever-arm swing into the "ground truth" z (the tracker itself only bobs 5.6 mm). Real keeps raw tracker z; sim mounts a virtual tracker with the same guess, so the guess cancels. Adds offset-immune pitch_std/roll_std statistics; height_std is detrended. - Hardware command slew: go2web ramps operator commands per-axis before the policy sees them (VEL_DV 0.05/0.04/0.10 per 20 ms tick). Known constants, not a fitted knob; walk() now applies the same ramp. This alone puts sim yaw_lag at 0.170 vs real 0.170 with default physics -- the axis-dependent lag no uniform delay could fit. Also: foot_friction / foot_friction_torsional search dimensions (feet are condim=6 priority=1, so friction values are the open question, not contact dimensionality), command_delay range shrunk to genuine-transport scale, pitch/roll join the gait objective group.
…oco01 The 300-trial NSGA-II front collapsed from 36 points to effectively one: gait 0.23 / translation 0.85 / rotation 0.16 in default-floor noise units, every statistic sub-noise, and the fitted parameters physical (armature at spec, trunk +33% mass for the tracker payload, 23 ms transport delay). Rewrites FINDINGS around the matched state: three mechanisms, sensor-space judge, and next steps led by held-out v11 validation and a joint-data capture.
--view/--ghost previously always ran default menagerie physics -- the configuration measured to oscillate ~2.5x harder than the real robot -- because --physics only wired into --eval. The Pareto winner is now a named preset (FITTED_* in evaluate.py); --fitted applies it in both view and eval, and --physics/--command-delay/--actuator-tau pass through to the viewer for A/B against it. Also verified two execution-fidelity details against the go2web executor: the action path matches tick() exactly (clip, last_action = clipped raw, target = act*act_scale + act_mean, blob kp/kd), and MuJoCo free-joint qvel[3:6] is body-frame, matching the IMU gyro the policy was trained on.
…knobs Track now records joint angles and foot heights (clearance()) every control tick, and _physics grows trunk_com_x / leg_mass_scale, so leg behavior is measurable instead of eyeball-only. Findings from the front-leg investigation (sim visibly high-steps its front legs while every base statistic matches): the recording DOES carry joint-space commands after all -- policy/lowcmd (2466 msgs, 44 Hz, kp=40/kd=1) is the executor's commanded targets; only rt/lowcmd is zeroed. FK on those targets: real front clearance p95 6.5 cm vs sim 9.6 cm, rear identical -- and sim commanded == sim executed, so the plant is exonerated: the policy itself asks for the extra lift under sim observations. Falsified as causes: contact solimp, torque limits, com shift, leg mass, joint friction/damping, dq smoothing (each null or worse). Next: leg-space statistics in the judge scored against policy/lowcmd, and let the search fit the plant that makes the policy choose the real gait.
read_policy_lowcmd() decodes the executor's commanded joint targets (the recording's only joint-space ground truth) onto the shared epoch; front_lift / rear_lift are p95 FK foot clearance of commanded targets, compared command-to-command so the unrecorded real leg state and the instability of open-loop replay never enter. Track records sim targets; Report and the noise floor carry the leg statistics; the search gets a fourth 'legs' objective plus trunk_com_x / leg_mass_scale dimensions, and the damping floor drops to 0.05 (the old front leaned on 0.2). At default physics the new statistic reads front_lift 0.217 sim vs 0.065 real (SNR 25) -- the visible front-leg prancing, now measurable.
… 0.065 The 400-trial four-objective search's min-max point holds gait 1.06 / translation 0.98 / rotation 0.45 / legs 0.81 at once: commanded front foot lift 0.060 vs 0.065 real (the base-only fit commanded 0.217), rear 0.039 vs 0.035, base statistics intact. Parameters became physically coherent -- com +4.4 cm forward where the lidar and head sit, torsional friction 0.030 vs the shipped 0.02, trunk +20% payload. FITTED_* and FINDINGS updated; the base-only episode is recorded as the "a judge only constrains what it can see" trap.
…stats Three gaps Ivan's viewing session exposed: - _physics never patched load_with_ghost, so --view --ghost ran stock physics under --fitted -- the "almost falling" was the unfitted model. Both loaders are patched now, with a regression test. - The judge scored only t=6-26 of a 48 s recording; unscored time is unconstrained time. seconds=None (now the default everywhere) scores the entire run. - tilt_p99 joins the gait group: the stability tail, so a config that occasionally almost falls pays even when its oscillation statistics look right. Real reference: p99 8.6 deg, max 16.7. - Leg statistics widen from two clearance scalars to thigh/calf command spans per front/rear pair plus the command-space gait frequency -- using the joint-angle streams instead of collapsing them.
…e, basin limit The fitted config is visually confirmed on the robot's recording; the follow-up full-run search found no dominating point, and re-scoring the same config across processes showed the 4-seed noise floor drifting (BLAS-order chaos), so single-recording fitting has hit its resolution limit. The discriminator from here is the held-out v11 recording.
The per-heading envelope made feasibility motion-conditioned but left the seed witness on the all-gait union, so the planner threaded a gap only a drift row clears, the robot followed the plan in, and the next replan from mid-gap refused forever. Standing occupies the intersection of the envelope's rows (0.781 x 0.416 at -0.039 for the go2, mirrored in both drift signs), which is nested in every shape an edge can be cleared by -- so a pose the route justified always passes the witness. Turn-in-place edges and the smoothing floor keep the union: those are real motion sweeping the full shape. Pinned by test_replan_invariance: every 5th published pose of every route on 25 worlds, replayed as a fresh query. On the union gold refused on 5 worlds / 11 poses and the candidate on 2 / 7; on the intersection, none.
Same shape, same formula as embodiment.py: Emb::stand_box intersects the envelope rows in both drift signs, Fps interns it, and the seed reads it -- both at the true start pose and in fit_bin's repair, which was asking the same question on a stricter shape. Union first there as everywhere: the standing box is nested in it, so open space pays nothing and an embodiment with no measured rows keeps id 0 and the old reading exactly.
…ying The go2 bridge stamps clouds and odometry off the same sweep clock, so the right pose matches a cloud EXACTLY -- every accepted cloud in 20260807-190044.mcap sits on an odometry stamp to the nanosecond. What it does not do is deliver them in that order. Both inputs land in one tokio::select! loop, which polls its ready arms in random order, so a cloud reaches the handler ahead of the odometry queued beside it whatever the load. That alone would be survivable if the previous sweep's pose were an acceptable fallback. It is not, by 0.8 ms: the sweep measures 100.801 ms, not the nominal 100, and POSE_MATCH_TOLERANCE_S is 0.1. So the loser of that coin flip found nothing inside the gate and the cloud was discarded outright -- 13 of 213 clouds, 6.1%, on an idle laptop with odometry flowing at a clean 30 Hz and no holes in the stamp grid. So hold it. An unmatched cloud waits for the pose of its own sweep rather than being matched loosely against the wrong one, which keeps registration exact where widening the tolerance would have bought the drops back as motion smear. At most one sweep is ever held -- a newer cloud retires the older -- and pose_still_due drops it once odometry has run past its window unmatched, so a pose that genuinely never comes cannot wedge the queue. deployment_plan.md blamed this on load and the 3% it cost on the robot. Load was never the whole story, and saying so hid how little margin there was. Corrected there too (markdown table alignment rode along).
… robot runs Two things the baked host needed and did not have. reset_carrot_m crossed into the rust planner config with the commitment work, and native_config forbids serde defaults -- python owns them and sends them -- so a field absent from the stdin blob is not defaulted, it is fatal. With Restart=on-failure the unit would have crashlooped rather than said so once. Caught by cross-checking all four baked configs against their Config structs and against `bake --emit-config`; the other three matched. max_speed said 0.95 here while the robot has been running 0.6. The 0.95 is sim-calibrated against freewalk_mcf and, per the blueprint's own warning, not re-measured against the deployed gait -- so the 0.6 is the deliberate number and this file was the stale one. Deploying it as written would have raised the ceiling 58% on speeds nothing has verified. The authoritative file now says what actually runs.
…dy filled The commitment path ran a full lattice search to carry the incumbent to a moved goal BEFORE asking whether the trimmed route was still walkable at all -- 199 ms on the door2 tick where the map closed the corridor, spent on a route the very next test threw away. The two are independent tests joined by an AND, so ask the cheap one first. And the extension no longer builds its own clearance table. `Clear` is a pure memo of (world, footprints, margin, lattice), so the table the fresh search just filled answers the extension's questions too: its own footprint scans fall ~94%. door2.zenoh, incumbent chained every tick: 26.5 -> 16.8 ms/tick, worst tick 198.6 -> 59.6. Fresh-search-only unchanged at 12.4. Bit-stable: 63/63 plans identical across the two builds, crate tests 19/19, referee 118/118, battery 114.80 with 40/40 goals and 0 rerolls.
Connect endpoints only ever came from --robot-ip, and only when the stack
was zenoh-transported -- both right for a process that owns its robot, both
wrong for one that just wants to watch someone else's. `dimos spy` is the
case: it observes every transport at once, so it has no DIMOS_TRANSPORT to
set, and pointing it at a Go2 meant setting one anyway.
--zenoh-connect names endpoints outright and is honoured whatever the
transport -- naming an endpoint is itself the intent to dial it. It feeds
_default_connect_endpoints(), so the native children inherit it through the
DIMOS_ZENOH_CONNECT the launcher already derives from there.
dimos --zenoh-connect go2 --zenoh-mode client spy --transport zenoh
Client mode stays the load-bearing half: the robot's helper runs a router,
and a router forwards to clients, never to peers.
…ctrl # Conflicts: # dimos/msgs/test_msg_construction.py
mcap is an extra, not a default dep -- importorskip the two writer imports so the vive tests skip instead of erroring where it is absent. The pure math in that file still runs. The banner comments go: the rule wants files, not subfiles. Prose kept, decoration dropped. object.__new__ stays in the adapter tests, now whitelisted with the reason: the real constructors stand up an LCM RPC transport per instance (a run_forever + _lcm_loop pair) that pure geometry tests cannot tear down -- 21s and a leaked-thread failure against 0.5s. Lab code is not a runnable module. The blueprint scan matched ActorCritic(nn.Module) because it compares the last dotted segment of a base class, so a torch net was about to be registered as a dimos module; excluding research/ keeps the registry to production modules.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.