Skip to content

Benchmark replay - #3347

Open
Dreamsorcerer wants to merge 29 commits into
mainfrom
sam/benchmark-replay
Open

Benchmark replay#3347
Dreamsorcerer wants to merge 29 commits into
mainfrom
sam/benchmark-replay

Conversation

@Dreamsorcerer

Copy link
Copy Markdown
Collaborator

No description provided.

@mintlify

mintlify Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
dimensional 🟢 Ready View Preview Aug 4, 2026, 12:53 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds configurable replay speed and windowing and introduces a CodSpeed benchmark for draining the Go2 replay pipeline.

  • Forwards replay speed, seek, and duration from global configuration into ReplayConnection.
  • Adds a bounded replay benchmark with stream-delivery validity floors and lifecycle cleanup.
  • Adds a self-hosted CodSpeed workflow and the corresponding test dependency.
  • Documents the new replay CLI options.

Confidence Score: 5/5

The PR appears safe to merge based on the follow-up evidence available.

No blocking failure remains within the eligible follow-up review scope.

Important Files Changed

Filename Overview
.github/workflows/codspeed.yml Adds the self-hosted, containerized CodSpeed workflow with replay-data preparation and wall-time benchmark execution.
dimos/core/global_config.py Adds process-wide replay speed, seek, and duration settings exposed through the dynamic CLI configuration.
dimos/robot/unitree/go2/connection.py Forwards replay timing and window settings into the memory replay implementation.
dimos/robot/unitree/go2/test_replay_benchmark.py Adds a bounded end-to-end benchmark that validates delivery floors and waits for stream quiescence.
pyproject.toml Adds the pinned pytest-codspeed benchmark dependency.
uv.lock Locks pytest-codspeed 5.0.3 and its resolved artifacts.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  CLI["Replay CLI options"] --> Config["GlobalConfig"]
  Config --> Connection["ReplayConnection"]
  Connection --> Store["SQLite replay store"]
  Store --> Streams["Go2 replay streams"]
  Streams --> Coordinator["Unitree Go2 blueprint"]
  Coordinator --> Watchers["Benchmark subscribers"]
  Watchers --> Gate["Count floors + quiescence"]
  Gate --> CodSpeed["Wall-time result"]
Loading

Reviews (9): Last reviewed commit: "Fix" | Re-trigger Greptile

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
3787 1 3786 175
View the top 1 failed test(s) by shortest run time
dimos.codebase_checks.test_no_underscore_assign::test_no_underscore_assignment
Stack Traces | 7.09s run time
def test_no_underscore_assignment():
        """Fail if any file assigns to a bare `_`."""
        dimos_dir = DIMOS_PROJECT_ROOT / "dimos"
        hits = find_underscore_assignments()
        if hits:
            listing = "\n".join(f"  - {p.relative_to(dimos_dir)}:{lineno}" for p, lineno in hits)
>           raise AssertionError(
                f"Found assignment(s) to `_` in dimos/:\n{listing}\n\n"
                "Assigning to `_` is not allowed: it hides an unused variable instead "
                "of removing it. Delete the variable. If you only need the "
                "expression's side effect, evaluate it directly with a call "
                "(`obj.method()`, `getattr(obj, 'attr')`) or log it; a bare attribute "
                "access needs `# noqa: B018`. Tuple unpacking (`a, _ = f()`) is fine "
                "and not flagged by this rule."
            )
E           AssertionError: Found assignment(s) to `_` in dimos/:
E             - .../unitree/go2/test_replay_pipeline_benchmark.py:127
E             - .../unitree/go2/test_replay_pipeline_benchmark.py:129
E           
E           Assigning to `_` is not allowed: it hides an unused variable instead of removing it. Delete the variable. If you only need the expression's side effect, evaluate it directly with a call (`obj.method()`, `getattr(obj, 'attr')`) or log it; a bare attribute access needs `# noqa: B018`. Tuple unpacking (`a, _ = f()`) is fine and not flagged by this rule.

dimos_dir  = PosixPath('.../dimos/dimos/dimos')
hits       = [(PosixPath('.../dimos/dimos/dimos/.../unitree/go2/test_replay_pipeline_benchmark.py'), 127), (PosixPath('.../dimos/dimos/dimos/.../unitree/go2/test_replay_pipeline_benchmark.py'), 129)]
listing    = '  - .../unitree/go2/test_replay_pipeline_benchmark.py:127\n  - .../unitree/go2/test_replay_pipeline_benchmark.py:129'

dimos/codebase_checks/test_no_underscore_assign.py:55: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@codspeed-hq

codspeed-hq Bot commented Aug 10, 2026

Copy link
Copy Markdown

Congrats! CodSpeed is installed 🎉

🆕 1 new benchmark was detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmark


Open in CodSpeed

Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml Outdated
Comment thread .github/workflows/codspeed.yml
Comment thread dimos/robot/unitree/go2/test_replay_benchmark.py Outdated
Comment thread dimos/robot/unitree/go2/test_replay_benchmark.py Outdated
Comment thread dimos/robot/unitree/go2/test_replay_benchmark.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant