Skip to content

Add metrics to runner - #22283

Open
metascroy wants to merge 3 commits into
add-text-streamfrom
add-metrics
Open

Add metrics to runner#22283
metascroy wants to merge 3 commits into
add-text-streamfrom
add-metrics

Conversation

@metascroy

@metascroy metascroy commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Adds timing instrumentation to the batched runner, which previously reported nothing. metrics.h defines two tiers: GenerationMetrics, published on GenerationHandle beside finish_reason(), covering the submit→first-step→first-token→end timeline, token counts, and an inter-token latency summary; and EngineMetrics, owned by the runner and read after shutdown(), covering step counts and latency, decode/prefill sequences and tokens, admitted-versus-ready decode sequences, and TTFT rollup.

@pytorch-bot

pytorch-bot Bot commented Aug 28, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22283

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job, 1 Unrelated Failure

As of commit 0d4e638 with merge base 0447246 (image):

NEW FAILURE - The following job has failed:

CANCELLED JOB - The following job was cancelled. Please retry:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant