benchmarking: add substrate-row SLO-knee sweep (gvisor + microvm) - #885
Open
Alex Bulankou (AlexBulankou) wants to merge 1 commit into
Open
Conversation
tests-slo-ladder.yaml is a dedicated, on-demand tests file (NOT the standing hourly tests.yaml) for a substrate-row SLO-knee measurement run. Point the orchestrator at it via --tests. It sweeps the warm-path AteAPIUser at a concurrency ladder (1/5/10/25/50 users) once per sandbox class (gvisor + microvm) -- 10 rungs total. Together the rungs form the throughput/latency sweep that benchmarking/analysis/slo_knee.py reduces to a knee (max sustained RPS whose chosen latency percentile stays under the SLO ceiling). Kept out of the hourly tests.yaml on purpose: each rung triggers a full substrate deploy + teardown, so this is a deliberate on-demand sweep. Analysis convention baked into the header: stats partition by test NAME (runs/<name>/...), and both classes share tag+metric, so the two ladders are analysed as separate name-prefix globs (slo_gvisor_* vs slo_microvm_*). Passing --tests-yaml back at this file feeds slo_knee the per-rung user count (verified: users_map_from_tests_yaml parses all 10 rungs). [a4s1] reference patch for #6595 benchmarking spike. Signed-off-by: Alex Bulankou <alexbu@google.com>
a4-a4s1
Bot
force-pushed
the
a4s1/benchmarking-slo-ladder-tests
branch
from
August 12, 2026 09:16
01e187f to
f602013
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
automation/tests-slo-ladder.yaml: a dedicated on-demand concurrency ladder (5 rungs ×sandboxClass: gvisor|microvm, 10 entries) for SLO-knee measurement — deliberately separate from the standingtests.yamlso the sweep is invoked explicitly, not hourly. The file header documents the two per-class result globs for the SLO-knee post-processor.Config-only; no cluster is touched by merging this.