feat(chat): add optional fast-model output router - #1732
Open
sentry-junior[bot] wants to merge 14 commits into
Open
feat(chat): add optional fast-model output router#1732sentry-junior[bot] wants to merge 14 commits into
sentry-junior[bot] wants to merge 14 commits into
Conversation
Route completed tool-free assistant messages through AI_FAST_MODEL before delivery when experimental output-router is enabled. Handles NO_REPLY silence and long-reply compression with a tight structured prompt. Co-Authored-By: David Cramer <david@sentry.io>
Prepare only the visible reply with the fast model. Leave the original agent message in history. Simplify the prompt and API surface. Co-Authored-By: David Cramer <david@sentry.io>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Judge mixed [[NO_REPLY]] with the fast model instead of stripping deterministically. Snapshot the status-only silence failure, keep real answers that mention the marker, and wire cases into the guardian eval suite. Co-Authored-By: David Cramer <david@sentry.io>
Give visible-reply prepare its own config, workflow, path triggers, and label so it runs only when needed. Keep Guardian separate and drop the shared-suite coupling. Co-Authored-By: David Cramer <david@sentry.io>
Stop stripping the silence marker from model replies. Real answers that explain or quote [[NO_REPLY]] must stay visible; only exact marker-only output is silent. Co-Authored-By: David Cramer <david@sentry.io>
Replace isolated text snapshots with full Slack/runtime cases. Script assistant replies from real transcripts (long steering essay, silence tag, protocol explanation) and assert what actually posts after prepare. Co-Authored-By: David Cramer <david@sentry.io>
Pass JUNIOR_PERSONALITY into the prepare-assistant system prompt so shortened visible replies still match the bot voice. Co-Authored-By: David Cramer <david@sentry.io>
dcramer
marked this pull request as ready for review
August 29, 2026 01:01
Drop model-judgment and static prompt string cases. Evals own reply quality; unit tests cover only local fixed prepare rules. Co-Authored-By: David Cramer <david@sentry.io>
Extract createFullRuntimeEvalConfig so behavioral, integration, and output-router suites only declare name/include/env. Drop the one-off output-router setup file in favor of JUNIOR_EVAL_OUTPUT_ROUTER. Co-Authored-By: David Cramer <david@sentry.io>
Mirror guardian: harness calls prepareAssistantReply directly with real assistant text. Drop full Slack/runtime deps from the dedicated suite. Co-Authored-By: David Cramer <david@sentry.io>
Clarify that trailing NO_REPLY after internal work status is silent, keep protocol explanations as replies, and make eval failure text include the prepared output. Co-Authored-By: David Cramer <david@sentry.io>
Simplify prepare-reply wording, separate message body from instructions, and keep the system prompt short and direct per current lab guidance. Co-Authored-By: David Cramer <david@sentry.io>
A final line that is only [[NO_REPLY]] means suppress the message. Keep inline marker mentions on the model path so silence explanations still post. Avoid fixture-shaped prompt rules. Co-Authored-By: David Cramer <david@sentry.io>
Keep hard max as the ceiling. Require real shortening vs the original text instead of exact soft-max length. Co-Authored-By: David Cramer <david@sentry.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optional experimental path that prepares the visible assistant reply with the fast model before delivery.
[[NO_REPLY]]stays silent locallycreateApp({ experimental: { "output-router": true } })Isolated snapshot evals live under
evals/output-router/and run with the Guardian eval suite. They cover the real maintain-PR failure shape where process chatter ends with[[NO_REPLY]]and should not post.Requested by David Cramer.
--
View Junior Session [Sentry]