Skip to content

feat: render inputs, latency, and token usage in agent test results @W-23524159@ - #481

Open
niharpatel-salesforce wants to merge 7 commits into
salesforcecli:mainfrom
niharpatel-salesforce:feat/w-23524159-prompt-results-rendering
Open

feat: render inputs, latency, and token usage in agent test results @W-23524159@#481
niharpatel-salesforce wants to merge 7 commits into
salesforcecli:mainfrom
niharpatel-salesforce:feat/w-23524159-prompt-results-rendering

Conversation

@niharpatel-salesforce

@niharpatel-salesforce niharpatel-salesforce commented Aug 27, 2026

Copy link
Copy Markdown

What does this PR do?

Renders per-test-case Inputs and Latency/Tokens in the human-readable output of sf agent test results, for AgentforceStudio (NGT) results — the format currently shared by both AGENT and PROMPT subject types.

  • Inputs line (getTestCaseInputs, formatInputsLine in src/handleTestResults.ts): reads the per-test-case inputs: [{name, value}] array, which exists on the wire (confirmed by tracing the SDK's normalizeAgentforceStudioResults, which preserves it via object spread) but isn't declared on the AgentforceStudioTestCaseResult type in @salesforce/agents — read via a locally type-guarded cast, no upstream type change. Falls back to the existing subjectResponse.userInput parsing when inputs is absent, so anything relying on that today is unaffected. Field names are shown exactly as returned by the API (no re-labeling/title-casing — inputs[].name is an API field name, not a display label). Shows the first 3 inputs, (+N more) beyond that.
  • Latency/Tokens line (parseSubjectResponseMetrics, formatMetricsLine): parses performance.latency.duration and tokenUsage.{completion, prompt.total, total} out of the subjectResponse JSON string. Renders only the parts that exist; omits the line entirely when neither is present.
  • Both lines are presence-based, not subject-type-gated — the results payload has no subjectType field, so rendering triggers purely on whether the underlying data exists, for either subject type.
  • CLI-only change — no modifications to @salesforce/agents, no Connect API / results-contract changes. Scoped to src/handleTestResults.ts plus tests and fixtures.
  • JUnit and TAP formats are unchanged — they continue to render only the subject-agnostic scorer pass/fail fields already used for AGENT (result + failure reasoning), with no inputs/latency/tokens; this PR only touches the human-readable format.
  • Adds unit test coverage for humanFormatAgentforceStudio, which had none before this change: 8 new tests covering inputs-present, legacy-userInput-fallback, graceful omission when no data is present, 4+ inputs truncation, latency-only, tokens-only, and the combined case. Full suite: 455/455 passing, no regressions.
  • Verified against a real completed AGENT run (sf agent test results --test-runner agentforce-studio) — User Input renders unchanged, no spurious Inputs/Latency/Tokens lines appear since AGENT's subjectResponse doesn't carry those fields today.

Screenshot of the rendered output to follow in a comment.

What issues does this PR fix or reference?

@W-23524159@

Scopes the CLI-only rendering of PROMPT test-case inputs, latency,
and token usage in `sf agent test results` human format.
Two-task plan: inputs line, then latency/tokens line, both in
humanFormatAgentforceStudio human-readable output only.
Bring PR salesforcecli#481 up to date with upstream main (agents 2.0.6 bump,
AFS nut test hardening, plugin-agent 2.0.5 release).
These design/plan markdown files were an artifact of my own tooling
defaults, not a repo convention (no prior PR has anything like them) -
removing them so the PR only carries the actual code change. The
design/plan content lives in the PR description instead.
inputs[].name is an API field name, not a display label - show it
verbatim instead of title-casing it.
Comment thread src/handleTestResults.ts
label: 'TAP',
get: () => convertAgentforceStudioTestResultsToFormat(results, 'tap'),
strip: false,
},

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hunk is a pure Prettier auto-format fix - no logic change.
This repo's config sets printWidth: 120;
The original single-line json/junit/tap entries were 125–131 chars each (over the limit), while human was 115 chars (under it), which is why only those three got wrapped and human was left alone.
Confirmed the original file already failed prettier --check at this exact spot before this PR.

@niharpatel-salesforce
niharpatel-salesforce marked this pull request as ready for review August 27, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant