From d3d0e08ecbc4deb9d073d629f1ce60bbc9675163 Mon Sep 17 00:00:00 2001 From: D-K-P <8297864+D-K-P@users.noreply.github.com> Date: Tue, 11 Aug 2026 15:27:29 +0100 Subject: [PATCH 1/3] docs(ai-chat): correct version-pinning claim in agents overview The overview said an in-progress chat resumes on the new version after a redeploy, which contradicts the version-upgrades and backend pages. Chat agent runs are pinned to the version they started on; moving onto new code is an explicit version upgrade. --- docs/ai-chat/overview.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ai-chat/overview.mdx b/docs/ai-chat/overview.mdx index 3c37661eb4c..ebfd7807cd9 100644 --- a/docs/ai-chat/overview.mdx +++ b/docs/ai-chat/overview.mdx @@ -47,7 +47,7 @@ See [Quick Start](/ai-chat/quick-start) for the matching server actions and a ru ## Why use AI Agents on Trigger.dev -- **Resume across refreshes, deploys, and crashes.** A chat in progress when you redeploy keeps streaming on the new version. Mid-stream refreshes pick up where they left off. +- **Resume across refreshes, deploys, and crashes.** A chat in progress keeps streaming through a redeploy, pinned to the version it started on. Move it onto new code when you choose with a [version upgrade](/ai-chat/patterns/version-upgrades). Mid-stream refreshes pick up where they left off. - **Native AI SDK support.** Text, tool calls, reasoning, and custom `data-*` parts all flow through `useChat` over a custom `ChatTransport`. No custom protocol to maintain. - **Multi-turn for free.** Each turn is a step inside the same durable task; conversation history accumulates server-side, so clients only ship the new message. - **Fast cold starts.** Opt-in [Head Start](/ai-chat/fast-starts#head-start) runs the first `streamText` step in your warm Next.js / Hono / SvelteKit server while the agent boots in parallel — cuts time-to-first-chunk roughly in half. From de558fe19ab8cffca710204178fc256459361ee9 Mon Sep 17 00:00:00 2001 From: D-K-P <8297864+D-K-P@users.noreply.github.com> Date: Tue, 11 Aug 2026 15:27:29 +0100 Subject: [PATCH 2/3] docs(ai): add LLM observability page Document capturing Vercel AI SDK calls as spans in the run trace: turning it on per call, what each span inspector tab shows, linking a call to its prompt version, and querying usage across runs with TRQL. --- docs/ai/observability.mdx | 165 ++++++++++++++++++++++++++++++++++++++ docs/docs.json | 1 + 2 files changed, 166 insertions(+) create mode 100644 docs/ai/observability.mdx diff --git a/docs/ai/observability.mdx b/docs/ai/observability.mdx new file mode 100644 index 00000000000..cb847f79018 --- /dev/null +++ b/docs/ai/observability.mdx @@ -0,0 +1,165 @@ +--- +title: "LLM observability" +sidebarTitle: "LLM observability" +description: "Capture Vercel AI SDK calls in a task as spans in the run trace, with model, token usage, cost, and latency. Opt in per call, link calls to prompt versions, and query usage across runs." +--- + +**LLM observability turns a Vercel AI SDK call inside a task into its own span in the run trace, next to your logs and other spans.** Each span carries the model, provider, input, output, and total token counts, cost, and latency, so you can see what each generation did and what it cost without leaving the run. + +Everything shows up inline in the run trace you already use to debug runs. There is no separate product and no dashboard to set up. + + + Observability is opt-in per call and only covers [Vercel AI SDK](https://ai-sdk.dev) functions (`generateText`, `streamText`, `generateObject`). Calls you make with a raw `fetch`, a provider's own SDK, or any other HTTP client are not captured automatically. + + +## Turn it on + +Set `experimental_telemetry: { isEnabled: true }` on the AI SDK call. There is nothing to install for AI SDK 6, and nothing to configure on the Trigger.dev side. + +```ts /trigger/summarize.ts +import { task } from "@trigger.dev/sdk"; +import { generateText } from "ai"; +import { openai } from "@ai-sdk/openai"; + +export const summarize = task({ + id: "summarize", + run: async (payload: { text: string }) => { + const result = await generateText({ + model: openai("gpt-4o"), + prompt: `Summarize the following text:\n\n${payload.text}`, + experimental_telemetry: { isEnabled: true }, + }); + + return { summary: result.text }; + }, +}); +``` + +Trigger the task and open the run. The `generateText` call appears as a span in the trace. `streamText` and `generateObject` work the same way: add the same `experimental_telemetry` flag to each call you want captured. + + + On AI SDK 7, span emission moved out of the `ai` core into the `@ai-sdk/otel` adapter. Install `@ai-sdk/otel` and Trigger.dev registers it for you at run start. AI SDK 6 emits spans from `ai` directly, so no extra package is needed. + + +## What each span shows + +Open an AI generation span in the run trace to get a dedicated inspector with three tabs: + +- **Overview**: model, provider, token usage, cost, and a preview of the input and output. +- **Messages**: the full message thread, including the system prompt and any tool results. +- **Tools**: the tool definitions passed to the model, plus every tool call the model made with its arguments. + +A fourth **Prompt** tab appears when the call is linked to an [AI Prompt](/ai/prompts) (see below). + +## Link a call to its prompt + +If you manage prompts with [AI Prompts](/ai/prompts), resolve the prompt and spread `toAISDKTelemetry()` into the call. This sets `experimental_telemetry` for you and links the span back to the exact prompt version that produced it. + +```ts /trigger/support.ts +import { task, prompts } from "@trigger.dev/sdk"; +import { generateText } from "ai"; +import { openai } from "@ai-sdk/openai"; +import type { supportPrompt } from "./prompts"; + +export const handleSupport = task({ + id: "handle-support", + run: async (payload: { name: string; plan: string; issue: string }) => { + const resolved = await prompts.resolve("customer-support", { + customerName: payload.name, + plan: payload.plan, + issue: payload.issue, + }); + + const result = await generateText({ + model: openai(resolved.model ?? "gpt-4o"), + system: resolved.text, + prompt: payload.issue, + ...resolved.toAISDKTelemetry(), + }); + + return { response: result.text }; + }, +}); +``` + +The span's **Prompt** tab now shows the linked template, its version, and the input variables the prompt was resolved with. + +Pass custom attributes to `toAISDKTelemetry()` to tag the span with your own metadata: + +```ts +const result = await generateText({ + model: openai(resolved.model ?? "gpt-4o"), + system: resolved.text, + prompt: payload.issue, + ...resolved.toAISDKTelemetry({ + "task.type": "summarization", + "customer.tier": "enterprise", + }), +}); +``` + +Custom attributes are stored on the span's `metadata`, so you can filter or group by them in TRQL, for example `metadata['task.type']`. + + + When you build an agent with `chat.agent()`, `chat.toStreamTextOptions()` already sets `experimental_telemetry` for you, so generations inside a chat are captured without adding the flag by hand. See [Prompts](/ai/prompts#using-with-chatagent). + + +## Query usage across runs + +Every captured generation is also written to the `llm_metrics` table, which you can query with [TRQL](/observability/query). This lets you aggregate token usage, cost, and latency across many runs rather than inspecting one span at a time. + +Cost and token usage by model: + +```sql +SELECT + response_model, + gen_ai_system AS provider, + count() AS calls, + sum(total_tokens) AS tokens, + round(sum(total_cost), 4) AS cost_usd +FROM llm_metrics +GROUP BY response_model, gen_ai_system +ORDER BY cost_usd DESC +LIMIT 20 +``` + +Spend per task: + +```sql +SELECT + task_identifier, + sum(input_tokens) AS input_tokens, + sum(output_tokens) AS output_tokens, + round(sum(total_cost), 4) AS cost_usd +FROM llm_metrics +GROUP BY task_identifier +ORDER BY cost_usd DESC +LIMIT 20 +``` + +Cost by prompt version, when calls are linked to an [AI Prompt](/ai/prompts): + +```sql +SELECT + prompt_slug, + prompt_version, + count() AS calls, + round(sum(total_cost), 4) AS cost_usd +FROM llm_metrics +WHERE prompt_slug != '' +GROUP BY prompt_slug, prompt_version +ORDER BY prompt_slug, prompt_version +``` + +Set the time window with the query's [period filter](/observability/query#time-ranges) rather than in the SQL itself. Run these from the [Query dashboard](/observability/query#using-the-query-dashboard), the SDK with `query.execute()`, or the REST API. `llm_metrics` also exposes `ms_to_first_chunk` and `tokens_per_second` for latency and throughput, plus `finish_reason`, `request_model`, `cached_read_tokens`, `reasoning_tokens`, and per-direction `input_cost` / `output_cost` for finer breakdowns. + +## Next steps + + + + Version prompts as code and link generations to the exact prompt version that produced them. + + + Write custom queries against your runs, metrics, and LLM usage. + + diff --git a/docs/docs.json b/docs/docs.json index 609ff7b3e16..79a41b9dae6 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -107,6 +107,7 @@ "group": "Features", "pages": [ "ai/prompts", + "ai/observability", "ai-chat/fast-starts", "ai-chat/compaction", "ai-chat/prompt-caching", From 88fe26b44e8481f31114c9b8772c56c12566af51 Mon Sep 17 00:00:00 2001 From: D-K-P <8297864+D-K-P@users.noreply.github.com> Date: Tue, 11 Aug 2026 15:34:48 +0100 Subject: [PATCH 3/3] docs(ai): scope AI SDK 7 telemetry setup to the real behavior AI SDK 7 auto-registration of @ai-sdk/otel only happens for chat agents, so plain tasks must register the OpenTelemetry integration themselves. Also note that chat agents auto-capture only when a prompt is set. --- docs/ai/observability.mdx | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/ai/observability.mdx b/docs/ai/observability.mdx index cb847f79018..2d10c46b51b 100644 --- a/docs/ai/observability.mdx +++ b/docs/ai/observability.mdx @@ -38,7 +38,16 @@ export const summarize = task({ Trigger the task and open the run. The `generateText` call appears as a span in the trace. `streamText` and `generateObject` work the same way: add the same `experimental_telemetry` flag to each call you want captured. - On AI SDK 7, span emission moved out of the `ai` core into the `@ai-sdk/otel` adapter. Install `@ai-sdk/otel` and Trigger.dev registers it for you at run start. AI SDK 6 emits spans from `ai` directly, so no extra package is needed. + **AI SDK 7** moved span emission out of `ai` core into the `@ai-sdk/otel` adapter. In a task, install `@ai-sdk/otel` and register it once yourself, for example at the top of your task file: + + ```ts /trigger/summarize.ts + import { registerTelemetry } from "ai"; + import { OpenTelemetry } from "@ai-sdk/otel"; + + registerTelemetry(new OpenTelemetry()); + ``` + + A [`chat.agent()`](/ai-chat/overview) run registers the adapter for you at run start, so chat agents need only the install. On AI SDK 5 and 6, `ai` core emits spans directly and no adapter is needed. ## What each span shows @@ -101,7 +110,7 @@ const result = await generateText({ Custom attributes are stored on the span's `metadata`, so you can filter or group by them in TRQL, for example `metadata['task.type']`. - When you build an agent with `chat.agent()`, `chat.toStreamTextOptions()` already sets `experimental_telemetry` for you, so generations inside a chat are captured without adding the flag by hand. See [Prompts](/ai/prompts#using-with-chatagent). + When you build an agent with `chat.agent()` and store a prompt with `chat.prompt.set()`, `chat.toStreamTextOptions()` sets `experimental_telemetry` for you, so those generations are captured without adding the flag by hand. Without a stored prompt, set `experimental_telemetry` on the call yourself. See [Prompts](/ai/prompts#using-with-chatagent). ## Query usage across runs