From 619c8ef7acd9f4f7bef71900b984a6022048f7ca Mon Sep 17 00:00:00 2001 From: Alexis Georges Date: Tue, 11 Aug 2026 16:12:01 -0400 Subject: [PATCH] docs: describe the span tree the SDK now emits README and AGENTS.md both still described one flat span per call with four attributes, which is what this SDK emitted before the span work and what neither SDK emits now. A reader following either document would have built the wrong thing. README gains the tree, the rule that tool spans are siblings of chat rather than children, why the root is the only span carrying the LaunchDarkly identity and the run total, and what prompt caching does to the input count. The Anthropic example is the one worth keeping in mind: the provider reports an input of 3 for a turn that processed 23,554 tokens. It also documents that conversation content is off by default and how to turn it on, since that is the change most likely to surprise someone who was reading prompts off their spans. AGENTS.md now points at TELEMETRY-CONTRACT.md as the authority rather than restating a summary that can drift from it, and lists the shared helpers with what each one writes. The instruction that matters most is not to hand-write a span.set_attribute for anything a helper covers: six hand-rolled copies is how these spans drifted apart in the first place. It also records the three things a new handler author would otherwise get wrong: that cache folding belongs at the call site and not in the shared writer, that finish reasons have three mechanisms rather than one, and that `except Exception` does not catch the GeneratorExit a streaming consumer triggers by breaking out of the loop. Each handler package's agents.md gets a four-line note with its span shape and a pointer to the contract, so someone opening one package sees it without reading the root document first. Fixes the README's install command, which named a package that does not exist: launchdarkly-ai rather than launchdarkly-ai-python. --- AGENTS.md | 79 +++++++++++++++++++++++---- README.md | 63 ++++++++++++++++++++- packages/claude-agents/agents.md | 6 ++ packages/claude-messages/agents.md | 6 ++ packages/langchain-agents/agents.md | 6 ++ packages/langchain-messages/agents.md | 6 ++ packages/openai-agents/agents.md | 6 ++ packages/openai-messages/agents.md | 6 ++ 8 files changed, 163 insertions(+), 15 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 179a297..69f1975 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -484,21 +484,76 @@ If `config["tools"]` is absent or empty, tool handling should be skipped entirel ### Telemetry -Handlers must wrap the provider call in an OTel span, following [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/): +`TELEMETRY-CONTRACT.md` at the repo root is the authority for everything in this section. Read it +before changing any span code. What follows is the summary, not the specification. -**Span attributes:** -- `gen_ai.system` — provider identifier (e.g. `"anthropic"`, `"openai"`) -- `gen_ai.operation.name` — operation type (e.g. `"chat"`) -- `gen_ai.request.model` — the model name from `config["model"]["name"]` -- `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.usage.total_tokens` +Every handler emits three levels of span, and all six must agree: -**Span events:** -- `gen_ai.content.prompt` — emitted before the call with the final prompt text -- `gen_ai.content.completion` — emitted after the call with the model's output text +``` +invoke_agent one per call +├── chat {model} one per model turn +└── execute_tool {tool_name} one per tool call, a sibling of chat +``` + +Each package keeps its span construction in a `spans.py` beside its handler, so the tool loop reads +as a tool loop rather than as span bookkeeping with a provider call in the middle. + +Do not hand-write a `span.set_attribute` for anything a shared helper covers. The helpers live in +`launchdarkly_ai_server` and exist because six hand-rolled copies is how these spans drifted apart: + +| Helper | Writes | +|---|---| +| `set_model_identity_attributes` | `gen_ai.system`, `gen_ai.provider.name`, `gen_ai.request.model` | +| `set_usage_span_attributes` | all seven `gen_ai.usage.*` keys, always, including zeros | +| `set_ld_span_attributes` | the `launchdarkly.*` identity and the `feature_flag` event | +| `set_input_content_attributes` | prompts, system instructions, tool catalog, gated | +| `set_output_content_attributes` | model output, gated | +| `set_tool_call_content_attributes` | tool arguments and results, gated | +| `end_span_once` | an idempotent end, marking abandonment | + +#### Where things go + +The root is the only span carrying `launchdarkly.*` and the `feature_flag` +event, because it is the span a config-scoped query finds. It also carries the run's token total, +since summing the children requires having already found them. Children carry neither, and a test +asserts it. + +#### Parent context is explicit + +These handlers open a plain span rather than an active one, so there +is no ambient span for a child to inherit. Pass the parent through. + +#### Cache folding belongs at the call site, never in the shared writer + +Anthropic reports cache +beside the input count, so its handlers add it in. OpenAI and LangChain already count it inside the +input, so theirs pass the figure through. Centralising that rule would double-count for two +providers out of three. `SpanUsage` is the type that means the folding is already done. + +#### Content is off by default + +Every factory takes `capture_content: bool = False`. Guard at the +call site as well as inside the helper: the helper's guard makes a forgotten call site harmless, and +the call site's guard avoids serialising JSON that would then be discarded, once per turn, in a loop. + +#### Finish reasons have three mechanisms, not one + +The Anthropic and LangChain handlers map the +provider's word through the shared table. The two OpenAI handlers use the Responses API, which has no +such field, and derive the value instead. Check the contract before writing one. + +#### Span status + +OK on success. ERROR with the exception recorded on failure, then re-raise. An +abandoned stream is neither: it is marked and left unset. + +#### Streaming needs a `finally` -**Span status:** -- Set to OK on success. -- Set to ERROR and record the exception on failure. Re-raise the error after recording. +`except Exception` does not catch `GeneratorExit`, which is a +`BaseException`, so a consumer that breaks out of the loop skips the error path entirely. Without the +cleanup the root span never ends, never exports, and the run disappears from AI Config Monitoring +along with the `feature_flag` event it carries. Two handlers additionally have a vendor generator or +run to close there; the contract names them. ### Return Shape diff --git a/README.md b/README.md index 4c34a29..6aefc65 100644 --- a/README.md +++ b/README.md @@ -103,7 +103,7 @@ pip install launchdarkly-ai-python launchdarkly-ai-openai-messages **With telemetry** (recommended for production) — traces export to the LaunchDarkly Observability dashboard: ```bash -pip install "launchdarkly-ai[otel]" launchdarkly-ai-openai-messages +pip install "launchdarkly-ai-python[otel]" launchdarkly-ai-openai-messages ``` No code changes are needed — `init_client()` detects whether the OTel packages are present at runtime and configures the tracer provider automatically. If they are absent, the SDK logs a one-time warning and continues normally. @@ -484,7 +484,64 @@ result2 = await config( ## Telemetry -Every handler wraps its provider call in an OpenTelemetry span following [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). The core client also emits LaunchDarkly AI telemetry events (duration, token counts, generation success/failure) automatically on every `config().invoke()` call. No extra instrumentation code is required. +Every handler emits OpenTelemetry spans following the [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). The core client also emits LaunchDarkly AI telemetry events (duration, token counts, generation success and failure) on every `config().invoke()` call. No extra instrumentation code is required. + +### The span tree + +One run produces three levels of span, the same shape from every handler: + +``` +invoke_agent one per call. Carries the LaunchDarkly identity, +│ the feature_flag event, and the run's token total. +├── chat {model} one per model turn, with that turn's own tokens +│ and finish reason. +└── execute_tool {tool_name} one per tool call. +``` + +Tool spans are siblings of `chat`, not children of it. Both hang off the root. + +The root is the only span carrying `launchdarkly.config.key`, `launchdarkly.variation.key`, +`launchdarkly.run.id` and the `feature_flag` event. That is what a config-scoped query in AI Config +Monitoring finds, which is also why the root carries a run total: summing the children requires +having already found them. + +A multi-turn run therefore reports each turn's cost separately, and a run that failed partway still +reports what its completed turns cost. + +### Token counts and prompt caching + +Cached tokens are reported per turn, in `gen_ai.usage.cache_read.input_tokens` and +`gen_ai.usage.cache_creation.input_tokens`, and folded into `gen_ai.usage.input_tokens` so that the +input figure is always the total the model actually processed. + +This matters most on Anthropic, which reports cache reads and writes beside the input count rather +than inside it. A turn that reads 19,971 tokens from cache and writes 3,580 more reports an input of +3 from the provider; the span reports 23,554. + +### Conversation content is off by default + +Prompts, model output, tool arguments and tool results are personal data, so a span carries only +metadata unless you ask for more: models, token counts, timings, tool names. + +Pass `capture_content=True` to a handler factory to include the conversation: + +```python +from launchdarkly_ai_claude_messages import create_claude_messages_handler + +handler = create_claude_messages_handler(capture_content=True) +``` + +Turning this on sends the text of every request and response to whatever collector the SDK points +at. + +### Streaming + +The streaming path emits the same spans as the blocking path. A consumer that stops reading early is +a normal thing, not a failure: the run's spans still close and export, marked with +`launchdarkly.stream.abandoned` and left at an unset status rather than an error, which matches what +LaunchDarkly's own metrics record for an abandoned stream. + +### Graph runs When running inside `graph()`, every node's events carry the graph key, tool invocations emit `$ld:ai:tool_call`, and the graph run itself emits graph-level events (`$ld:ai:graph:invocation_success`/`invocation_failure`, `duration:total`, `total_tokens`, `path`, `handoff_success`/`handoff_failure`). @@ -492,7 +549,7 @@ When running inside `graph()`, every node's events carry the graph key, tool inv The OpenTelemetry SDK packages are **optional** — detected at runtime via `importlib`. The LaunchDarkly server SDK (`launchdarkly-server-sdk`) is also an optional dependency; pass a pre-initialized client to `init_client(client=...)` if you bring your own. -**OTel packages** (installed via `pip install "launchdarkly-ai[otel]"` or `pip install "launchdarkly-ai-server[otel]"`): +**OTel packages** (installed via `pip install "launchdarkly-ai-python[otel]"` or `pip install "launchdarkly-ai-server[otel]"`): - **If installed:** `init_client()` sets up a `TracerProvider` with a GZIP-compressed OTLP HTTP exporter and W3C trace-context/baggage propagators — no code changes needed. - **If not installed:** `init_client()` logs a warning and continues. Feature flags and AI calls work normally; spans become no-ops. diff --git a/packages/claude-agents/agents.md b/packages/claude-agents/agents.md index 45f64c0..4028234 100644 --- a/packages/claude-agents/agents.md +++ b/packages/claude-agents/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-claude-agents` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/claude-messages/agents.md b/packages/claude-messages/agents.md index 801a760..0e43ef0 100644 --- a/packages/claude-messages/agents.md +++ b/packages/claude-messages/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-claude-messages` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/langchain-agents/agents.md b/packages/langchain-agents/agents.md index c27997c..4ca0e15 100644 --- a/packages/langchain-agents/agents.md +++ b/packages/langchain-agents/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-langchain-agents` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/langchain-messages/agents.md b/packages/langchain-messages/agents.md index 1100c24..aa649a4 100644 --- a/packages/langchain-messages/agents.md +++ b/packages/langchain-messages/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-langchain-messages` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/openai-agents/agents.md b/packages/openai-agents/agents.md index 1af937a..88305cd 100644 --- a/packages/openai-agents/agents.md +++ b/packages/openai-agents/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-openai-agents` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/openai-messages/agents.md b/packages/openai-messages/agents.md index cd00791..f5c7372 100644 --- a/packages/openai-messages/agents.md +++ b/packages/openai-messages/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-openai-messages` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. ---