diff --git a/AGENTS.md b/AGENTS.md index 179a297..69f1975 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -484,21 +484,76 @@ If `config["tools"]` is absent or empty, tool handling should be skipped entirel ### Telemetry -Handlers must wrap the provider call in an OTel span, following [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/): +`TELEMETRY-CONTRACT.md` at the repo root is the authority for everything in this section. Read it +before changing any span code. What follows is the summary, not the specification. -**Span attributes:** -- `gen_ai.system` — provider identifier (e.g. `"anthropic"`, `"openai"`) -- `gen_ai.operation.name` — operation type (e.g. `"chat"`) -- `gen_ai.request.model` — the model name from `config["model"]["name"]` -- `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.usage.total_tokens` +Every handler emits three levels of span, and all six must agree: -**Span events:** -- `gen_ai.content.prompt` — emitted before the call with the final prompt text -- `gen_ai.content.completion` — emitted after the call with the model's output text +``` +invoke_agent one per call +├── chat {model} one per model turn +└── execute_tool {tool_name} one per tool call, a sibling of chat +``` + +Each package keeps its span construction in a `spans.py` beside its handler, so the tool loop reads +as a tool loop rather than as span bookkeeping with a provider call in the middle. + +Do not hand-write a `span.set_attribute` for anything a shared helper covers. The helpers live in +`launchdarkly_ai_server` and exist because six hand-rolled copies is how these spans drifted apart: + +| Helper | Writes | +|---|---| +| `set_model_identity_attributes` | `gen_ai.system`, `gen_ai.provider.name`, `gen_ai.request.model` | +| `set_usage_span_attributes` | all seven `gen_ai.usage.*` keys, always, including zeros | +| `set_ld_span_attributes` | the `launchdarkly.*` identity and the `feature_flag` event | +| `set_input_content_attributes` | prompts, system instructions, tool catalog, gated | +| `set_output_content_attributes` | model output, gated | +| `set_tool_call_content_attributes` | tool arguments and results, gated | +| `end_span_once` | an idempotent end, marking abandonment | + +#### Where things go + +The root is the only span carrying `launchdarkly.*` and the `feature_flag` +event, because it is the span a config-scoped query finds. It also carries the run's token total, +since summing the children requires having already found them. Children carry neither, and a test +asserts it. + +#### Parent context is explicit + +These handlers open a plain span rather than an active one, so there +is no ambient span for a child to inherit. Pass the parent through. + +#### Cache folding belongs at the call site, never in the shared writer + +Anthropic reports cache +beside the input count, so its handlers add it in. OpenAI and LangChain already count it inside the +input, so theirs pass the figure through. Centralising that rule would double-count for two +providers out of three. `SpanUsage` is the type that means the folding is already done. + +#### Content is off by default + +Every factory takes `capture_content: bool = False`. Guard at the +call site as well as inside the helper: the helper's guard makes a forgotten call site harmless, and +the call site's guard avoids serialising JSON that would then be discarded, once per turn, in a loop. + +#### Finish reasons have three mechanisms, not one + +The Anthropic and LangChain handlers map the +provider's word through the shared table. The two OpenAI handlers use the Responses API, which has no +such field, and derive the value instead. Check the contract before writing one. + +#### Span status + +OK on success. ERROR with the exception recorded on failure, then re-raise. An +abandoned stream is neither: it is marked and left unset. + +#### Streaming needs a `finally` -**Span status:** -- Set to OK on success. -- Set to ERROR and record the exception on failure. Re-raise the error after recording. +`except Exception` does not catch `GeneratorExit`, which is a +`BaseException`, so a consumer that breaks out of the loop skips the error path entirely. Without the +cleanup the root span never ends, never exports, and the run disappears from AI Config Monitoring +along with the `feature_flag` event it carries. Two handlers additionally have a vendor generator or +run to close there; the contract names them. ### Return Shape diff --git a/README.md b/README.md index 4c34a29..6aefc65 100644 --- a/README.md +++ b/README.md @@ -103,7 +103,7 @@ pip install launchdarkly-ai-python launchdarkly-ai-openai-messages **With telemetry** (recommended for production) — traces export to the LaunchDarkly Observability dashboard: ```bash -pip install "launchdarkly-ai[otel]" launchdarkly-ai-openai-messages +pip install "launchdarkly-ai-python[otel]" launchdarkly-ai-openai-messages ``` No code changes are needed — `init_client()` detects whether the OTel packages are present at runtime and configures the tracer provider automatically. If they are absent, the SDK logs a one-time warning and continues normally. @@ -484,7 +484,64 @@ result2 = await config( ## Telemetry -Every handler wraps its provider call in an OpenTelemetry span following [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). The core client also emits LaunchDarkly AI telemetry events (duration, token counts, generation success/failure) automatically on every `config().invoke()` call. No extra instrumentation code is required. +Every handler emits OpenTelemetry spans following the [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). The core client also emits LaunchDarkly AI telemetry events (duration, token counts, generation success and failure) on every `config().invoke()` call. No extra instrumentation code is required. + +### The span tree + +One run produces three levels of span, the same shape from every handler: + +``` +invoke_agent one per call. Carries the LaunchDarkly identity, +│ the feature_flag event, and the run's token total. +├── chat {model} one per model turn, with that turn's own tokens +│ and finish reason. +└── execute_tool {tool_name} one per tool call. +``` + +Tool spans are siblings of `chat`, not children of it. Both hang off the root. + +The root is the only span carrying `launchdarkly.config.key`, `launchdarkly.variation.key`, +`launchdarkly.run.id` and the `feature_flag` event. That is what a config-scoped query in AI Config +Monitoring finds, which is also why the root carries a run total: summing the children requires +having already found them. + +A multi-turn run therefore reports each turn's cost separately, and a run that failed partway still +reports what its completed turns cost. + +### Token counts and prompt caching + +Cached tokens are reported per turn, in `gen_ai.usage.cache_read.input_tokens` and +`gen_ai.usage.cache_creation.input_tokens`, and folded into `gen_ai.usage.input_tokens` so that the +input figure is always the total the model actually processed. + +This matters most on Anthropic, which reports cache reads and writes beside the input count rather +than inside it. A turn that reads 19,971 tokens from cache and writes 3,580 more reports an input of +3 from the provider; the span reports 23,554. + +### Conversation content is off by default + +Prompts, model output, tool arguments and tool results are personal data, so a span carries only +metadata unless you ask for more: models, token counts, timings, tool names. + +Pass `capture_content=True` to a handler factory to include the conversation: + +```python +from launchdarkly_ai_claude_messages import create_claude_messages_handler + +handler = create_claude_messages_handler(capture_content=True) +``` + +Turning this on sends the text of every request and response to whatever collector the SDK points +at. + +### Streaming + +The streaming path emits the same spans as the blocking path. A consumer that stops reading early is +a normal thing, not a failure: the run's spans still close and export, marked with +`launchdarkly.stream.abandoned` and left at an unset status rather than an error, which matches what +LaunchDarkly's own metrics record for an abandoned stream. + +### Graph runs When running inside `graph()`, every node's events carry the graph key, tool invocations emit `$ld:ai:tool_call`, and the graph run itself emits graph-level events (`$ld:ai:graph:invocation_success`/`invocation_failure`, `duration:total`, `total_tokens`, `path`, `handoff_success`/`handoff_failure`). @@ -492,7 +549,7 @@ When running inside `graph()`, every node's events carry the graph key, tool inv The OpenTelemetry SDK packages are **optional** — detected at runtime via `importlib`. The LaunchDarkly server SDK (`launchdarkly-server-sdk`) is also an optional dependency; pass a pre-initialized client to `init_client(client=...)` if you bring your own. -**OTel packages** (installed via `pip install "launchdarkly-ai[otel]"` or `pip install "launchdarkly-ai-server[otel]"`): +**OTel packages** (installed via `pip install "launchdarkly-ai-python[otel]"` or `pip install "launchdarkly-ai-server[otel]"`): - **If installed:** `init_client()` sets up a `TracerProvider` with a GZIP-compressed OTLP HTTP exporter and W3C trace-context/baggage propagators — no code changes needed. - **If not installed:** `init_client()` logs a warning and continues. Feature flags and AI calls work normally; spans become no-ops. diff --git a/packages/claude-agents/agents.md b/packages/claude-agents/agents.md index 45f64c0..4028234 100644 --- a/packages/claude-agents/agents.md +++ b/packages/claude-agents/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-claude-agents` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/claude-messages/agents.md b/packages/claude-messages/agents.md index 801a760..0e43ef0 100644 --- a/packages/claude-messages/agents.md +++ b/packages/claude-messages/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-claude-messages` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/langchain-agents/agents.md b/packages/langchain-agents/agents.md index c27997c..4ca0e15 100644 --- a/packages/langchain-agents/agents.md +++ b/packages/langchain-agents/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-langchain-agents` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/langchain-messages/agents.md b/packages/langchain-messages/agents.md index 1100c24..aa649a4 100644 --- a/packages/langchain-messages/agents.md +++ b/packages/langchain-messages/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-langchain-messages` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/openai-agents/agents.md b/packages/openai-agents/agents.md index 1af937a..88305cd 100644 --- a/packages/openai-agents/agents.md +++ b/packages/openai-agents/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-openai-agents` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. --- diff --git a/packages/openai-messages/agents.md b/packages/openai-messages/agents.md index cd00791..f5c7372 100644 --- a/packages/openai-messages/agents.md +++ b/packages/openai-messages/agents.md @@ -1,5 +1,11 @@ # Agent Guide — `launchdarkly-ai-openai-messages` +> **Span shape.** This package emits `invoke_agent` → `chat {model}` → `execute_tool {name}`, with +> tool spans as siblings of `chat`. Span construction lives in `spans.py` beside the handler. +> Conversation content is off unless the caller passes `capture_content=True`. +> `TELEMETRY-CONTRACT.md` at the repo root is the authority; read it before changing span code. + + This document tells an agent exactly how this package is implemented so it can be correctly modified, debugged, or used as a reference when building a new handler. ---