fix(instructor): add model/provider metadata, drop __future__ import#592
Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit intoJul 17, 2026
Merged
Conversation
…ions import Audit against `.agents/skills/sdk-integrations/SKILL.md` found two deviations: - Missing `metadata.model` / `metadata.provider` on the parent `task` span. Per SKILL "framework `llm`-like spans that delegate to a separately instrumented client" — Instructor calls into `wrap_openai`/`wrap_anthropic` which own the `llm` leaf — the framework span should stay `task` but carry `metadata.model` and `metadata.provider` for attribution. Read `model` from call kwargs (`instance.default_model` fallback) and `provider` from Instructor's `Provider` enum (`instance.provider.value`, which is already the short "openai"/"anthropic"/... string we want). - `from __future__ import annotations` in `tracing.py` — violates CLAUDE.md rule 8. Only integration doing this. Removed; `requires-python = ">=3.10"` so `str | None` and `dict[str, Any]` work at runtime unchanged. No new tests, no cassette re-recording — attribution reads from the client instance and call kwargs, not the HTTP response. Extended the two `TestInstructorOpenAISpans` assertions to cover the new keys. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 17, 2026
Abhijeet Prasad (AbhiPrasad)
enabled auto-merge (squash)
July 17, 2026 20:06
Abhijeet Prasad (AbhiPrasad)
deleted the
fix/instructor-provider-metadata
branch
July 17, 2026 20:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Audit of
py/src/braintrust/integrations/instructor/against.agents/skills/sdk-integrations/SKILL.mdturned up two deviations. Everything else was already spec-compliant (task-typed parent, no token metrics on parent to avoid double-counting the OpenAI/Anthropicllmchild, allowlist-only metadata, no eagermodel_dump()/bt_safe_deep_copy, no try/except aroundspan.log,instrumentation.name = "instructor-auto",CompositeFunctionWrapperPatcherover the stableinstructor.core.clientsurface, VCR-backed tests, subprocess auto_instrument test).metadata.model/metadata.provideron the parenttaskspan. SKILL's "Frameworkllm-like spans that delegate to a separately instrumented client" paragraph says the framework span staystaskbut keepsmetadata.model/metadata.providerfor attribution. Instructor is exactly that shape — it calls into a wrapped OpenAI/Anthropic client which emits thellmleaf. Added:_model_name(kwargs, instance)— readskwargs["model"](Instructor passes it straight through) withinstance.default_modelfallback._provider_name(instance)— readsinstance.provider.valuefrom Instructor's built-inProviderenum. Value is already the short string we want ("openai","anthropic","cohere","bedrock","mistral", …). Falls back to.name.lower()for older builds._build_metadatanow emitsmodel+provideralongside the existing 5 keys (response_model,mode,max_retries,retry_count,validation_errors). Still a strict allowlist — no raw kwargs / no denylist.from __future__ import annotationsattracing.py:24violates CLAUDE.md rule 8 ("Do not add … unless absolutely required"). Only integration doing this. Removed;requires-python = ">=3.10"meansstr | Noneanddict[str, Any]already work at runtime unchanged.Explicitly not changed (verified against SKILL and left alone):
task, notllm— provider integrations own thellmleaf and its token accounting.inputstill allowlist{response_model, mode, messages}; no dump of raw kwargs.model_dump()in tracing code._extract_outputis structural (unwrapscreate_with_completion's(model, raw_completion)tuple), not serialization.Test plan
cd py && nox -s "test_instructor(latest)"— 8/8 passed (instructor 1.15.4, openai 2.45.0)cd py && nox -s "test_instructor(1.11.0)"— 8/8 passedTestInstructorOpenAISpans.test_instructor_openai_single_successandtest_instructor_openai_retries_then_succeedsto assertmeta["model"] == "gpt-4o-mini"andmeta["provider"] == "openai".auto_instrumenttest still passes.🤖 Generated with Claude Code
Created by abhijeet
Slack thread