Skip to content

fix(OPEN-11695): recognize LangChain Gemini provider/model and populate cost + usageDetails#658

Open
viniciusdsmello wants to merge 1 commit into
mainfrom
vini/open-11695-langchain-provider-string-chat-google-generative-ai-not
Open

fix(OPEN-11695): recognize LangChain Gemini provider/model and populate cost + usageDetails#658
viniciusdsmello wants to merge 1 commit into
mainfrom
vini/open-11695-langchain-provider-string-chat-google-generative-ai-not

Conversation

@viniciusdsmello

@viniciusdsmello viniciusdsmello commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Problem

LangChain traces reported $0 cost with empty costDetails/usageDetails. The customer (Globo, Python SDK + ChatGoogleGenerativeAI) saw provider: chat-google-generative-ai, model: models/gemini-3.5-flash, costDetails: {}. The cost table keys on (provider, model), so an unrecognized provider or model silently yields $0.

Closes OPEN-11695.

Context after rebase

Rebased onto main, which now includes the v1 callback modernization (OPEN-11315). That work resolves the provider from metadata["ls_provider"] (so ChatGoogleGenerativeAIGoogle when ls_provider is present) and captures granular token_details into step metadata. This PR closes the gaps that still produced $0 / empty cost fields on top of it.

Fix

  1. Provider anchor for the exact reported string. Map the chat-google-generative-ai _typeGoogle in the legacy _type map, so the string named in this issue still resolves when metadata["ls_provider"] is absent (older langchain-google-genai, or callers that don't forward it) — not only via the ls_provider path.

  2. Strip the Gemini models/ prefix in _extract_model_infoChatGoogleGenerativeAI reports models/gemini-3.5-flash, which the cost table (bare names) doesn't match, so the lookup misses → $0. Runs before the existing LiteLLM prefix handling; models is never a provider, so it's safe.

  3. Emit a priced usageDetails map so the backend populates costDetails. Main only surfaced token_details as informational metadata; this adds the priced column. LangChain reports cache/audio categories as overlapping subsets of the input/output totals, so they're re-partitioned to be non-overlapping and mapped to the cost table's keys:

    • cache_readcached_tokens, cache_creationcache_creation_tokens
    • input audioaudio_input_tokens, output audioaudio_output_tokens
    • reasoning stays folded into output_tokens (billed at the output rate; no separate price).

    ChatCompletionStep gains an optional usage_details field, serialized as usageDetails only when set — other integrations are unaffected.

Verification

  • 44 tests in tests/lib/integrations/test_langchain_callback.py (main's + those added here), all passing; ruff clean. Guarded by pytest.importorskip("langchain_core") (langchain is an optional dep). Includes a regression test that chat-google-generative-ai resolves to Google with ls_provider absent.

  • End-to-end against a live pipeline with the customer's inputs (27131 prompt / 17739 completion):

    field before after
    provider chat-google-generative-ai Google
    model models/gemini-3.5-flash gemini-3.5-flash
    cost $0 0.2003475
    costDetails {} {input_tokens, output_tokens} populated
  • With caching (10 000 cached input tokens): usageDetails = {input_tokens: 17131, cached_tokens: 10000, output_tokens: 17739} (non-overlapping, sums to total), costDetails prices cached_tokens at the cheaper rate, cost = 0.1868475 — correctly below the non-cached 0.2003475. A cross-provider audit confirms the cost table uses these category keys uniformly (anthropic/openai/bedrock/azure/… all use cached_tokens/cache_creation_tokens/audio_*).

Scope / follow-up

  • SDK-only: helps future traces from updated SDKs; does not backfill existing $0 rows.
  • The cost table is LiteLLM-fragmented across google/gemini/vertex_ai; Google + bare model resolves the reported case and most current Gemini models but not every variant — a general backend normalize-before-lookup layer would close that fully. The same $0 symptom also appears on the Langfuse/OTLP ingestion path, untouched here. Both are candidate backend companion tickets.

🤖 Generated with Claude Code

@viniciusdsmello viniciusdsmello force-pushed the vini/open-11695-langchain-provider-string-chat-google-generative-ai-not branch 3 times, most recently from 7684bb5 to eabfd4e Compare July 13, 2026 15:22
@viniciusdsmello viniciusdsmello self-assigned this Jul 13, 2026
@viniciusdsmello viniciusdsmello force-pushed the vini/open-11695-langchain-provider-string-chat-google-generative-ai-not branch from eabfd4e to a103ae1 Compare July 13, 2026 15:44
@viniciusdsmello viniciusdsmello changed the title fix(OPEN-11695): normalize LangChain provider/model and emit usageDetails for cost estimation fix(OPEN-11695): recognize LangChain Gemini provider/model and populate cost + usageDetails Jul 13, 2026
…sageDetails so LangChain cost estimation stops returning $0

Builds on the v1 callback modernization (OPEN-11315), which resolves the
provider from `metadata["ls_provider"]` (ChatGoogleGenerativeAI -> "Google").
Remaining gaps that still produced $0 / empty cost fields:

- Provider anchor: also map the `chat-google-generative-ai` _type to "Google"
  in the legacy _type map, so the exact reported string still resolves when
  `metadata["ls_provider"]` is absent (older langchain-google-genai, or callers
  that don't forward it) -- not just via the ls_provider path.

- Model: strip the Gemini Developer API "models/" prefix (e.g.
  "models/gemini-3.5-flash" -> "gemini-3.5-flash"), which the cost table (bare
  names) otherwise misses -> $0. Runs before the LiteLLM prefix handling
  ("models" is never a provider, so it's safe).

- costDetails/usageDetails: the handler surfaced granular token_details only as
  informational step metadata, never as a priced column. Emit a per-category
  `usageDetails` map so the backend prices it into `costDetails`. LangChain
  reports cache/audio categories as overlapping subsets of the input/output
  totals, so they are re-partitioned to be non-overlapping and mapped to the
  cost table's keys (cache_read -> cached_tokens, cache_creation ->
  cache_creation_tokens, audio -> audio_input_tokens/audio_output_tokens);
  reasoning stays folded into output_tokens. ChatCompletionStep gains an
  optional usage_details field, serialized only when set.

Verified end-to-end against the live pipeline with the customer's inputs
(27131 prompt / 17739 completion): provider=Google, model=gemini-3.5-flash,
cost=0.2003475, costDetails populated; and with 10000 cached input tokens the
partition prices cached tokens at the cheaper rate (cost=0.1868475). Cross-
provider audit confirms the cost table uses these category keys uniformly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018t3JHc1kK6pPMdVBVwMbTs
@viniciusdsmello viniciusdsmello force-pushed the vini/open-11695-langchain-provider-string-chat-google-generative-ai-not branch from a103ae1 to fc3bfd3 Compare July 13, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant