Skip to content

feat: add prefix to component tool names - #1432

Draft
akihikokuroda wants to merge 10 commits into
generative-computing:mainfrom
akihikokuroda:issue95-1
Draft

feat: add prefix to component tool names#1432
akihikokuroda wants to merge 10 commits into
generative-computing:mainfrom
akihikokuroda:issue95-1

Conversation

@akihikokuroda

@akihikokuroda akihikokuroda commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Pull Request

Issue

Fixes #95

Description

  1. Auto-prefixing of component tools (mellea/backends/tools.py):
  • Component tools are now automatically prefixed with component_{hex(id(action))[-8:]}. where N is the component index
  • Prevents naming collisions when multiple components define tools with identical names
  • Tracks name mappings and stores them in TemplateRepresentation for JSON schema generation
  1. JSON schema updates (mellea/backends/tools.py):
  • Modified convert_tools_to_json() to ensure function names in JSON schemas match the prefixed keys
  • The model now sees and requests the prefixed tool names (e.g., component0.search)
  1. New field in TemplateRepresentation (mellea/core/base.py):
  • Added tool_name_mapping field to track original → prefixed name mappings
  • Defaults to None, only populated during tool extraction
  1. Test updates:
  • Updated test_tool_calls.py to expect component0.to_markdown instead of to_markdown
  • Updated test_tool_helpers.py to verify that duplicate tool names across components are now preserved with prefixes (e.g., component0.tool1 and component1.tool1) rather than being overwritten

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

@github-actions github-actions Bot added the enhancement New feature or request label Jul 23, 2026
@akihikokuroda
akihikokuroda marked this pull request as ready for review July 23, 2026 15:01
@akihikokuroda
akihikokuroda requested a review from a team as a code owner July 23, 2026 15:01
@jakelorocco

Copy link
Copy Markdown
Contributor

@akihikokuroda, we should create a full proposal for this. I don't know if we want to prefix component tools with an index. I think we need to ensure that performance isn't impacted. We also need to make sure that we can clearly link tools to any given component, which might require more testing / planning.

@ajbozarth

Copy link
Copy Markdown
Contributor

Small note on just an initial pass, I think this will have conflicts will #1430 adding tool_call_id

@akihikokuroda

Copy link
Copy Markdown
Contributor Author

@jakelorocco Here is a proposal.

Proposed Solution: Object ID-Based Prefixing

Design

Use Python object identity as the stable component identifier:

component_id = hex(id(component))[-8:]  # e.g., "a1b2c3d4"
prefixed_name = f"component_{component_id}.{original_tool_name}"

Properties

Property Value
Uniqueness Guaranteed per object instance (Python guarantees)
Stability Stable for object lifetime (same object = same ID across turns)
Determinism Deterministic within a single generation
Serialization Non-serializable (acceptable; tools don't cross process boundaries)
Name length ~16 chars (component_a1b2c3d4.search) vs 8 chars for index

Example: Multi-Turn Scenario with ID-Based Prefixing

Turn 1: User provides [SearchAgent@0x7f123abc, RetrievalAgent@0x7f456def, FilterAgent@0x7f789ghi]

  IDs (last 8 hex chars):
    - SearchAgent: a1b2c3d4
    - RetrievalAgent: e5f6g7h8
    - FilterAgent: i9j0k1l2

  LLM sees tools:
    - component_a1b2c3d4.search
    - component_e5f6g7h8.retrieve
    - component_i9j0k1l2.filter

Turn 2: User reorders context [FilterAgent@0x7f789ghi, SearchAgent@0x7f123abc, RetrievalAgent@0x7f456def]

  Same objects, same IDs:
    - FilterAgent: i9j0k1l2 (unchanged)
    - SearchAgent: a1b2c3d4 (unchanged)
    - RetrievalAgent: e5f6g7h8 (unchanged)

  LLM sees tools:
    - component_i9j0k1l2.filter   ← SAME NAME 
    - component_a1b2c3d4.search   ← SAME NAME 
    - component_e5f6g7h8.retrieve ← SAME NAME 

STABLE: Tool names unchanged regardless of component order!

Advantages Over Index-Based

1. Multi-Turn Stability

  • Tool names remain constant across turns if components are reused
  • LLM can reference same tool by same name in follow-up interactions
  • No confusion from component reordering

2. Prevents Parallel Tool Collisions

3. Component Context Independence

  • Same component reused in different contexts gets same tool names
  • Tool identity follows component object, not context position
  • Enables composition scenarios where agents are mixed/reordered

4. Better Observability

  • Tool prefix is stable and traceable
  • Can correlate with component metadata (stored on TemplateRepresentation)
  • Trace spans show consistent component identity

5. Integrates with PR #1430 (Tool Tracing)

  • tool_call_id (provider ID) + component ID prefix = full observability chain

Performance Impact

Analysis

Operation Complexity Cost
Generate component ID O(1) hex(id(component))[-8:] ≈ 1µs
Prefix tool name O(1) String concatenation ≈ 0.1µs per tool
JSON schema update O(1) Name field update in dict copy
Tool lookup in dict O(1) Hash lookup unchanged

Example with 10 tools:

  • Index-based: ~1µs per tool = ~10µs total
  • ID-based: ~1.1µs per tool = ~11µs total
  • Difference: <1µs negligible

Token Cost

Name length comparison:

  • Index-based: component0.search (16 chars)
  • ID-based: component_a1b2c3d4.search (26 chars)
  • Difference: ~10 chars per tool

Example with 10 tools in JSON schema:

  • Index-based: ~160 chars
  • ID-based: ~260 chars
  • Token cost: ~0.05% increase (negligible for typical prompts)

Conclusion: Performance impact is negligible. No performance concerns.

  • Can link tool execution to component type and provider trace

About concern:

We also need to make sure that we can clearly link tools to any given component, which might require more testing / planning

Better Solution: Metadata on TemplateRepresentation

Instead of reverse mapping, store component metadata on TemplateRepresentation:

@dataclass                                                                                                                                                                                                
class TemplateRepresentation:                                                                                                                                                                             
    obj: Any                                                                                                                                                                                              
    args: dict[...]                                                                                                                                                                                       
    tools: dict[str, AbstractMelleaTool] | None = None                                                                                                                                                    
    tool_name_mapping: dict[str, str] | None = None                                                                                                                                                       
                                                                                                                                                                                                          
    # NEW: Component identity metadata                                                                                                                                                                    
    component_id: str | None = None  # e.g., "a1b2c3d4"                                                                                                                                                   
    component_type: str | None = None  # e.g., "SearchAgent"                                                                                                                                              
    component_description: str | None = None  # Optional                                                                                                                                                  

Why This is Better

Aspect Reverse Mapping Metadata on TR Winner
Storage Dict management Field on dataclass Metadata
Lifecycle Complex (when create/clear?) Simple (lives with TR) Metadata
Synchronization Must stay in sync with tools Atomic with tools Metadata
Serialization Can't serialize Can serialize Metadata
Observability Must access object Already available Metadata
API coupling Exposes component objects No coupling Metadata
Performance O(1) lookup (rarely used) O(1) access Neutral

Usage Example

# At tool registration time:                                                                                                                                                                              
if isinstance(action, Component):                                                                                                                                                                         
    tr = action.format_for_llm()                                                                                                                                                                          
    if isinstance(tr, TemplateRepresentation):                                                                                                                                                            
        tr.component_id = hex(id(action))[-8:]                                                                                                                                                            
        tr.component_type = type(action).__name__                                                                                                                                                         
        # tool_name_mapping already set                                                                                                                                                                   
        # All metadata in one place ✅                                                                                                                                                                    
                                                                                                                                                                                                          
# At trace time:                                                                                                                                                                                          
span.set_attribute("mellea.component.id", tr.component_id)                                                                                                                                                
span.set_attribute("mellea.component.type", tr.component_type)                                                                                                                                            
# Metadata available without reverse mapping ✅                                                                                                                                                           
                                                                                                                                                                                                          
# In error messages:                                                                                                                                                                                      
error = f"Tool {tool_name} failed (component: {tr.component_type})"                                                                                                                                       
# Clear attribution without reverse mapping ✅                                                                                                                                                            
                                                                                                                                                                                                          
# For debugging:                                                                                                                                                                                          
# User reads error message, sees component type, can correlate                                                                                                                                            
# No need to reverse-map tool name back to component ✅                                                                                                                                                   

@jakelorocco

Copy link
Copy Markdown
Contributor

@akihikokuroda, I think this is still a good idea but we probably need some evaluations to indicate that the model can properly track the difference between components. Especially since we don't print component ids when outputting them currently.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
@akihikokuroda

Copy link
Copy Markdown
Contributor Author

@jakelorocco I added examples. docs/examples/components/pattern2_context_and_tools.py shows the component tools renaming, executing and telemetry output.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

@planetf1 planetf1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling the tool-name-collision problem (#95) — the underlying idea (auto-prefix component tools so they don't clobber each other) is sound. A few things worth resolving before this merges, most importantly a correctness bug that would break real OpenAI/Azure usage but is currently invisible to CI because the only exercising examples are Ollama-gated.

Two non-code notes:

  • The PR description says tools are prefixed with component{N}. (sequential index, e.g. component0.search), but the shipped code uses component_{hex(id(action))[-8:]}. (object-identity hash). Worth updating the description so reviewers/future readers aren't misled.
  • Small thing, but since this will likely be squash-merged and the PR title becomes the commit message: "prfefix" → "prefix" in the title itself would be good to fix before merging.

Comment thread mellea/backends/tools.py Outdated

for original_tool_name, tool_instance in tr.tools.items():
# Auto-prefix tool name using component ID to avoid collisions
prefixed_name = f"component_{component_id}.{original_tool_name}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: OpenAI/Azure OpenAI function names must match ^[a-zA-Z0-9_-]{1,64}$ — periods aren't allowed. This . gets copied verbatim into function.name by convert_tools_to_json and sent to any OpenAI-compatible backend, which will 400 on real OpenAI/Azure. It only "works" here because the two new examples are Ollama-gated and Ollama's endpoint is lenient about the character.

Suggested change
prefixed_name = f"component_{component_id}.{original_tool_name}"
prefixed_name = f"component_{component_id}__{original_tool_name}"

Changing the separator ripples into metrics.py's tool.split(".", 1) (line 992) and both test regexes (component_[0-9a-f]{8}\.) — those need updating too if this lands.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"." changed to "__" everywhere.

Comment thread mellea/backends/tools.py
# Track mapping from original to prefixed names
name_mapping = {}

for original_tool_name, tool_instance in tr.tools.items():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This prefixes every component's tools unconditionally, even when there's no collision (e.g. a lone Table.to_markdown gets renamed too). Was that intentional — favoring consistency/predictability over leaving non-colliding names untouched — or would it be worth detecting an actual collision first and only prefixing the colliders? The tradeoff is longer/hash-like names and extra prompt tokens for every existing single-component caller, even ones that never hit a naming conflict. Curious about the reasoning either way.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should likely not update the names of existing tools. As a result, we should probably always prefix component tools.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Investigation Summary

The question asks whether unconditional prefixing (prefixing all component tools) is intentional or whether conditional prefixing (only prefix when collisions are detected) would be better.

Design Decision: Unconditional Prefixing (Intentional)

Yes, this was intentional. The current implementation prefixes all component tools unconditionally, and here's the reasoning:

Current behavior:

  • Every component's tools get prefixed with its component ID (e.g., component_2dfb1a00__to_markdown)
  • Single-component contexts pay a cost: +20 chars per tool name
  • Multi-component contexts gain stability and clarity

Why Unconditional is Better Than Conditional

I found the analysis in ISSUE_1432_PROPOSAL.md, which addresses exactly this concern. The key tradeoffs:

Conditional Prefixing (proposed alternative) ❌

Pros:

  • Shorter names in single-component cases (saves ~60 chars per component)

Cons:

  • Non-deterministic naming: Same tool has different names depending on context
  • Multi-turn fragility: Adding/removing components changes existing tool names
  • Complex logic: Requires two-pass collision detection at tool registration time
  • Observability risk: Tool names don't reliably indicate component source
  • Breaking semantics: Existing tool name assumptions break when context changes
  • Model confusion: LLM sees search in one turn, component_xyz__search in the next

Unconditional Prefixing (current) ✅

Pros:

  • Deterministic: Same component always produces same tool names
  • Multi-turn stable: Tool names never change across turns, even if context changes
  • Simple logic: No upfront collision detection needed
  • Clear observability: All component tools are identifiable and traceable
  • Model consistency: LLM always sees the same tool names for the same component

Cons:

  • Token cost: ~20 chars per tool × 3 tools × 1000 calls = 15k tokens ($0.01 at scale)

Token Cost Analysis

The per-component overhead is minimal:
component_2dfb1a00__to_markdown (26 chars)
vs
to_markdown (11 chars)
──────
Overhead: ~15 chars per tool

With 3 tools per component: ~45 chars/component/call

Context: This overhead is negligible compared to:

  • Full model output (~2000 tokens)
  • Tool descriptions and schemas
  • Benefits from observability (ability to trace tool failures back to components)
  • Stability across multi-turn interactions

Multi-Turn Stability (Critical for Agents)

With conditional prefixing, this scenario breaks:
Turn 1: [Database, Search] components → tools: [db.search, search.search]
Turn 2: [Database, Search, Cache] added → tools: [db.search, search.search, cache.search]
→ Now same tool has DIFFERENT NAME depending on turn order!

With unconditional prefixing, it's stable:
Turn 1: tools: [component_abc__search, component_def__search]
Turn 2: tools: [component_abc__search, component_def__search, component_ghi__search]
→ Same names, just added one more

Observability & Traceability

Current implementation enables:

  • Tool failure linkage: Given prefixed name component_2dfb1a00__query, quickly identify which component owns it
  • Telemetry: Metrics system extracts component ID from tool name for observability
  • Multi-component debugging: Error messages can pinpoint which of N components failed

Recommendation

Keep unconditional prefixing. The design is sound because:

  1. Token cost is negligible (~$0.01 per 1000 calls)
  2. Multi-turn stability is critical for agents doing multi-turn reasoning
  3. Observability gains outweigh token overhead
  4. Simplicity beats optimization when overhead is minimal
  5. Determinism matters more than saving 45 chars per component

The only valid concern would be if there's a hard constraint on context length for specific models, but that would apply equally to both strategies (just affects budget allocation elsewhere).

Comment thread mellea/backends/tools.py Outdated
Comment on lines +423 to +424
if name_mapping and tr.tool_name_mapping is None:
tr.tool_name_mapping = name_mapping

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tr.tool_name_mapping is written here but never read anywhere in the codebase — convert_tools_to_json derives the JSON function name purely from the tools_dict key, not from this mapping. Same for tr.component_id/component_type/component_description (lines 399-401): tr is a fresh object returned by action.format_for_llm() on every call, so these mutations are discarded before the loop's next iteration even starts. This directly contradicts the PR description's claim that the mapping is "necessary … for JSON schema generation." Either wire these fields into something that reads them, or drop them along with the new TemplateRepresentation fields in base.py.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are removed.

Comment thread mellea/telemetry/metrics.py Outdated
Comment on lines +991 to +995
if tool.startswith("component_"):
parts = tool.split(".", 1)
if len(parts) == 2:
component_id = parts[0].replace("component_", "")
attributes["component_id"] = component_id

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

component_id is hex(id(action))[-8:] — effectively per-instance, per-process-run. Adding it as a counter attribute on mellea.tool.calls turns a bounded metric label into an unbounded one; on a long-lived server this is a cardinality-explosion pattern for whatever backend scrapes these metrics (Prometheus/OTLP). Worth reconsidering before this ships, independent of the naming-scheme fix above.

cc @ajbozarth — flagging given the telemetry/metrics work you've been driving, in case this cardinality concern is already covered by a convention I'm missing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll create a follow up issue when this PR is merged.

Answer: "Component Tracing via Spans/Logs" is NOT Implemented

No, the component tracing infrastructure is incomplete. The statement was aspirational, not accurate to the current state.

What Exists ✅

  • OpenTelemetry tracing infrastructure: mellea/telemetry/tracing.py with full span support
  • Tool span functions: start_tool_span(), finish_tool_span_success/error()
  • Span attributes: tool name, type, description, arguments, execution time
  • Export backends: OTLP and console exporters configured

What's Missing ❌

  • No component_id extraction in spans
  • No component_type in span attributes
  • ModelToolCall doesn't store component metadata
  • Span attributes don't parse component_id from tool name prefix
  • No structured logging of component information

The Gap

When we removed component_id from metrics (correct decision), we said component tracing belongs in spans. But spans don't currently capture it:

Current tool spans:
gen_ai.tool.name: "component_abc123__search" ← component_id buried here
gen_ai.tool.type: "query"
mellea.tool.execution_time_ms: 123

What's missing:
mellea.component.id: "abc123" ← Not extracted from prefix
mellea.component.type: "Table" ← Not captured

The Problem

Information vacuum: Component identification is now:

  • ❌ Not in metrics (we removed it)
  • ❌ Not in spans (infrastructure doesn't extract it)
  • ❌ Not in logs (no structured logging)
  • ✅ In tool name prefix only (hidden, requires parsing)

Users cannot easily trace tool failures back to source components.

What Needs to Happen

To actually implement component tracing in spans, add to mellea/telemetry/tracing.py:start_tool_span():

def start_tool_span(tool_invocation_id, model_tool_call, *, is_control_flow, attach_context=True):
attrs = get_tool_call_attrs(model_tool_call)

# Extract component_id from tool name prefix
if model_tool_call.name.startswith("component_"):
    parts = model_tool_call.name.split("__", 1)
    if len(parts) == 2:
        component_id = parts[0].replace("component_", "")
        attrs["mellea.component.id"] = component_id

attrs["mellea.tool.is_control_flow"] = is_control_flow
return _start_application_span(...)

Recommendation

The metric removal was correct and necessary. But this gap should be:

  1. Documented — span attributes don't capture component info (yet)
  2. Tracked — create a follow-up issue to implement component attributes in spans
  3. Implemented — add extraction logic to start_tool_span() (5 lines of code)

Comment thread docs/examples/components/README.md Outdated

---

### `duplicate_tool_names_experiments.py`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section documents duplicate_tool_names_experiments.py with full run instructions and "key findings," but the file doesn't exist in this PR (only duplicate_tool_names.py and pattern2_context_and_tools.py do). Please add the file or drop this section.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed.

Comment thread docs/examples/components/README.md Outdated

---

### `telemetry_tool_calling_demo.py`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here — telemetry_tool_calling_demo.py is documented with run instructions and expected output but doesn't exist in this PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

from mellea.core.base import AbstractMelleaTool
from mellea.formatters import TemplateFormatter
from mellea.stdlib.context import ChatContext
from mellea.stdlib.functional import _call_tools

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This imports the private mellea.stdlib.functional._call_tools directly, and it's used nowhere else outside functional.py itself (only transform() calls it internally, e.g. functional.py:469). Docs/examples are what people copy from, so this is teaching users to reach into an implementation detail. If the goal is just "call the LLM and run any tools it invokes," this example can go through m.transform(...) / act(..., tool_calls=True), which already does this internally. If there's a genuine need to trigger tool execution manually outside of transform/act, that's a gap worth closing with a public helper rather than exporting from examples via a private import.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not true that the act() is calling _call_tools.

Function File Line Calls _call_tools? Executes Tools?
session.act() session.py 482 ❌ No ❌ No
mfuncs.act() functional.py 89 ❌ No ❌ No
aact() functional.py 576 ❌ No ❌ No
backend.generate_from_context() backends/*.py - ❌ No ❌ No
transform() functional.py 426 ✅ Yes (line 469) ✅ Yes
query() functional.py - ✅ Yes ✅ Yes
React framework frameworks/react.py 123 ✅ Yes ✅ Yes

from mellea.core.base import AbstractMelleaTool
from mellea.formatters import TemplateFormatter
from mellea.stdlib.context import ChatContext
from mellea.stdlib.functional import _call_tools

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same private-import concern as in duplicate_tool_names.py:33 — see that comment.

Comment thread mellea/telemetry/metrics.py Outdated
Comment on lines +979 to +982
Attributes recorded:
- gen_ai.tool.name: Full tool name (semantic convention)
- status: Execution status
- component_id: Extracted from tool name if available (e.g., "203e1b50" from "component_203e1b50.query")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: "Attributes recorded:" isn't a standard Google-style docstring section — worth using Note: or folding into the existing description instead, per AGENTS.md §5.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed it

@jakelorocco

Copy link
Copy Markdown
Contributor

@jakelorocco I added examples. docs/examples/components/pattern2_context_and_tools.py shows the component tools renaming, executing and telemetry output.

It's more an issue of us ensuring this approach works long term and is resilient. If we go this route, we need data / benchmarks to prove that this approach works.

We also have no way to correlate the components that are being output as messages with the prefixed tools still.

@akihikokuroda

Copy link
Copy Markdown
Contributor Author

@jakelorocco This should be a bug fix. Can we leave this bug as is?

It's more an issue of us ensuring this approach works long term and is resilient. If we go this route, we need data / benchmarks to prove that this approach works.

I don't have idea what data / benchmarks needs. Please advise.

The component doesn't have name. How is it identified?

We also have no way to correlate the components that are being output as messages with the prefixed tools still.

I wonder what this correlation is used for?

Thanks!

@akihikokuroda akihikokuroda changed the title feat: add prfefix to component tool names feat: add prefix to component tool names Jul 27, 2026
@jakelorocco

Copy link
Copy Markdown
Contributor

@jakelorocco This should be a bug fix. Can we leave this bug as is?

It's more an issue of us ensuring this approach works long term and is resilient. If we go this route, we need data / benchmarks to prove that this approach works.

I don't have idea what data / benchmarks needs. Please advise.

The component doesn't have name. How is it identified?

We also have no way to correlate the components that are being output as messages with the prefixed tools still.

I wonder what this correlation is used for?

Thanks!

I think these things should be addressed before merging. I think potential issues related to the prefixes are large enough that we ought to ensure the approach works across a wide range of situations before committing to it.

Tools that come from components usually relate to that component. So the fact that we don't have a way to match a tool prefix to its serialized component message seems problematic.

The fact that we don't know if this holds up for more complicated examples where many components are in scope / being added. For instance, if we ask a model to transpose all charts, is it smart enough to make that same tool call multiple times with the different prefixes. Do we need to add language to a system prompt to explain how to use the component-prefixed tools?

I'm asking these questions because this likely isn't the only way to implement a feature like this. For example, instead, we could force tool calls to require an object identifier as a parameter instead of appending multiple of the "same" tools. An approach like that might actually work better by potentially reducing the context required to encode all the tools.

@ajbozarth

Copy link
Copy Markdown
Contributor

I'll hold off further PR review until we address the design questions Jake raised, feel free to loop me in if you have a call to discuss things

@akihikokuroda
akihikokuroda marked this pull request as draft July 27, 2026 18:26
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
@akihikokuroda

Copy link
Copy Markdown
Contributor Author

Prototyped the "we could force tool calls to require an object identifier as a parameter"

It hits an issue and here is the analysis.

The issue is that only ONE tool is in the tools dict (['query']), so the LLM doesn't know there are two different "query" tools from different components. It just sees one tool and calls it.

This reveals a fundamental problem with the parameter approach when tools have duplicate names:

  • ✅ Works well when you have one component per tool name
  • ❌ Breaks down when multiple components have the same tool name because only the last one survives in the dict

With multiple components having the same tool name, we need a different strategy:

  1. Either keep them both in the dict using a different key structure (like prefixing)
  2. Or provide a way for the LLM to differentiate between them (maybe through better prompting or tool descriptions)

The parameter approach I implemented assumes one-to-one tool names. To truly support multiple components with identical tool names, we'd need either:

  • The prefix approach (component_xxx__query, component_yyy__query)
  • Or a list-based tools structure instead of a dict

Here is the list-based implementation analysis:

Tool Structure: Where It's Defined and Used

Current Structure

1. Definition

File: mellea/core/base.py:1710

class TemplateRepresentation:
    tools: dict[str, AbstractMelleaTool] | None = None

Type: dict[str, AbstractMelleaTool]

  • Keys: Tool name (string)
  • Values: AbstractMelleaTool instance
  • Problem: Dict only allows ONE tool per name

2. Tool Extraction

File: mellea/backends/tools.py:367

def add_tools_from_context_actions(
    tools_dict: dict[str, AbstractMelleaTool],
    ctx_actions: list[Component | CBlock | ModelOutputThunk] | None,
):
    for action in ctx_actions:
        tr = action.format_for_llm()
        for tool_name, func in tr.tools.items():
            wrapped_tool = wrap_tool_with_component_id(func, component_id)
            tools_dict[tool_name] = wrapped_tool  # ← OVERWRITES duplicates!

3. Tool Usage

File: mellea/backends/tools.py:430

def convert_tools_to_json(tools: dict[str, AbstractMelleaTool]) -> list[dict]:
    return [t.as_json_tool for t in tools.values()]

Converts dict values to JSON array for LLM.

4. Backend Tool Passing

Example: mellea/backends/ollama.py:496

chat_response = self._async_client.chat(
    model=self._model_id,
    messages=conversation,
    tools=[t.as_json_tool for t in tools.values()],  # ← Dict values to list
    ...
)

The Problem with Dict Structure

When multiple components have the same tool name:

  1. First component's tool: tools_dict["query"] = tool_1
  2. Second component's tool: tools_dict["query"] = tool_2OVERWRITES tool_1
  3. Result: Only tool_2 exists in the dict

Solutions

Option A: Keep Dict, Add Prefixes (Current PR #1432 approach)

tools_dict["component_abc123__query"] = tool_1
tools_dict["component_def456__query"] = tool_2

Pros: Works, but tool names become verbose
Cons: Breaks with specific prompts (MObject issue)

Option B: Change to List-Based Structure

@dataclass
class TemplateRepresentation:
    tools: list[tuple[str, AbstractMelleaTool]] | None = None

Pros: Preserves all tools, natural names + parameters work
Cons: Breaks all existing code using tools dict

Option C: Dict with Composite Keys

tools_dict[(component_id, "query")] = tool

Pros: No overwrites, clean structure
Cons: Changes API, breaks backwards compat

Option D: Dict of Lists

tools_dict["query"] = [tool_1, tool_2, ...]

Pros: Simple, preserves all tools
Cons: Requires changes to tool extraction and usage

Files That Need Changes

If changing to list-based (Option B):

  1. mellea/core/base.py - TemplateRepresentation.tools type
  2. mellea/backends/tools.py - add_tools_from_context_actions()
  3. mellea/backends/tools.py - convert_tools_to_json()
  4. All backends (ollama, openai, litellm, etc.)
  5. Any code iterating over tools dict

Current Parameter Approach Impact:

  • ✅ Works with single tool per name
  • ❌ Fails with multiple components having same tool name (dict collision)
  • ✅ component_id parameter IS added correctly
  • ✅ Works for MObject (single component)

Recommended Fix

To make parameter approach work with multiple tools:
Change tools structure from dict to list to preserve all tools, then use component_id parameters to route correctly.

@jakelorocco

Copy link
Copy Markdown
Contributor

Makes sense; I was just trying to describe that there are multiple ways to do something like this. And this is a big enough change that we should test out multiple options and ensure the chosen approach works in many different scenarios.

We don't quite have a process for that right now; but moving this to a discussion might help. We can also probably figure out what performance metrics we are looking for with a change like this.

@akihikokuroda

Copy link
Copy Markdown
Contributor Author

Discussion (#1455) is created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

allow tools to be renamed

4 participants