fix: change tool_calls from dict to list - #1435
Conversation
|
This has broader changes than expected. |
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
jakelorocco
left a comment
There was a problem hiding this comment.
Thank you! Looks mostly good. A few minor things I noticed.
Can you also search through our documentation for these uses? I believe examples like this will need to be modified as well:
response = m.instruct(
"Use the code interpreter tool to compute 7 factorial.",
requirements=[uses_tool("python")],
model_options={ModelOption.TOOLS: [tool]},
tool_calls=True,
)
# tool_calls=True makes .tool_calls available on the result
code = response.tool_calls["python"].args["code"]
exec_result = response.tool_calls["python"].call_func()
print(exec_result)
| assert isinstance(model_tool_calls, list), ( | ||
| f"Expected list[ModelToolCall], got {type(model_tool_calls)}" | ||
| ) |
There was a problem hiding this comment.
Same as prior; must already be a list here.
There was a problem hiding this comment.
Removing assertion
| tool_chunk = extract_model_tool_requests(tools, choice_response) | ||
| if tool_chunk is not None: | ||
| if mot.tool_calls is None: | ||
| mot.tool_calls = {} | ||
| # Merge the tool_chunk dict. | ||
| for key, val in tool_chunk.items(): | ||
| mot.tool_calls[key] = val | ||
| if not isinstance(tool_chunk, list): | ||
| MelleaLogger.get_logger().error( | ||
| f"extract_model_tool_requests returned {type(tool_chunk).__name__} " | ||
| f"instead of list[ModelToolCall]" | ||
| ) |
There was a problem hiding this comment.
If this isn't None, do we need to check that it's a list? Doesn't our type system / the extract_model_tool_requests function ensure it's a list?
There was a problem hiding this comment.
Yes, it is. Removing check.
| assert isinstance(model_tool_calls, list), ( | ||
| f"Expected list[ModelToolCall], got {type(model_tool_calls)}" | ||
| ) |
There was a problem hiding this comment.
I don't think it's possible for this to not be a list here.
There was a problem hiding this comment.
Removing assertion
| @@ -806,38 +811,50 @@ def _extract_model_tool_requests( | |||
| self, | |||
| tools: dict[str, AbstractMelleaTool], | |||
| chat_response: litellm.ModelResponse, # type: ignore | |||
| ) -> dict[str, ModelToolCall] | None: | |||
| model_tool_calls: dict[str, ModelToolCall] = {} | |||
| ) -> list[ModelToolCall] | None: | |||
| model_tool_calls: list[ModelToolCall] = [] | |||
There was a problem hiding this comment.
Are we actually using this function anywhere? It looks like this got missed in a refactor.
There was a problem hiding this comment.
No, it is not. Removing it.
| assert isinstance(model_tool_calls, list), ( | ||
| f"Expected list[ModelToolCall], got {type(model_tool_calls)}" | ||
| ) |
There was a problem hiding this comment.
Same as above. I think this must be a list already.
There was a problem hiding this comment.
Removing check.
| if not isinstance(tool_chunk, list): | ||
| MelleaLogger.get_logger().error( | ||
| f"extract_model_tool_requests returned {type(tool_chunk).__name__} " | ||
| f"instead of list[ModelToolCall]" | ||
| ) |
There was a problem hiding this comment.
Removing check.
| are silently overwritten). For duplicate tool names, use component ID-based prefixing to ensure | ||
| unique names across components. |
There was a problem hiding this comment.
Can we leave out the component ID based prefixing for now?
There was a problem hiding this comment.
Agreed here, I don't think we have component id prefixing?
| if not isinstance(tool_chunk, list): | ||
| MelleaLogger.get_logger().error( | ||
| f"extract_model_tool_requests returned {type(tool_chunk).__name__} " | ||
| f"instead of list[ModelToolCall]" | ||
| ) |
There was a problem hiding this comment.
Removing check.
| def _extract_model_tool_requests( | ||
| self, tools: dict[str, AbstractMelleaTool], chat_response: dict | ||
| ) -> dict[str, ModelToolCall] | None: | ||
| model_tool_calls: dict[str, ModelToolCall] = {} | ||
| ) -> list[ModelToolCall] | None: | ||
| model_tool_calls: list[ModelToolCall] = [] |
There was a problem hiding this comment.
I also think this function isn't getting used anywhere?
There was a problem hiding this comment.
Correct. Removing.
| assert isinstance(model_tool_calls, list), ( | ||
| f"Expected list[ModelToolCall], got {type(model_tool_calls)}" | ||
| ) |
There was a problem hiding this comment.
Same as previous comments.
There was a problem hiding this comment.
Removing assertion
ajbozarth
left a comment
There was a problem hiding this comment.
Some feedback from Claude — core dict→list conversion looks correct and complete. Agree with the cleanup items already in the existing review (redundant asserts, unreachable list-guards); not re-posting to avoid duplicate threads. Two test notes to add:
- No execution-level test for the fix. The added test proves two same-name calls survive extraction, but nothing runs them through
_acall_toolsto confirm both execute and produce twoToolMessages — the behavior #1431 is about. - A round-trip test worth front-loading. Result-turn correlation is #1389's territory and correctly untouched here, but a test now exercising the full tool→model→toolcall→model loop for parallel same-name calls (asserting today's behavior) gives #1389 a regression guard already in place.
AngeloDanducci
left a comment
There was a problem hiding this comment.
I think Jake's comments covered everything I found, overall looks good pending the feedback cycle.
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
|
@ajbozarth a test for "No execution-level test for the fix" is added. |
ajbozarth
left a comment
There was a problem hiding this comment.
Re-review from Claude — thanks for the updates. The cleanup items and the execution-level test are all addressed. One marker fix inline; the fix itself looks good.
| from mellea.core.base import ModelOutputThunk, ModelToolCall | ||
| from mellea.stdlib.functional import _acall_tools | ||
|
|
||
| pytestmark = [pytest.mark.ollama, pytest.mark.e2e] |
There was a problem hiding this comment.
These are integration tests, not e2e/ollama. _acall_tools only touches the backend for formatter.print(), never for inference — they construct ModelToolCalls directly with local Python functions, so no model runs (they pass in ~0.2s with no Ollama server). Per test/README.md, wiring multiple real project components together without external I/O is integration (no backend marker, and not skipped when Ollama is absent) — right now the ollama marker skips this regression guard in every lane without a running Ollama. Suggest pytestmark = [pytest.mark.integration]. The backend fixture (L26) is only used for .formatter.print(), so a real OllamaModelBackend isn't required either.
There was a problem hiding this comment.
Made changes.
| automatic tool execution loop through _acall_tools() correctly processes | ||
| multiple same-name tool calls from the list-based tool_calls structure. | ||
|
|
||
| Unlike test_parallel_same_name_tool_execution.py which manually calls |
There was a problem hiding this comment.
This references test_parallel_same_name_tool_execution.py, which doesn't exist in the tree — dangling reference. Worth removing or pointing at the actual extraction test.
There was a problem hiding this comment.
Made changes, too. Thanks!
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Pull Request
Issue
Fixes #1431
Description
This changed tool_calls from dict to list instead of set because of the follow reason
__hash__/__eq__Testing
Attribution
Adding a new component, requirement, sampling strategy, or tool?
If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.
NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.