feat: batched call_tools() for parallel upstream calls in the code-execution sandbox - #989
Merged
Merged
Conversation
Deploying mcpproxy-docs with
|
| Latest commit: |
51ee14b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://4f4f9765.mcpproxy-docs.pages.dev |
| Branch Preview URL: | https://096-batched-call-tools.mcpproxy-docs.pages.dev |
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…ream calls in the code-exec sandbox Related #987
…, cancellation, parity-as-today semantics Related #987 Resolves all 11 cross-model review findings: execution-context cancellation for batch workers, parity defined as call_tool's current enforcement (gate expansion out of scope), full max_parallel definition (code_execution_max_parallel, default 8, range 1-32, per-call override), input-order pre-dispatch checks so budget cannot race, envelope-based malformed-call rejection, batch cap 100, timeout semantics reconciled with SC-003, dropped the incorrect ES5.1 claim, stub-based reproducible SCs.
… quickstart Related #987 Phase 0/1 artifacts. Key design decisions from code research: workers produce plain Go values with all goja conversion on the script goroutine after join; upstreamToolCaller is already concurrency-safe while ec.ToolCalls stays script-thread-only; Spec-093 admission applies automatically inside managed.Client.CallTool (queue-vs-shed corrected in the spec); execution ctx threaded via a new ExecutionContext field so workers cancel at the deadline; 7-point config wiring for code_execution_max_parallel plus two pre-existing hot-reload breaks (DetectConfigChanges clause, currentConfig() read) fixed as part of FR-004.
…n, one-record-per-dispatch, prefilled queue, no REST max_parallel Related #987
…ty; drop stale execution-level max_parallel plan lines Related #987
… stories + polish Related #987
…ecution sandbox Related #987 call_tools(requests, options?) dispatches independent upstream calls concurrently (bounded worker pool, prefilled closed channel) and returns per-slot {ok,result}|{ok,error} envelopes in input order. Workers produce plain Go values; all goja conversion happens on the script goroutine after an unconditional join. Per-element checks run in input order with lone-call parity (budget -> allow-list -> agent-scope -> permission-tier); every dispatched element yields exactly one ToolCallRecord, including under cancellation. Spec-093 per-server admission applies unchanged inside the call path. ## Changes - internal/jsruntime: call_tools binding, batch engine, ExecutionContext ctx threading (workers cancel with the execution), error-code constants, gate extraction; extensive batch_test.go incl. -race cancellation tests - internal/config: code_execution_max_parallel (default 8, range 1-32) - internal/runtime: DetectConfigChanges clause for CodeExecution* fields (pre-existing hot-reload break) - internal/server: defaults resolved via currentConfig() (second pre-existing hot-reload break), MaxParallel resolution, deduplicated code_execution descriptions into shared constants documenting call_tools - oas/frontend/docs: swagger regen, settings field, 7 docs files ## Testing - TDD throughout (red observed first; two invariants mutation-tested) - go test -race ./internal/... green; server-edition green; lint v2 0 issues; e2e 65/65
…elp states override semantics Related #987 Review findings: code_execution_pool_size is sized once at server construction and never resized in-process, so it now reports its own changed field with requires_restart instead of riding the hot-applied code_execution group; the settings help text now says a script can override (not merely lower) batch concurrency per call within 1-32.
…ut without synchronizing Related #987
The Unix step has no explicit -timeout (Go default 10m per package); the Windows step capped at 5m, which internal/runtime and internal/server now exceed on slow runners as the suites have grown. Two consecutive Windows failures on this branch were pure 'panic: test timed out after 5m0s' with different tests holding the bag each time.
Dumbris
force-pushed
the
096-batched-call-tools
branch
from
August 14, 2026 18:28
882a79a to
51ee14b
Compare
📦 Build ArtifactsWorkflow Run: View Run Available Artifacts
How to DownloadOption 1: GitHub Web UI (easiest)
Option 2: GitHub CLI gh run download 31828813455 --repo smart-mcp-proxy/mcpproxy-go
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements spec 096 (
specs/096-batched-call-tools/): acall_tools(requests, options?)host primitive in the code-execution JS sandbox that runs independent upstream calls in parallel.Related #987
What it does
N independent calls cost ~the slowest call instead of the sum. One failing element never poisons the batch — every slot resolves with a per-slot envelope.
Design (reviewed plan is binding —
specs/096-batched-call-tools/plan.md)vm.ToValuehappens on the script goroutine after an unconditional WaitGroup join.call_tool.max_tool_calls; every dispatched element yields exactly one ToolCallRecord, including under cancellation.min(max_parallel, dispatchable)workers; Spec-093 per-server admission applies unchanged insidemanaged.Client.CallTool(queue or shed per that server's config — never bypassed).ExecutionContext.ctx); lonecall_toolbehavior untouched.max_parallelrange 1–32; malformed calls return a singleINVALID_ARGSenvelope naming the first offending element — never a throw.Config
New
code_execution_max_parallel(int, default 8, range 1–32); precedence: per-batch override > config > 8. Full wiring: validation, post-load defaulting, swagger regen, frontend settings field, 7 docs files. Also fixes two pre-existing hot-reload breaks found during research:DetectConfigChangeshad no clause for anyCodeExecution*field (edits were silently swallowed), and exec defaults were resolved from the construction-time config snapshot instead ofcurrentConfig().code_execution_pool_sizeis now honestly reported as restart-required (the pool is sized once at startup).Review
Spec and plan each went through multi-round cross-model review (opencode gpt-5.6-sol; Codex is out of credits) — 11 spec findings and 7 plan findings resolved before implementation. The implementation diff went through a 9-agent adversarial review (3 lenses × verify-per-finding): 2 confirmed minors, both fixed (pool_size restart honesty, settings help text).
Testing
go test -race ./internal/...green (incl. cancellation tests under -race), server-edition green, lint v2 (CI config) 0 issues, e2e 65/65.