feat: batched call_tools() for parallel upstream calls in the code-execution sandbox

.github/workflows/unit-tests.yml

@@ -123,7 +123,7 @@ jobs:
       - name: Run unit tests (Windows)
         if: matrix.os == 'windows-latest'
         shell: pwsh
-        run: go test -v -race -timeout 5m '-coverprofile=coverage.out' -covermode=atomic '-skip=E2E|Binary|MCPProtocol|TestInfoEndpoint|TestGracefulShutdownNoPanic|TestSocketInfoEndpoint' ./...
+        run: go test -v -race -timeout 10m '-coverprofile=coverage.out' -covermode=atomic '-skip=E2E|Binary|MCPProtocol|TestInfoEndpoint|TestGracefulShutdownNoPanic|TestSocketInfoEndpoint' ./...
 
       - name: Run unit tests (Unix)
         if: matrix.os != 'windows-latest'

CLAUDE.md

@@ -158,6 +158,6 @@ tail -f ~/Library/Logs/mcpproxy/main.log  # main log (macOS; Linux: ~/.mcpproxy/
 - **Windows installer**: [docs/github-actions-windows-wix-research.md](docs/github-actions-windows-wix-research.md). **Prerelease** (`next` branch + `v*-rc.*` tags, opt-in, off stable channels): [docs/prerelease-builds.md](docs/prerelease-builds.md).
 
 ## Recent Changes
+- 096-batched-call-tools: Added Go 1.24 module toolchain (repo builds with local Go 1.25) + existing only — goja (sandbox), mark3labs/mcp-go (tool surface), zap. **No new dependencies.**
 - 095-update-failure-ux: Added Swift 5.9 (tray, AppKit + Sparkle 2.9.3 vendored via SwiftPM) · Go 1.24 module toolchain (repo builds with local Go 1.25) + existing only — Sparkle 2.9.3 (`SPUUpdater`, `SPUStandardUserDriver`), chi (httpapi), bbolt (diagnostics counters), swaggo/swag v2 (contract regen). **No new dependencies.**
 - 094-filter-diagnostics: Added Go 1.24 (module toolchain; repo builds with local Go 1.25) + existing only — `mark3labs/mcp-go` (tool registration), stdlib `encoding/json`. No new dependencies.
-- 091-connect-client-form: Added Swift 5.9 (SwiftUI sheet + AppKit menu) + Go 1.25 (core) + existing `internal/connect` package (registry, preview, connect, undo, backup), `internal/httpapi/connect.go` routes; Swift `APIClient` over Unix socket (admin context)

ROADMAP.md

@@ -797,3 +797,4 @@ Legend: `shipped` ≥95% checked · `in-flight` 1–94% · `drafted` 0% · `—`
 | [093-concurrency-limits](./specs/093-concurrency-limits/) | — | — |
 | [094-filter-diagnostics](./specs/094-filter-diagnostics/) | `shipped` | 14/14 (100%) |
 | [095-update-failure-ux](./specs/095-update-failure-ux/) | `shipped` | 28/28 (100%) |
+| [096-batched-call-tools](./specs/096-batched-call-tools/) | `in-flight` | 15/16 (94%) |

docs/code_execution/api-reference.md

@@ -275,6 +275,80 @@ if (!res.ok) {
 var data = res.result;
 ```
 
+#### `call_tools(requests, options)`
+
+Calls **independent** upstream MCP tools in parallel and returns one result slot
+per request, in input order.
+
+**Parameters**:
+- `requests` (array, required): Up to 100 elements of `{server, tool, args}`.
+  `server` and `tool` are non-empty strings; `args` is optional and defaults to `{}`.
+- `options` (object, optional): `{max_parallel}` — integer 1-32, defaults to the
+  configured `code_execution_max_parallel` (8). Unknown keys are ignored.
+
+**Returns**: An array with `slots.length === requests.length`, where each slot is
+the same envelope `call_tool()` returns:
+
+```javascript
+// slots[i] for a successful requests[i]
+{
+  "ok": true,
+  "result": <tool result>
+}
+
+// slots[i] for a failed requests[i]
+{
+  "ok": false,
+  "error": {
+    "message": "<error message>",
+    "code": "<error code>"
+  }
+}
+```
+
+**Example**:
+```javascript
+var slots = call_tools(
+  [1, 2, 3, 4, 5].map(function (n) {
+    return {server: 'github', tool: 'get_pull_request',
+            args: {owner: 'acme', repo: 'api', pullNumber: n}};
+  }),
+  {max_parallel: 5}
+);
+
+var titles = slots.map(function (r) {
+  if (!r.ok) { return 'ERR: ' + r.error.code; }
+  return JSON.parse(r.result.content[0].text).title;
+});
+({titles: titles});
+```
+
+**Semantics**:
+- Per-element enforcement matches a lone `call_tool()`: the same gates, the same
+  error codes, the same activity records. One failing element never affects its
+  siblings.
+- Each element costs one unit of `max_tool_calls`, checked in input order before
+  anything is dispatched.
+- Concurrency never exceeds the effective `max_parallel`, and per-server
+  concurrency limits still apply inside the call path.
+- The whole batch runs inside the execution timeout; a timeout cancels in-flight
+  elements.
+- `call_tools([])` returns `[]` and costs nothing. Like `call_tool()`, the
+  function is **synchronous** — do not use `await`.
+
+**Whole-call errors**: a malformed call returns a **single** envelope (not an
+array) and dispatches nothing:
+
+```javascript
+{ok: false, error: {code: "INVALID_ARGS", message: "call_tools: element 3: ..."}}
+```
+
+This happens when `requests` is not an array, an element is not an object with
+non-empty `server`/`tool` strings, a supplied `args` is not an object, the array
+has a sparse hole, `options` is not an object, `max_parallel` is not an integer
+in 1-32, or the batch exceeds 100 elements. The message names the first
+offending element index.
+
 ### Available JavaScript Features
 
 #### JavaScript Standard Library (ES2020+)
@@ -330,6 +404,7 @@ var isObject = typeof value === 'object' && value !== null;
 | `MAX_TOOL_CALLS_EXCEEDED` | Tool call limit exceeded | Code called `call_tool()` more than `max_tool_calls` times | Reduce tool calls, increase limit, or use pagination |
 | `SERVER_NOT_ALLOWED` | Server not in allowed list | Attempted to call server not in `allowed_servers` | Add server to allowed list or remove restriction |
 | `SERIALIZATION_ERROR` | Result not JSON-serializable | Return value contains functions, circular refs, etc. | Return only plain objects, arrays, primitives |
+| `INVALID_ARGS` | Host function called with arguments it cannot interpret | Wrong arity for `call_tool()`, or a malformed `call_tools()` batch (bad element shape, bad `max_parallel`, >100 elements) | Fix the offending argument — the message names the first offending element index |
 
 ### Error Examples
 
@@ -463,7 +538,8 @@ Edit `~/.mcpproxy/mcp_config.json`:
   "enable_code_execution": false,
   "code_execution_timeout_ms": 120000,
   "code_execution_max_tool_calls": 0,
-  "code_execution_pool_size": 10
+  "code_execution_pool_size": 10,
+  "code_execution_max_parallel": 8
 }
 ```
 
@@ -475,6 +551,7 @@ Edit `~/.mcpproxy/mcp_config.json`:
 | `code_execution_timeout_ms` | number | `120000` | Default timeout in milliseconds (range: 1-600000) |
 | `code_execution_max_tool_calls` | number | `0` | Default max tool calls (0 = unlimited) |
 | `code_execution_pool_size` | number | `10` | Number of JavaScript VM instances in pool (range: 1-100) |
+| `code_execution_max_parallel` | number | `8` | Default concurrency for `call_tools()` batches (range: 1-32). Hot-reloaded; applies to executions started after the change |
 
 ### Per-Request Overrides
 
@@ -493,6 +570,10 @@ Per-request options override global configuration:
 
 **Priority**: Request options > Global config > Built-in defaults
 
+`max_parallel` is deliberately **not** a request option: batch concurrency is
+overridden inside the script, per batch, with `call_tools(requests, {max_parallel})`.
+Its priority is per-batch override > `code_execution_max_parallel` > built-in 8.
+
 ---
 
 ## CLI Reference
@@ -628,6 +709,15 @@ mcpproxy code exec --file=/tmp/script.js
 - **max_tool_calls**: Must be >= 0
 - **allowed_servers**: Must be array of strings (server names)
 
+### `call_tools()` Batch Validation
+
+- **requests**: Must be a dense array of at most 100 elements
+- **element**: Must be an object with non-empty `server` and `tool` strings; a
+  supplied `args` must be an object (omitted = `{}`)
+- **options.max_parallel**: Must be an integer between 1 and 32
+- A violation returns one `INVALID_ARGS` envelope naming the first offending
+  index; no element is dispatched and no budget is consumed
+
 ### Return Value Validation
 
 **Valid return values**:
@@ -655,6 +745,20 @@ The pool size determines how many concurrent executions can run simultaneously:
 
 **Recommendation**: Start with default (10) and adjust based on load.
 
+### Batch Concurrency (`call_tools`)
+
+`code_execution_max_parallel` (default 8) bounds how many elements of one batch
+run at once; `call_tools(requests, {max_parallel})` overrides it per batch
+(1-32). A batch of N independent calls costs roughly `ceil(N / max_parallel) ×
+slowest-call` instead of the sum of all calls.
+
+**Interaction with per-server limits**: Spec 093 concurrency limits are enforced
+inside the call path and are never bypassed by batching. A server with
+`max_concurrent_requests: 1` and `queue_size: 9` serializes a 10-element batch;
+the same server with **no** `queue_size` sheds the overflow, returning 1 result
+and 9 per-slot `queue_full` errors. Give limited servers `queue_size` headroom —
+or lower `max_parallel` to match their cap — before fanning out against them.
+
 ### Timeout Settings
 
 | Use Case | Recommended Timeout |

docs/code_execution/cookbook.md

@@ -19,7 +19,9 @@ annotations and omit `language`.
 > **New to code execution?** Read [overview.md](overview.md) first, then the
 > [api-reference.md](api-reference.md) for the full tool schema. This cookbook
 > assumes you know that `call_tool(server, tool, args)` returns
-> `{ ok: true, result }` or `{ ok: false, error }`, and that the script's
+> `{ ok: true, result }` or `{ ok: false, error }`, that
+> `call_tools(requests, options)` returns one such envelope per request in input
+> order, and that the script's
 > **last expression** becomes the result `value`. A bare top‑level `return`
 > is a **SyntaxError** (`Illegal return statement`) — `return` is only legal
 > inside a function. To early‑exit, wrap the body in an IIFE (see the
@@ -64,22 +66,35 @@ the #1 source of surprises:
 
 | Capability | Status | Implication for recipes |
 |------------|--------|-------------------------|
-| `call_tool(server, tool, args)` | ✅ | The only way to reach upstream tools. Synchronous — returns when the tool responds. |
+| `call_tool(server, tool, args)` | ✅ | Reaches one upstream tool. Synchronous — returns when the tool responds. |
+| `call_tools(requests, options)` | ✅ | Reaches **independent** upstream tools in parallel: an array of ≤100 `{server, tool, args}` objects in, one `{ok, result}` / `{ok, error}` slot per request out, in input order. Also synchronous. |
 | `input` global | ✅ | Your parameters. Type it with `as` or an `interface` for IDE‑grade safety. |
 | ES2020+ stdlib (`map`/`filter`/`reduce`, `JSON`, `Math`, `Date`) | ✅ | Use it freely for transforms and aggregation. |
 | `console.log` | ✅ | Goes to **server logs**, not the result. Use for debugging. |
 | Top‑level `return` | ❌ | `return` outside a function is a **SyntaxError** (`Illegal return statement`). The result is the script's **last expression**. To early‑exit, wrap the body in an IIFE: `(() => { … return x; })()`. |
 | `setTimeout` / `setInterval` | ❌ | **No wall‑clock sleep.** "Backoff" and "rate‑limit" recipes work by *bounding* and *chunking*, never by sleeping. |
 | `require` / `import` / `fetch` / `fs` | ❌ | No modules, no network, no filesystem. All I/O goes through `call_tool`. |
-| Concurrency | ❌ (sequential) | Tool calls run **one at a time** server‑side. "Fan‑out" saves *round‑trips*, not wall‑clock from parallelism. Be honest about this when estimating latency. |
+| Concurrency | ✅ only via `call_tools` | Loops of `call_tool` run **one at a time**; a `call_tools` batch runs up to `max_parallel` elements at once (default `code_execution_max_parallel`, 8). Sequential loops save *round‑trips*; batches also save wall‑clock. |
 
-Two control knobs you will reach for constantly (set in `options`):
+Control knobs you will reach for constantly (set in `options`):
 
 - `max_tool_calls` — a hard ceiling that aborts the script with
   `MAX_TOOL_CALLS_EXCEEDED`. Always set it on loops so a bad `input` can't fan
-  out unbounded.
+  out unbounded. Each `call_tools` element counts as one call.
 - `timeout_ms` — wall‑clock budget (default 120 000, max 600 000). The
-  transpile step counts toward it (negligibly).
+  transpile step counts toward it (negligibly), and a whole `call_tools` batch
+  lives inside it.
+
+Batch concurrency is **not** an `options` field: it comes from the
+`code_execution_max_parallel` config key (default 8) and is overridden per batch
+with `call_tools(requests, {max_parallel})` (1–32).
+
+> **Check the target server's limits before you fan out.** Per‑server
+> [concurrency limits](../configuration.md#concurrency-limits--request-queueing)
+> still apply and are never bypassed: a server with `max_concurrent_requests: 1`
+> and `queue_size: 9` serializes a 10‑element batch, while the same server with
+> **no** `queue_size` sheds the overflow as nine per‑slot `queue_full` errors.
+> Set `queue_size` headroom, or match `max_parallel` to the cap.
 
 ---
 
@@ -110,6 +125,34 @@ read the previous result before issuing the next.
 **Guardrail:** set `options.max_tool_calls` to `usernames.length` (or a sane
 cap) so an oversized input can't run away.
 
+**Parallel variant:** the lookups are independent, so `call_tools` turns the
+sequential loop into one bounded fan‑out — the batch takes about as long as its
+slowest element instead of the sum:
+
+```typescript
+// language: "typescript"
+// input: { "usernames": ["octocat", "torvalds", "gaearon"] }
+const slots = call_tools(
+  (input.usernames as string[]).map((login: string) => ({
+    server: "github", tool: "get_user", args: { username: login },
+  })),
+  { max_parallel: 5 },
+);
+
+const users = slots.map((res: any, i: number) => {
+  const login = (input.usernames as string[])[i];
+  if (!res.ok) return { login, error: res.error.message };
+  const u = res.result as User;
+  return { login, name: u.name, followers: u.followers };
+});
+
+({ users, count: users.length });
+```
+
+Slots come back in input order (`slots[i]` ↔ `requests[i]`), one failing element
+never poisons the rest, and a batch is capped at 100 elements — chunk longer
+lists into several `call_tools` calls.
+
 ---
 
 ## Recipe 2 — Fan‑out + merge (many tools, one object)
@@ -136,8 +179,30 @@ const ci     = call_tool("ci",     "latest_pipeline", { project: input.repo });
 **Replaces:** 3 round‑trips + a final model turn to stitch the pieces together.
 Here the merge happens server‑side; the model sees one tidy object.
 
-**Note on "parallel":** the three calls run sequentially in the sandbox. The win
-is collapsing 4 model turns into 1 — not parallel network I/O.
+**Note on "parallel":** written this way the three calls run sequentially. Since
+none of them depends on another, `call_tools` runs them at once and the merge
+reads the slots by position:
+
+```typescript
+// language: "typescript"
+// input: { "repo": "octocat/Hello-World" }
+const [owner, name] = (input.repo as string).split("/");
+
+const [repo, issues, ci] = call_tools([
+  { server: "github", tool: "get_repo",        args: { owner, repo: name } },
+  { server: "github", tool: "list_issues",     args: { owner, repo: name, state: "open" } },
+  { server: "ci",     tool: "latest_pipeline", args: { project: input.repo } },
+]) as any[];
+
+({
+  repo:       repo.ok   ? { stars: repo.result.stargazers_count } : { error: repo.error.message },
+  openIssues: issues.ok ? issues.result.length                    : null,
+  ci:         ci.ok     ? ci.result.status                        : "unknown",
+});
+```
+
+Now the win is both: 4 model turns collapse into 1, **and** the dashboard costs
+one slow call instead of three.
 
 ---
 
@@ -362,6 +427,12 @@ the per‑script call count with `max_tool_calls` and let the agent resume acros
 turns. (See [troubleshooting.md](troubleshooting.md) for the
 `setTimeout`‑is‑unavailable rationale.)
 
+**With `call_tools`:** keep `max_parallel` at or below the server's
+`max_concurrent_requests` — a batch wider than the cap does not go faster, it
+just queues (or, with no `queue_size`, sheds the overflow into per‑slot
+`queue_full` errors). The proxy‑side cap is the durable fix; `max_parallel` is
+the script‑side courtesy.
+
 **Replaces:** 100 individual round‑trips with ~10 bulk calls in one script.
 
 ---
@@ -432,11 +503,15 @@ wall‑clock. An N‑step orchestration costs:
 
 Each eliminated round‑trip removes a full model turn — its latency, its
 generated tool‑call JSON, and its re‑reading of the intermediate result. For a
-5‑step recipe that is roughly a **5×** reduction in model turns. Server‑side the
-N tool calls still run sequentially (the sandbox is single‑threaded — see [the
-sandbox contract](#the-sandbox-contract-read-this-first)), so `code_execution`
-optimizes the *agent loop*, not raw upstream I/O. Quote round‑trip savings, not
-parallelism, when you describe the win.
+5‑step recipe that is roughly a **5×** reduction in model turns.
+
+Server‑side, sequential `call_tool` steps still run one at a time (the sandbox is
+single‑threaded — see [the sandbox contract](#the-sandbox-contract-read-this-first)),
+so a chained pipeline optimizes the *agent loop*, not raw upstream I/O. Where the
+steps are **independent**, `call_tools` also cuts server‑side wall‑clock: N calls
+cost about `ceil(N / max_parallel) × slowest-call` instead of their sum. Quote
+parallelism only for `call_tools` batches; for chained recipes, quote round‑trip
+savings.
 
 ---
 
@@ -456,7 +531,8 @@ deliberate opt‑in):
   "enable_code_execution": true,
   "code_execution_timeout_ms": 120000,
   "code_execution_max_tool_calls": 0,
-  "code_execution_pool_size": 10
+  "code_execution_pool_size": 10,
+  "code_execution_max_parallel": 8
 }
 ```