refactor(etl-uvicorn): bind /invoke envelope without body replay - #75
Conversation
…endency, drop the sealed-settings opt-in The body-replay middleware is replaced by bind_invocation_envelope, a FastAPI dependency that reads the framework's Starlette-cached body parse, so the /invoke body is buffered and decoded exactly once — no receive replay, no second copy of a large body. The request-size cap stays below the framework as InvokeBodyLimitMiddleware, a streaming byte counter that never buffers. /metadata now advertises invoke_with_sealed_dag_node_settings unconditionally: sealed per-invoke settings are the platform's required settings path, so the wrap_in_fastapi/generate_fast_api parameter and --sealed-dag-node-settings CLI flag are removed.
136431f to
30b0112
Compare
There was a problem hiding this comment.
1 issue found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="unstructured_platform_plugins/invocation_settings.py">
<violation number="1" location="unstructured_platform_plugins/invocation_settings.py:248">
P1: Streaming `/invoke` plugins lose the reserved invocation bindings on the repository's locked FastAPI 0.117.1. This `yield` dependency resets `_INVOCATION` when the path operation returns, but `wrap_in_fastapi` only starts iterating an async-generator plugin after that point while sending its `StreamingResponse`; the plugin therefore sees `current_invocation_settings()` and `current_invocation_context()` as `None`. Keeping the context around the stream (and adding a streaming binding test), or requiring FastAPI >= 0.118.0 and updating the lock, would preserve sealed settings and request identity for streaming plugins.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
Re-trigger cubic
| ) from exc | ||
|
|
||
| with invocation_envelope(invocation_settings, invocation_context): | ||
| yield |
There was a problem hiding this comment.
P1: Streaming /invoke plugins lose the reserved invocation bindings on the repository's locked FastAPI 0.117.1. This yield dependency resets _INVOCATION when the path operation returns, but wrap_in_fastapi only starts iterating an async-generator plugin after that point while sending its StreamingResponse; the plugin therefore sees current_invocation_settings() and current_invocation_context() as None. Keeping the context around the stream (and adding a streaming binding test), or requiring FastAPI >= 0.118.0 and updating the lock, would preserve sealed settings and request identity for streaming plugins.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At unstructured_platform_plugins/invocation_settings.py, line 248:
<comment>Streaming `/invoke` plugins lose the reserved invocation bindings on the repository's locked FastAPI 0.117.1. This `yield` dependency resets `_INVOCATION` when the path operation returns, but `wrap_in_fastapi` only starts iterating an async-generator plugin after that point while sending its `StreamingResponse`; the plugin therefore sees `current_invocation_settings()` and `current_invocation_context()` as `None`. Keeping the context around the stream (and adding a streaming binding test), or requiring FastAPI >= 0.118.0 and updating the lock, would preserve sealed settings and request identity for streaming plugins.</comment>
<file context>
@@ -147,6 +156,98 @@ async def plugin_metadata() -> dict:
+ ) from exc
+
+ with invocation_envelope(invocation_settings, invocation_context):
+ yield
+
+
</file context>
There was a problem hiding this comment.
1 issue found across 6 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="unstructured_platform_plugins/invocation_settings.py">
<violation number="1" location="unstructured_platform_plugins/invocation_settings.py:135">
P1: Default `wrap_in_fastapi` plugins no longer advertise sealed-settings support, so the controller will not send per-invoke sealed settings and handlers can fall back to boot-time state. Always include `invoke_with_sealed_dag_node_settings` as required by this PR's no-opt-out contract.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
Re-trigger cubic
| capabilities = [RESERVED_ENVELOPE_KEY, RESERVED_CONTEXT_KEY] | ||
| if invoke_with_sealed_dag_node_settings: |
There was a problem hiding this comment.
P1: Default wrap_in_fastapi plugins no longer advertise sealed-settings support, so the controller will not send per-invoke sealed settings and handlers can fall back to boot-time state. Always include invoke_with_sealed_dag_node_settings as required by this PR's no-opt-out contract.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At unstructured_platform_plugins/invocation_settings.py, line 135:
<comment>Default `wrap_in_fastapi` plugins no longer advertise sealed-settings support, so the controller will not send per-invoke sealed settings and handlers can fall back to boot-time state. Always include `invoke_with_sealed_dag_node_settings` as required by this PR's no-opt-out contract.</comment>
<file context>
@@ -114,33 +112,33 @@ def invocation_envelope(
is registered once. A host wrapper may register at app construction and a plugin can still
re-register with its own identifier afterwards, with no route-order dependence.
"""
+ capabilities = [RESERVED_ENVELOPE_KEY, RESERVED_CONTEXT_KEY]
+ if invoke_with_sealed_dag_node_settings:
+ capabilities.append(INVOKE_WITH_SEALED_DAG_NODE_SETTINGS_CAPABILITY)
</file context>
Summary
/invokebody buffering and replay with a FastAPI dependency that reads Starlette's cached JSON parse.InvokeBodyLimitMiddlewarebelow FastAPI as a streaming byte counter, so oversized bodies are rejected without a second buffer.POST /invokeis registered; remove privateroute.dependantmutation.invoke_with_sealed_dag_node_settingscapability opt-in and its wrapper, generator, and CLI arguments.invocation_settings.dag_node_settings; a bare envelope fails closed.Why
ASGI middleware has to consume the raw receive channel before FastAPI can parse it, which forced the old implementation to buffer, parse, and replay the request. Route-level extraction shares the framework's body and JSON caches instead.
The initial dependency version still had three correctness problems: FastAPI 0.117.1 closed yield dependencies before
StreamingResponseiteration, installation mutated FastAPI's private dependency graph after route registration, and unconditional sealed-capability advertisement conflated transport support with handler consumption. This revision fixes all three without raising the FastAPI floor.Impact
install_invocation_envelope(app)must run beforePOST /invokeis registered. It may run after unrelated routes such as/metadata, which preserves the hand-written plugin integration order.POST /invoke, including mixed-method routes and rooted deployments.Validation
pytest -qon FastAPI 0.117.1 / Starlette 0.48.0 with the current utic-invocation-settings 0.4.0 branch — 153 passed.ruff check .— clean.ruff format --checkon changed Python files — clean.git diff --check— clean.