feat(tui): ingest image attachments through Runtime Host (#4171) - #4248
feat(tui): ingest image attachments through Runtime Host (#4171)#4248abhinav-phi wants to merge 2 commits into
Conversation
|
Thanks for taking on TUI image attachments — the user need is real, and reusing the Runtime Host Artifact authority is the right durable boundary. Before we patch the individual lifecycle findings, could we revisit the shape of the first slice? Desktop already keeps selected files or approval tokens local to the draft and performs Artifact ingestion inside the message-submit boundary. This PR instead ingests during That is not a second durable store, but it is a parallel draft lifecycle, and it appears to be the source of most of the 1,600-line change and the current concurrency/recovery findings. My suggestion would be to stage only a local image descriptor, ingest through the existing Host boundary at submit, and let the committed Message’s Please do push back if inline marker placement is itself a demonstrated product requirement rather than an implementation choice; that would materially change the smallest complete design. AI-assisted review disclosure: Codex assisted with comparing the TUI and Desktop attachment lifecycles; Astro-Han verified the production boundaries and owns this position. |
Add a self-contained TUI image-attachment ingestion path that mirrors the Desktop composer's shape: a /attach pick stages only a client-local descriptor, the message-submit boundary owns Artifact ingestion, and the committed Message's AttachmentRef is the durable identity. - new tui-attachments.ts: draft-scoped staging (local descriptors and retained refs) plus the client-side capped file reader - /attach <path> (TUI surface) and /detach <n>: stage and remove client- locally with MIME/count guards; staged images render as a separate composer strip beside the pending queues - name + media type only, never a local path, and never inside the editable draft text - submit boundary: staged descriptors are read (byte-capped) and ingested through the Runtime Host artifact.ingest authority (begin/chunk/commit, abort on failure) in staging order, then ride content.attachments on turn.message.submit / turn.start; Send stays enabled the whole time - failed dispatch or refusal keeps the already-committed refs staged as retained items, so a retry reuses the exact Artifacts instead of orphaning a second copy; a mid-batch ingest failure deletes only the freshly committed refs and leaves staging untouched - Alt+Up / interrupt retraction: retracted queue entries return their attachments and are restaged as retained refs - resubmission reuses the same Artifacts with no re-ingest - cleanup semantics: local descriptors drop with the draft (editor clear, session switch, /new, graceful exit) with nothing to clean; retained refs are deleted best-effort via artifact.delete - transcript rendering: stored user messages, steering echoes, and the transient row render committed attachments as name + MIME + size chips - never a local path - tests: staging unit semantics and TUI flows (stage-then-submit, exact- ref dispatch, ingest failure, dispatch retry, /detach, retraction reuse, session-switch cleanup + recovery rendering, authority and non-image refusals, count cap) plus driver-level artifact.ingest and artifact.delete wire coverage Refs apache#4171 Refs apache#4079 Refs apache#4080
623eff3 to
152426a
Compare
|
Thanks for the review, @Astro-Han — the diagnosis is correct, and I've reworked the PR along exactly the shape you suggested (pushed as 152426a, rebased onto current main). On your question first: inline marker placement was an implementation choice, not a demonstrated product requirement — I have no evidence users need What changed:
CI note: the failing All TUI flows from the issue are covered by tests in the new shape: stage-then-submit with exact refs, dispatch failure → retry reuses same Artifacts, ingest failure → nothing dispatched and nothing orphaned, /detach, retraction reuse, session-switch cleanup + recovery rendering, and the authority/non-image/count refusals. |
Astro-Han
left a comment
There was a problem hiding this comment.
Thanks for reworking this around the existing Runtime Host Artifact authority; the new shape is materially simpler than the previous design. I found one submit-ownership issue and several recovery/lifecycle gaps below. The exact-head required test check is also currently red in the changed CLI autocomplete surface, so I could not treat the current gate as passing. This was an AI-assisted review using Codex; I independently traced the production runner/driver paths and exact-head CI. These are suggestions from an outside read, so please push back if I have missed an enforced TUI invariant.
| // same Artifact instead of orphaning a second copy of the bytes. | ||
| const attachments: AttachmentRef[] = []; | ||
| const freshlyIngested: AttachmentRef[] = []; | ||
| for (const item of imageAttachments.list()) { |
There was a problem hiding this comment.
Thanks for moving ingestion to the submit boundary. [P1] (category ② — supported submit/session concurrency) This copies the current items but does not atomically claim them: while the loop awaits file reads and Host ingestion, /attach, /detach, a second Send, /session, /new, and close can still mutate or reset the shared staging object. The first attempt can therefore send an item the user detached, clear an item staged for the next message, duplicate one image across two messages, or ingest under one Session and submit under another because ingest and submit each resolve the live Session separately. Could Send synchronously take an attempt-owned batch plus Session generation, with later draft edits going to a fresh batch and only that exact batch being retired? Gated-ingest regressions for second Send, attach/detach, and Session switch would pin the boundary down. Please push back if the input layer serializes all of these operations through an invariant I missed.
| } catch (error) { | ||
| // Nothing dispatched, so this attempt's Artifacts would be orphaned: | ||
| // delete them and leave staging untouched for the retry. | ||
| for (const ref of freshlyIngested) deleteAttachmentBestEffort(ref); |
There was a problem hiding this comment.
Thanks for preserving retry state on an ingest failure. [P2] (category ② — partial batch failure) The earlier successful descriptors have already been replaced with retained refs at line 1227, then this branch deletes those Artifacts without restoring the descriptors. The strip still has the same item count, but the next retry submits refs that now point to deleted Artifacts and fails with Attachment Artifact was not found. Could the batch keep descriptors untouched until every ingest succeeds, or restore them during rollback, and extend this test through an actual second submit rather than only checking the strip count? Please push back if delete is guaranteed to leave these refs reusable.
| return; | ||
| } | ||
| // The Message is the attachments' durable owner now. | ||
| imageAttachments.clear(); |
There was a problem hiding this comment.
Thanks for keeping the transient row when admission is uncertain. [P2] (category ② — disconnect/outcome-unknown recovery) The driver deliberately returns undefined when a dispatched submit has an unknown outcome, but this branch treats every non-blocked result as durable ownership and clears the attachments. If the Message was not admitted, reconnect later retires the transient row while the user has neither staged refs to retry nor an owner that cleans up the ingested Artifacts. Could attachment ownership stay with the attempt until transcript/query reconciliation proves admission or cancellation, and keep staging on undefined? An attached outcome-unknown → reconnect regression would cover both branches. Please push back if another reconciliation path restores these refs.
| return; | ||
| } | ||
| const removed = imageAttachments.list()[index - 1]!; | ||
| imageAttachments.remove(removed.stagingKey); |
There was a problem hiding this comment.
Thanks for adding explicit draft removal. [P2] (category ① — normal /detach path) This is not always client-local: retracted queued Messages re-stage already committed retained refs. Removing one here drops the last draft/message owner but never calls the existing Artifact delete authority, so repeated retract → detach leaves persistent user-upload Artifacts behind. Could this delete best-effort when the removed item is retained, while local descriptors remain a pure list removal? Please push back if retained refs have a separate garbage-collection owner.
| // reference as a retained item, so resubmitting reuses the exact Artifact | ||
| // instead of re-ingesting the same bytes into a second copy. | ||
| for (const entry of retracted.entries ?? []) { | ||
| for (const attachment of entry.attachments) imageAttachments.stageRetained(attachment); |
There was a problem hiding this comment.
Thanks for preserving attachments during Alt+Up retraction. [P2] (category ① — normal multi-message retraction) The Host returns per-message entries, but this flattens every attachment into one draft and refillEditorFromQueues joins the texts. Two valid queued messages can each carry the allowed 8 images; retracting both creates a 16-image draft whose next submit is deterministically rejected by the per-message MAX_ATTACHMENT_COUNT. Could the retraction preserve entry grouping, or define a bounded restaging behavior that always remains submit-able, with a two-message regression? Please push back if the Host guarantees that only one attachment-bearing entry can be retracted here.
/attach now sorts ahead of /compact, so the initial slash-autocomplete cursor lands on it; the resize-visibility assertions must wait for the cursor on /attach. Counter, wrap, and resize behavior are unchanged.
|
Follow-up on 197968a (the rework commit plus one test fix):
|
Summary
Closes #4171. Adds a self-contained TUI image-attachment ingestion path: the user attaches an image with
/attach <path>, the bytes are ingested through the existing Runtime Hostartifact.ingestauthority, and the draft carries only an atomic[image N]marker. At submit time, each marker serializes to the exact committed SessionAttachmentRefalongside the visible label — on every path: idle send, mid-turn steering, Alt+Enter queued successors, failed-dispatch retry, and retraction.Before this change, the TUI's submit path carried text only, while Runtime Host accepts only canonical
session_fileattachments already committed to the Session ArtifactStore. The Host protocol, admission validation (validateTurnAttachments), and model-facing consumption (native image parts for vision models; name + MIME + Read reference otherwise) already existed — this PR builds the missing client ingestion surface and keeps its ownership boundaries explicit.How it works
/attach <path>(TUI slash catalog + guidance copy) reads the file client-side with a byte cap (MAX_ATTACHMENT_BYTES), refuses non-images and over-count drafts (MAX_ATTACHMENT_COUNT), and calls the new driveringestAttachment— which sequencesartifact.ingest begin → chunk… → commitover the same connection used for turns, aborting the staged upload on any failure. The[image N]marker is inserted atomically at the cursor immediately; the committed ref binds when the ingest lands.tui-attachments.tsholds a draft-scoped registry mapping ordinals to committed refs.bindForSubmitscans the outgoing text: every distinct[image N]token binds its staged ref (first-appearance order, deduped); tokens with no staged ordinal are inert literal text and bind nothing. Binding is token-identity based, so editing marker text can orphan, duplicate, or renumber markers — it can never make one image's bytes travel under another image's marker.MakaSubmitMessageOptions/MakaPreparePromptOptionscarryattachments, and the driver puts them oncontent.attachmentsforturn.message.submitandturn.start(steering and follow-up queue entries carry the fullMessageContent, so queued images ride the existing Host machinery unchanged).MakaRetractedMessagesgains per-entry grouping (entries: { text, attachments }). Alt+↑ and the interrupt path re-stage every retracted ref and renumber markers into one collision-proof draft sequence (planRestoredImageAttachments+admitRestoredPlan), so the recalled text keeps a marker where each image was and every marker rebinds to exactly the image it rode with.steering_messageechoes, and the transient row render committed attachments as📎 name · mime · sizechips — names only, never a local path; resumed sessions show them through the same hydration path.Cleanup behavior (defined in this PR)
artifact.delete.artifact.ingest abort); the marker is removed from the draft; nothing is staged and there is nothing to clean./new, and graceful exit all abandon staged refs and delete their committed Artifacts best-effort (refs carry their owning Session, so cleanup is correct even after switching).Ownership boundaries
artifact.ingest, deletion throughartifact.delete, and the TUI never touches an ArtifactStore. Local and remote Hosts take the identical protocol path.Non-goals honored
No second attachment store; no OCR/vision inference; no temp paths in prompts or transcripts.
Test plan
New
packages/cli/src/__tests__/tui-image-attachments.test.ts(20 tests): marker/registry unit semantics (binding order, dedupe, stranger tokens, restage, restore renumbering, capped file reads, path/MIME resolution) and end-to-end TUI flows — upload + exact-ref submission, Send-not-blocked by in-flight ingest, failed ingest, marker deletion → Artifact deletion, failed submit → retry rebind, queue → retraction → same-ref resubmission, session-switch recovery rendering without path leakage, no-authority refusal, non-image refusal.Extended
runtime-host-session-driver.test.ts:artifact.ingestbegin/chunk/commit chunking and payload SHA-256, abort-on-failure,artifact.deleteby exact reference (tolerating not-found), attachments on the submit wire, and per-entry retraction grouping.packages/cli: core + cli typecheck clean, Biome clean; the affected suites pass (73/73 driver tests; 20/20 attachment tests; pi-tui-runner and pi-transcript suites pass with the same pre-existing Windows-only environmental flakes — resize/SIGTERM child-process probes — verified identical on a clean baseline).Refs #4171
Refs #4079
Refs #4080