Skip to content

fix(suggestions): actually cancel superseded requests - #78

Merged
alpha5611331 merged 6 commits into
mainfrom
alpha/fix-suggestion-blocking
Aug 7, 2026
Merged

fix(suggestions): actually cancel superseded requests#78
alpha5611331 merged 6 commits into
mainfrom
alpha/fix-suggestion-blocking

Conversation

@alpha5611331

Copy link
Copy Markdown
Member

Closes #77

Depends on PowerInterviewAI/backend#45 - see Ordering.

Why

Superseding a suggestion never cancelled the previous request anywhere. stopRunningTasks only flipped a boolean, checked before await reader.read(), so a parked read never observed it. requestStream passed no signal to fetch. The abandoned body got releaseLock() without cancel() - which undici documents as leaking the connection and causing "stalls or deadlocks". Its bodyTimeout defaults to 300s, so a stuck card sat in Pending for five minutes.

And because the client never disconnected, the server had nothing to react to and drained the full completion - so superseded requests kept burning the provider budget the new request needed.

Ctrl+Shift+F11 "doesn't show at all"

That hotkey is the action path, and it had two independent permanent-failure modes:

  • One stalled action stream held the action lock forever. The same lock gates F9 and F12, so a single stall disabled all three hotkeys, with no recovery short of restarting - nothing in stop() or clear() releases it. The lock was also taken before the try, so a throw during setup leaked it with no network call involved.
  • startAssistant's catch never tore down transcription. A failure after transcription.start() succeeded left isActive true while runningState went back to Idle: live suggestions kept working while every action hotkey refused forever. That signature matches the report exactly and needs no stall at all.

Also fixed

  • An empty stream left a live suggestion in Pending forever. State was only promoted to Loading inside if (value), and the terminal check only fired for Loading. No timeout rescued it because the stream ended rather than stalled. Reachable in practice: _strip_think_stream yields nothing when a model emits only a <think> block, and LLM_MODEL_FREE is a thinking model. The action path had the mirror bug and rendered a blank Success.
  • clear() now bumps an epoch. An aborted task's terminal write lands a microtask after clear() empties the map and used to re-insert a dead session's suggestion. Real aborts make that the normal path, so this change would have made an existing bug fire more reliably.
  • An orphaned mic partial gated live suggestions until the candidate next finished speaking - which can span several interviewer questions if they stay quiet.

Review notes

Why not AbortSignal.timeout. It is a total wall-clock deadline and would truncate a long-but-healthy generation. The stall timer resets on every chunk instead. It is also deliberately not a race against reader.read(): a losing read promise stays pending having already consumed a read request, which would leave two outstanding reads on one reader.

TTFB is budgeted per service (20s live, 45s action) because an action request uploads up to four screenshots that the backend base64-encodes before the provider emits a token.

The staleness threshold is a trade-off. The gate exists to stop suggestions firing over someone mid-answer, so SELF_PARTIAL_STALE_MS is deliberately generous at 15s. The deterministic fix - having the ch_1 websocket signal main on close so the orphan is promoted to a final - is not in this PR and is worth a follow-up.

Instrumentation, not a fix. The transcript gate now logs blockedByPartial / skipDueToRecentSelf / lastSelfAgeMs. It is the only place a suggestion is suppressed without a trace. Worth keeping until we have field data, because acoustic echo (testing on speakers rather than headphones) can hold skipDueToRecentSelf true almost continuously.

Ordering

Land backend#45 first or simultaneously. This PR is what causes the server to start getting cancelled mid-stream; without the shielded close on the server, the socket leak moves from client to server rather than disappearing.

Verification

  • tsc (both configs), eslint, and pnpm test:main pass locally.
  • Lock repro: capture a screenshot, trigger F11, kill the backend mid-stream, then press F11 and F9 again. Before: both refused indefinitely, restart the only recovery. After: the stall timeout aborts, the lock releases, both work.
  • Empty stream: point at a stub returning 200 with an empty body - the card must resolve to a stated error rather than sit unresolved.
  • Long generation not truncated: an action suggestion over 4 screenshots producing a long patch must not be cut off.

🤖 Generated with Claude Code

alpha5611331 and others added 2 commits August 6, 2026 19:02
Superseding a suggestion never cancelled the previous request anywhere.
stopRunningTasks only flipped a boolean, and it was checked before
await reader.read(), so a parked read never observed it. requestStream
passed no signal to fetch, and the abandoned body got releaseLock()
without cancel(). Undici is explicit that an unconsumed, uncancelled
response body leaks the connection and can stall or deadlock later
requests. Its bodyTimeout defaults to 300s, so a stuck card sat in
Pending for five minutes.

Because the client never disconnected, the server had nothing to react
to and drained the full completion, so superseded requests kept burning
the provider budget the new request needed.

The stall timeout is a resettable timer, not AbortSignal.timeout: that
is a total wall-clock deadline and would truncate a long but healthy
generation. It is also not a race against reader.read(), because a
losing read promise stays pending having already consumed a read
request. Time-to-first-byte is budgeted separately per service, since an
action request uploads up to four screenshots before the provider emits
a token.

Ctrl+Shift+F11 reported as "doesn't show at all": one stalled action
stream held the action lock forever, and the same lock gates F9 and F12,
so a single stall disabled all three hotkeys with no recovery short of
restarting. The lock was also taken before the try block, so a throw
during setup leaked it with no network call involved.

Also fixed:

- an empty stream left a live suggestion in Pending forever, because
  state was only promoted to Loading inside if (value) and the terminal
  check only fired for Loading. No timeout rescued it, since the stream
  ended rather than stalled. The action path had the mirror bug and
  rendered a blank Success card
- clear() now bumps an epoch. An aborted task's terminal write lands a
  microtask later and used to re-insert a dead session's suggestion into
  the freshly cleared map - and real aborts make that the normal path
- startAssistant's catch never tore down transcription, so a failure
  after transcription.start() succeeded left isActive true while
  runningState went back to Idle: live suggestions kept working while
  every action hotkey refused forever
- an orphaned mic partial gated live suggestions until the candidate
  next finished speaking, which can span several interviewer questions

The transcript gate is now logged. It is the only place a suggestion is
suppressed without a trace, and it separates "the request was never
made" from "the request was made and stalled".

Refs #77

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to the cancellation work, covering the blocking paths that were
deferred from the first pass.

- promote an orphaned partial to a final when its ASR socket drops.
  A reconnect opens a fresh backend session, so the interrupted
  utterance never gets its final. The staleness guard was only a 15s
  backstop; this removes the window. The original endTimestamp is kept
  deliberately - stamping it now would make it the "recent self" the
  suggestion gate measures against, suppressing the next suggestion at
  exactly the moment the user is recovering from a dropped socket
- cap screenshot capture at CAPTURE_MAX_EDGE_PX. NativeImage.toPNG() is
  synchronous and runs on the main process, so capturing at full
  physical resolution stalled the event loop for hundreds of
  milliseconds per capture, blocking IPC, transcript ingest and any
  in-flight suggestion stream. Scaling at capture time rather than in
  sharp afterwards is what makes that cheap
- send only the most recent transcripts with a suggestion request. The
  backend already slices to its own window before building the prompt,
  so the rest was upload cost that grew for the whole interview. This
  does not bound retained history: the summary and .docx export read
  the full transcript from app state
- subscribe to app-state broadcasts before the first fetch. refreshState
  is an IPC round-trip and main never replays, so anything landing
  during that await was dropped. The subscription now lives for the
  app's lifetime; tearing it down at zero subscribers reopened the
  window on every re-init, and StrictMode's double mount plus ordinary
  route changes drive the count to zero routinely

Broadcast coalescing is deliberately not included. test/app-state.test.mjs
pins one-broadcast-per-change, and bounding payload size instead would
truncate the transcript panel. Both need a design decision, so they
belong in their own change.

Refs #77

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@alpha5611331

Copy link
Copy Markdown
Member Author

Pushed a second commit closing the blocking paths deferred from the first pass:

  • Orphaned partial promoted on ASR disconnect. The staleness guard was only a 15s backstop; the ch_1 socket now signals main on close so the interrupted utterance is closed out immediately and its text is kept. The original endTimestamp is preserved deliberately - stamping it now would make it the "recent self" the suggestion gate measures against, suppressing the next suggestion at exactly the moment the user is recovering from a dropped socket.
  • Capture resolution capped (CAPTURE_MAX_EDGE_PX). NativeImage.toPNG() is synchronous and runs on the main process, so capturing at full physical resolution stalled the event loop for hundreds of milliseconds per screenshot - blocking IPC, transcript ingest, and any in-flight suggestion stream. Scaling at capture time rather than in sharp afterwards is what makes it cheap. 1920 is a deliberate floor: the model has to read code off these, and over-shrinking fails silently.
  • Transcript upload bounded (TRANSCRIPT_UPLOAD_LIMIT). The backend already slices to its own window before building the prompt, so the rest was upload cost that grew for the whole interview. Retained history is untouched - exportTranscript reads the full transcript from app state for the summary and .docx, so bounding storage would have silently truncated the user's report.
  • Renderer subscribes before the first fetch. refreshState is an IPC round-trip and main never replays, so anything landing during that await was dropped. The subscription now lives for the app's lifetime; tearing it down at zero subscribers reopened the window on every re-init, which StrictMode's double mount and ordinary route changes both trigger.

Deliberately not included

Broadcast coalescing. test/app-state.test.mjs pins one-broadcast-per-change, with a header explaining the reasoning, and a 50ms coalesce breaks three of its assertions. Bounding payload size instead would truncate the transcript panel, since the renderer only ever sees what is broadcast. Both are real design decisions and neither belongs inside a PR about cancellation - happy to open a separate one if you want the main-process cost addressed.

tsc (both configs), eslint, and pnpm test:main all pass.

alpha5611331 and others added 3 commits August 6, 2026 19:21
Every broadcast structured-clones the whole renderer state, and they fire
on each streamed token and each ASR partial - roughly 20/second across
two channels, against a transcript array that grows for the whole
interview. That cost scaled with events and with session length, which
is the shape of a stall that only shows up late in a long interview.

Coalescing on a short timer bounds it per unit time instead. 50ms is
short enough that streaming still reads as streaming.

This changes a contract that test/app-state.test.mjs pinned, so the test
moves with it: it now flushes explicitly rather than counting one send
per mutation. The two invariants it actually protects are unchanged and
still asserted - the CV never reaches the renderer, and an update that
changes nothing does not broadcast. flushRenderer is deliberately a
no-op when nothing is scheduled, so flushing cannot manufacture a
broadcast that change detection suppressed.

Refs #77

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All three panels rendered every row on every state broadcast, so layout
and paint cost grew with session length and the UI got sluggish late in
a long interview. Transcripts grow fastest - one row per ASR final - but
the suggestion panels carry markdown, code blocks and screenshots, so
their per-row cost is far higher even at smaller counts.

Uses content-visibility rather than a windowing library. Rows wrap to
variable heights, which fixed-height windowing mis-measures, and `auto`
in contain-intrinsic-size remembers each row's last rendered size, so
scroll position and scrollIntoView stay accurate without measurement
plumbing or a new dependency. Off-screen rows skip style, layout and
paint while staying in the DOM, so scrollback is fully preserved.

On the suggestion panels this goes on the row content rather than
SuggestionReveal's wrapper, which is an animating grid that size
containment would fight. The newest card - the streaming one - is always
on screen, so containment never applies while it is being written.

Refs #77

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An aborted fetch rejects with the abort *reason*, so a stall abort
surfaces as TimeoutError rather than AbortError. The name checks
therefore missed stalls: requestStream wrapped a deliberate cancellation
as a network failure and logged it, and the services reported
aborted=false and logged a spurious error for a timeout they had
themselves requested.

Verified against a server that sends one chunk then goes silent:
reader.read() rejects within milliseconds of the abort carrying
name=TimeoutError, confirming both the mechanism and the misdetection.

Keying on signal.aborted covers both reasons. A five-deep supersede
chain against that server now resolves as four stopped plus one
timed-out, leaves the abort map empty, and closes every socket - which
is the leak that made superseding break suggestion generation.

Refs #77

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@alpha5611331

Copy link
Copy Markdown
Member Author

Review pass - one real bug found, plus end-to-end proof the blocking issue is fixed

Bug: abort detection was keyed on the wrong thing

An aborted fetch rejects with the abort reason, not a generic AbortError. Verified against a server that sends one chunk then goes silent:

read REJECTED after 313ms | name=TimeoutError | reason=TimeoutError

So a stall abort surfaces as TimeoutError, and the error.name === 'AbortError' checks missed it. Consequences: requestStream wrapped a deliberate cancellation as a network failure and logged it, and both services computed aborted = false for a timeout they had themselves requested. The user-visible message happened to come out right via the stalled branch, but the abort semantics were wrong and the logging was noise.

Now keyed on signal.aborted, which covers both reasons.

The blocking issue, demonstrated

Simulated the reported scenario - five supersedes in a row against a server that streams one chunk then stalls forever, mirroring the real abortMap / stopRunningTasks flow:

results: [[1,"stopped"],[2,"stopped"],[3,"stopped"],[4,"stopped"],[5,"timed-out"]]
abortMap size after all tasks (must be 0): 0
server sockets still open (must be 0): 0

Three things this establishes:

  1. Superseded requests resolve immediately rather than parking on reader.read(). Before this PR they sat until undici's 300s bodyTimeout.
  2. Stall is distinguished from supersede - the last task reports timed-out, the rest stopped.
  3. Every socket closed. This is the leak undici warns causes "stalls or deadlocks", and it is the mechanism by which superseding broke suggestion generation. Zero open sockets is the direct evidence it is fixed.

tsc (both configs), eslint and pnpm test:main all pass.

Also confirmed, not assumed

DOMException instanceof Error is true in this runtime, so the signal.reason narrowing in both services is sound.

Review of the diff turned up a latent lock leak. generateSuggestion
looked its controller up in the abort map and returned early if it was
missing - before the finally that releases the action lock, and
resolving normally so the caller's .catch could not release it either.
The result would be a permanently held lock disabling F9, F11 and F12
for the session, which is the exact bug this service is being fixed for.

Not reachable today: the map entry is set immediately before the call
and nothing awaits in between. But it becomes reachable with any future
await or reorder, and the failure is silent and unrecoverable.

Passing the controller removes the not-found branch entirely, so there
is no path that can skip the release. Same shape applied to the live
service, and its redundant empty-transcript guard dropped for the same
reason - the caller already returns before registering a task, and a
second check would have leaked the abort map entry.

Verified the lock is released on every exit path: normal completion,
network error, abort mid-stream, stall abort, and a throw before the try
block.

Also tidied the transcript gate: two comments had merged into one
unreadable block, and adjacent conditions were reading the clock from
different sources.

Refs #77

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@alpha5611331
alpha5611331 merged commit 55fabdf into main Aug 7, 2026
1 check passed
@alpha5611331
alpha5611331 deleted the alpha/fix-suggestion-blocking branch August 7, 2026 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suggestion generation blocks: superseded requests are never cancelled

1 participant