Listen mode poc - #32
Draft
sumerc wants to merge 8 commits into
Draft
Conversation
whisper_full encoded the same audio twice in auto mode: once inside whisper_lang_auto_detect_with_state, then again for the first decode window. Tag the encoder output with the (mel_offset, n_audio_ctx) it was computed from and skip the identical re-encode; assign exp_n_audio_ctx before detection so one call encodes at one window size. Auto-detect now costs the same as a forced language: 1.94x on dictation-length clips (530 -> 274 ms, M5 Pro, turbo-q5), with transcripts unchanged — the skipped work was bit-identical. The forced-language path never had the second encode and is untouched. Upstream as ggml-org/whisper.cpp#3954, unmerged, so it lives in patches/whisper.cpp and make whisper-lib applies it. Guarded twice, because an unpatched build is still correct and merely 2x slower on auto, which no test can distinguish from a fast one: WHISPER_BASE stops the build on a submodule bump, and TestWhisperPatchesApplied matches the checkout against the patch byte-for-byte (also catching hand-edits that ignore = dirty hides). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
What was measured while landing the encoder-reuse patch, so none of it gets re-derived: the before/after numbers, whisper's own time budget (the encoder is 94% of a dictation transcribe and flat in clip length), and the ggml 0.13 -> 0.18 bump measuring neutral on M5 Pro. Also corrects two entries the patch invalidates. The audio_ctx rejection is partly superseded: fault-matrix case H no longer garbles and whisper_init_state is ~10 ms, not the Metal setup it was assumed to be, so sizing is now a quality call rather than a correctness one. The silence-trimming entry gains a conditional, since "not a speed lever" holds only while the encoder window is fixed. Adds the alternative-engine survey: the search space collapses because the 30 s padding is a Whisper architecture property, not a whisper.cpp one. Notes that Parakeet's coverage excludes Turkish and every non-European language, so for those users Whisper is the only local engine and audio_ctx sizing is the only remaining lever; that upstream whisper.cpp now ships Parakeet but with an incompatible model format, so migrating costs a model release and buys build simplicity only; and that MLX is the one untested alternative that survives the coverage filter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.