Skip to content

Fix AMF encoders reported as broken, harden AMF ffmpeg args - #55

Open
StuartCameronCode wants to merge 1 commit into
mainfrom
fix/51-amf-encoder-issues
Open

Fix AMF encoders reported as broken, harden AMF ffmpeg args#55
StuartCameronCode wants to merge 1 commit into
mainfrom
fix/51-amf-encoder-issues

Conversation

@StuartCameronCode

Copy link
Copy Markdown
Owner

Addresses #51. Deliberately not "Fixes" — one half is confirmed, the other isn't (see below), so this shouldn't auto-close the issue.

AMF reported as broken — diagnosed, fixed

The functional availability probe encoded a 64x64 frame. Hardware encoders have minimum dimensions and AMF rejects that size, so VapourBox marked working AMF encoders unavailable. The reporter confirmed the same command succeeds at a larger size.

Probe is now 512x512 (AMF's floor is 192x128 on some ASICs, and NVENC/QSV have their own). Still a single frame, so the cost is unchanged. Verified locally that both VideoToolbox encoders still pass at the larger size.

CQP crash — addressed, not confirmed

Exit -1073741819 is 0xC0000005, an access violation inside the AMF runtime. I can't reproduce this — there's no AMD hardware here or in CI. The reporter notes CQP works for them outside VapourBox, which points at our invocation rather than CQP itself, so this fixes the two candidates in it that are independently justifiable:

  • AMF is pinned to nv12. We forced no pixel format, so a >8-bit source reached the encoder as p010. 10-bit HEVC encode exists only on some AMD ASICs, and where it doesn't the runtime tends to fault rather than fail cleanly; h264_amf has no 10-bit mode at all. Given the reporter is feeding camera footage, which is frequently 10-bit, this is the main suspect. Trade-off: AMD users lose 10-bit HEVC output by default. Custom FFmpeg Arguments still override it (a later -pix_fmt wins), so it stays reachable on hardware that supports it.
  • -qp_b is now set alongside -qp_i/-qp_p. B-frames were left on the encoder's default QP while I and P were pinned, so a GOP could mix the chosen quality with an unrelated one. Not a crash mechanism by itself, but wrong.

If neither lands, the next step needs a reporter-side test build rather than more guessing from here.

Adjacent bug found while reading

Encoder presets could cross families. Each has its own vocabulary — x264 ultrafast…placebo, NVENC p1…p7, QSV veryfast…veryslow, AMF speed|balanced|quality — and a saved preset can pair a codec with the wrong one. ffmpeg then rejects the option and the whole encode fails. VideoCodec::normalized_preset() substitutes the family default; the Rust preset lists must stay in step with app/lib/models/video_job.dart.

Testing

  • 3 new Rust tests: nv12 forcing for both AMF codecs, -qp_b, and preset fallback across three families (plus a check that valid presets pass through untouched)
  • 2 new Dart tests pinning the probe frame size and args, so this can't silently regress to a tiny frame
  • Full suites: 82 Rust, 229 Flutter. (subtitle_integration_test fails locally only because the whisper model isn't downloaded on this machine; CI provisions it.)

CLAUDE.md gains a Hardware Encoders section covering the three invariants, since each fails in a way that looks like "this GPU doesn't work" rather than an error.

Addresses #51.

The availability probe encoded a 64x64 frame. Hardware encoders have
minimum dimensions and AMD AMF rejects that size, so VapourBox marked
working AMF encoders as broken — the reporter confirmed the same command
succeeds at a larger size. Probe at 512x512 (AMF's floor is 192x128 on
some ASICs); still one frame, so the cost is unchanged. Verified the
larger frame keeps both VideoToolbox encoders passing.

The reported ffmpeg crash under CQP (exit -1073741819 = access
violation) can't be reproduced without AMD hardware, so this addresses
the two things in our invocation that could plausibly cause it:

- AMF is now pinned to nv12. Left to negotiate, a >8-bit source reaches
  the encoder as p010, and 10-bit HEVC encode only exists on some AMD
  ASICs — where it doesn't, the AMF runtime faults rather than failing
  cleanly. h264_amf has no 10-bit mode at all. Custom FFmpeg Arguments
  still override it, so 10-bit HEVC stays reachable where supported.
- -qp_b is set alongside -qp_i/-qp_p. B-frames were left on the
  encoder's default QP while I and P were pinned, so a GOP could mix the
  chosen quality with an unrelated one.

Separately, presets can no longer cross encoder families. Each family
has its own vocabulary (x264 ultrafast…placebo, NVENC p1…p7, QSV
veryfast…veryslow, AMF speed|balanced|quality) and a saved preset can
pair a codec with the wrong one; ffmpeg then rejects the option and
fails the encode. normalized_preset() substitutes the family default.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant