Skip to content

Optimize MoonBit primitive stream buffering#1664

Merged
alexcrichton merged 1 commit into
bytecodealliance:mainfrom
peter-jerry-ye:codex/moonbit-stream-buffer-budget
Jul 24, 2026
Merged

Optimize MoonBit primitive stream buffering#1664
alexcrichton merged 1 commit into
bytecodealliance:mainfrom
peter-jerry-ye:codex/moonbit-stream-buffer-budget

Conversation

@peter-jerry-ye

@peter-jerry-ye peter-jerry-ye commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Why

MoonBit async streams currently use a fixed 64-element window regardless of
the canonical element size. That leaves small primitive streams with tiny
transfers and causes avoidable canonical ABI callbacks and scheduler activity.

The async runtime also snapshots borrowed array views with explicit
element-by-element constructors instead of standard collection-copy APIs.

What changed

  • Budget fixed primitive stream windows at 4 KiB, resolving WIT aliases before
    selecting the element count.
  • Pass the selected write window into each generated Sink.
  • Keep the existing 64-element fallback for non-primitive streams and the
    single-element staging rule for nested endpoint payloads.
  • Use FixedArray::from_array and BytesView::to_fixedarray when the runtime
    must take an owned snapshot of a borrowed view.
  • Cover primitive widths, fallback behavior, and generated runtime copy shapes.

Performance

The WASI HTTP benchmark used a 64 KiB body, 64 connections, and three
interleaved 6-second runs per variant. Fixed-rate results below are medians;
all requests succeeded.

Endpoint Offered rate CPU/request, main → PR Instructions/request, main → PR p99, main → PR
181 B response 24,859 req/s 0.250 → 0.190 ms 1.06M → 0.89M 2.64 → 2.08 ms
64 KiB upload 8,000 req/s 1.57 → 0.593 ms 20.65M → 2.81M 3.03 → 2.16 ms
64 KiB one-chunk download 250 req/s 23.15 → 1.88 ms 90.70M → 5.92M 12.03 → 2.16 ms
64 KiB upload + download 200 req/s 24.48 → 2.62 ms 111.32M → 7.94M 13.07 → 3.32 ms

For the bidirectional 64 KiB case at saturation, median throughput increased
from 351 to 13,625 req/s (38.8x), while p99 fell from 667 to 7.87 ms.

Holding the byte window at 4 KiB and comparing the old makei snapshot with
the standard-library byte conversion isolates the copy change:

Endpoint CPU/request, makei → byte blit Instructions/request, makei → byte blit
64 KiB one-chunk download 2.59 → 1.87 ms (-27.6%) 11.36M → 5.91M (-48.0%)
64 KiB upload + download 2.94 → 2.53 ms (-14.1%) 13.39M → 7.93M (-40.8%)

The baseline component was generated from upstream main at 6b3ec405. It is
byte-identical to the previous v0.60.0 benchmark component because the MoonBit
generator did not change between those commits.

This workload exercises stream<u8>. The other primitive widths are covered by
the generated-code policy tests, but these byte-stream speedups are not claimed
for them.

Why this is correct and minimal

Only fixed primitive streams receive larger windows. Non-primitive streams and
nested future/stream payload staging retain their existing behavior.

The runtime still copies borrowed views before a callback can suspend, so the
lifetime invariant is unchanged. The byte-specialized path delegates to the
standard library's blit-backed conversion. Generic copies use the standard
API boundary, leaving any future bulk-copy implementation to MoonBit core.

This does not introduce owner buffers, dynamic window adjustment, or generic
canonical element-lowering changes.

@peter-jerry-ye
peter-jerry-ye marked this pull request as ready for review July 24, 2026 05:33
@alexcrichton
alexcrichton added this pull request to the merge queue Jul 24, 2026
Merged via the queue into bytecodealliance:main with commit 7c48901 Jul 24, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants