Add browser-compatible Headers and streamed Response bodies#210
Open
matthargett wants to merge 7 commits into
Open
Add browser-compatible Headers and streamed Response bodies#210matthargett wants to merge 7 commits into
matthargett wants to merge 7 commits into
Conversation
Vendor the ES5 web-streams-polyfill 4.3.0 ponyfill and expose an idempotent Streams initializer that preserves constructors supplied by the selected JavaScript engine. Cover readable, writable, transform, BYOB, error, tee, subclassing, and host-constructor behavior with focused ports from WPT plus Firefox and Chromium regression tests. Validate the implementation on JavaScriptCore under ASan/UBSan and QuickJS Release.
Implement iterable BlobPart conversion, MIME and endings normalization, zero-copy Blob composition and slicing, and lazy 64 KiB byte streams. Delegate File streaming and slicing to its backing Blob while preserving browser class identity.\n\nUse immutable shared segments so nested Blob and slice construction do not duplicate payload bytes. Add focused WPT, WebKit, and Firefox regression coverage for constructor ordering, iterator closure, UTF-8 decoding, BYOB reads, cancellation, lifetime safety, and large nested streams.
Replace the ad hoc buffered fetch response object with standard-shaped Headers and Response classes backed by ReadableStream bodies. Preserve host constructors, normalize and validate header values, enforce single-use bodies, and expose native responses with one transport-to-JavaScript copy. Reuse normalized header views between mutations, compact header storage in place, retain stream chunks only until consumption, and allocate a contiguous body result at most once when required. Add focused WPT, WebKit, Firefox, and Chromium regression coverage plus JavaScriptCore/QuickJS and initialization tests.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR upgrades the runtime’s Fetch surface to use browser-shaped Headers and Response objects (with streamed bodies), adds an optional WHATWG Streams ponyfill to support streaming semantics, and extends the native Blob/File polyfills to support slicing and streaming in a way that matches modern web platform behavior.
Changes:
- Added a Streams polyfill module that installs missing WHATWG Streams constructors from a vendored
web-streams-polyfillES5 ponyfill. - Reworked Fetch to use a JS
Headers/Responsepolyfill layer and to expose response bodies asReadableStreams with single-consumption + cloning behavior. - Extended Blob/File polyfills with
slice()andstream()support and expanded unit tests with focused WPT/WebKit/Firefox/Chromium-derived coverage.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| Tests/UnitTests/Shared/Shared.cpp | Initializes Streams before Fetch and adds idempotence/host-preservation unit tests for Streams and Fetch. |
| Tests/UnitTests/Scripts/tests.ts | Adds conformance-focused tests for Headers, Response, Streams behavior, and expands Blob/File streaming+slicing coverage. |
| Tests/UnitTests/CMakeLists.txt | Links UnitTests against the new Streams library. |
| Polyfills/Streams/ThirdParty/web-streams-polyfill/ponyfill.es5.js | Vendors the ES5 ponyfill bundle used to provide missing Streams constructors. |
| Polyfills/Streams/ThirdParty/web-streams-polyfill/LICENSE | Adds the upstream MIT license for the vendored ponyfill. |
| Polyfills/Streams/Source/StreamsScripts.h.in | Generates an embedded JS script by joining split ponyfill source parts at build time. |
| Polyfills/Streams/Source/Streams.cpp | Implements Streams::Initialize to fill only missing Streams globals via embedded ponyfill. |
| Polyfills/Streams/README.md | Documents behavior and provenance of the Streams polyfill. |
| Polyfills/Streams/Include/Babylon/Polyfills/Streams.h | Declares the Streams polyfill initialization entry point. |
| Polyfills/Streams/CMakeLists.txt | Builds the Streams library and generates embedded ponyfill header. |
| Polyfills/File/Source/File.h | Adds slice() and stream() method declarations to File. |
| Polyfills/File/Source/File.cpp | Delegates File slice/stream to Blob and sets Symbol.toStringTag for File instances. |
| Polyfills/Fetch/Source/FetchScripts.h.in | Adds a generated header for embedding the Fetch JS polyfill. |
| Polyfills/Fetch/Source/FetchPolyfill.js | Introduces JS implementations of Headers, Response, and createFetchResponse using ReadableStreams for bodies. |
| Polyfills/Fetch/Source/Fetch.cpp | Replaces the ad-hoc buffered response object with JS polyfill classes and a streamed body pipeline. |
| Polyfills/Fetch/Readme.md | Updates Fetch documentation to describe streamed response bodies and browser-shaped APIs. |
| Polyfills/Fetch/CMakeLists.txt | Embeds Fetch polyfill JS and updates include paths for generated headers. |
| Polyfills/CMakeLists.txt | Adds optional Streams polyfill subdirectory controlled by a CMake option. |
| Polyfills/Blob/Source/Blob.h | Refactors Blob storage to shared segmented representation and adds slice/stream APIs. |
| Polyfills/Blob/Source/Blob.cpp | Implements zero-copy segmented Blob composition/slicing and lazy streaming via ReadableStream. |
| Polyfills/Blob/README.md | Documents Blob streaming/slicing behavior and Streams dependency. |
| Polyfills/Blob/CMakeLists.txt | Includes Blob README in build sources. |
| CMakeLists.txt | Adds JSRUNTIMEHOST_POLYFILL_STREAMS option to enable Streams polyfill. |
Files not reviewed (1)
- Polyfills/Streams/ThirdParty/web-streams-polyfill/ponyfill.es5.js: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Close byte streams for zero-length BufferSource bodies without enqueueing an invalid empty chunk. Preserve the non-null body and bodyUsed semantics, and add focused regression coverage alongside the WPT empty-response cases.
Replace a partial or null host Streams surface as a complete constructor suite so stream products retain compatible instanceof relationships. Preserve a complete suite on repeated initialization.\n\nHarden the native initialization test so N-API failures complete the test promise instead of hanging, and cover partial host replacement plus null handling and cross-constructor compatibility.
Observe standard ReadableStream consumption paths with weak per-stream state instead of depending on web-streams-polyfill's private _disturbed field. Instrument stream readers and piping once during Fetch initialization, preserve stream identity, and cache the initialized Fetch implementation so repeated initialization does not stack wrappers or replace its internal state. Treat Headers and Response as an implementation pair when either host global is missing or null. Harden native initialization tests against unhandled N-API failures and add WPT-derived coverage for reads, cancellation, piping, and streams disturbed before Response construction.
Use ReadableByteStreamController.byobRequest views directly and respond with the produced byte count instead of allocating and enqueueing a 64 KiB intermediate chunk. Keep the existing bounded allocation path for default readers.\n\nAdd WPT-derived coverage for small offset BYOB views that cross immutable Blob segment boundaries.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #208 and #209. The commits remain separate so this can be rebased to the Fetch-only change after those prerequisites land.
Replaces the ad hoc reusable buffered response object with standard-shaped Headers and Response classes. Headers normalize and validate fields, preserve current Set-Cookie behavior, and support live sorted iteration. Response bodies are ReadableStreams with single-consumption semantics, cloning, and browser-shaped metadata and body readers. Empty BufferSource bodies retain a non-null stream without enqueueing an invalid zero-length byte chunk.
UrlLib still buffers the transport response, but completion now copies it once into JavaScript-owned memory. The Response wrapper does not make another snapshot; multi-chunk consumers retain chunk references and allocate one exact contiguous result only when needed. Header iteration is mutation-cached, and delete/set compact storage in place.
Focused coverage is adapted from WPT fetch/api/headers and fetch/api/response, WebKit body-init and empty-chunk regressions, Firefox open-stream teardown, and a bounded Chromium disturbed-stream regression.
Validated on JavaScriptCore ASan/UBSan, JavaScriptCore TSan, and QuickJS Release: 254 JavaScript cases plus 8 native tests in each configuration.