fix(wall): don't touch V8 handles in ~WallProfiler - #399
Merged
Conversation
Terminating a worker thread with profiling enabled aborts the process: FATAL ERROR: v8::HandleScope::CreateHandle() Cannot create a handle without a HandleScope 4: v8::HandleScope::CreateHandle(v8::internal::Isolate*, unsigned long) 5: dd::WallProfiler::~WallProfiler() [dd_pprof.node] 6: dd::WallProfiler::CleanupHook(void*) [dd_pprof.node] 7: node::CleanupQueue::Drain() 8: node::Environment::RunCleanup() 9: node::FreeEnvironment(node::Environment*) 10: node::worker::Worker::Run() Reported as DataDog/dd-trace-js#9891: a terminated worker (worker.terminate(), or process.exit() with a live worker) kills the process with SIGABRT/SIGSEGV. It bit drizzle-kit migrate, which runs migrations in a worker and terminates it, so deploy containers started failing after a successful migration. The fatal prints twice because OnFatalError -> DumpJavaScriptBacktrace -> GetCurrentStackTrace opens an EscapableHandleScope, which fails the same way and re-enters OnFatalError. Introduced by the "Add the zero-out-internal-field logic to PCP too" commit of #391, first shipped in 5.18.0. #391 nulled the holder's internal field in DrainLiveCtxWraps, which is right there: that slot is the OTEP-4947 reader's contract, so leaving it pointing at freed memory aims a loaded gun at a consumer we do not control. Copying the same logic into ~WallProfiler was symmetry, and the symmetry does not hold in either direction. It is not safe. node::FreeEnvironment wraps RunCleanup in a SealHandleScope, so handle_.Get() needs an inner HandleScope of its own — that much is fixable. The rest is not: WallProfiler is a Nan::ObjectWrap, whose weak callback is first-pass and deletes the wrap inline, and V8 documents that no API call may be made from a first-pass weak callback. That path is real, not theoretical (instrumenting the destructor shows it reached with a non-empty live list after dropping a profiler without dispose()), and where the holders die in the same GC nothing orders their weak callbacks before the profiler's, so Get() can materialize a Local to an already-condemned object. It is also not needed. A PCP holder is reachable only through that profiler's own cpedKey_: another WallProfiler in the same isolate uses a different key object, JS cannot read internal fields, and the out-of-process reader keys off the otel ALS. The slot dies with the profiler that could have read it. So drop the zero-out rather than scope it, which restores 5.17.0 behaviour for PCP. Verified against the reporter's Docker repro on node:24-alpine/arm64: 5/5 clean on terminate() and 3/3 on process.exit(), against 3/3 aborts before. Contrary to the report, glibc is affected too — node:24-bookworm-slim aborts 3/3 (exit 133 rather than 139, and with a populated stack trace, which is where the trace above comes from). It is a plain ApiCheck; libc has nothing to do with it. worker2 now establishes a sample context, because the existing "should not crash when worker is terminated" test missed all of this: it never set one, so liveContextPtrHead_ was always empty and the faulting walk never executed. With a context set, reinstating the unscoped zero-out fails the test. DrainLiveCtxWraps keeps its HandleScope and gains a comment saying why it is there, since that is exactly what was not obvious.
Overall package sizeSelf size: 2.53 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | pprof-format | 2.3.1 | 504.33 kB | 504.33 kB | | source-map | 0.8.0 | 185.66 kB | 185.66 kB | | node-gyp-build | 4.8.4 | 13.86 kB | 13.86 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
szegedi
marked this pull request as ready for review
August 20, 2026 13:49
szegedi
requested review from
IlyasShabi,
nsavoire and
r1viollet
as code owners
August 20, 2026 13:49
nsavoire
approved these changes
Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the 5.18.0 regression reported as DataDog/dd-trace-js#9891: terminating a worker thread with profiling enabled aborts the process.
Cause
e0ce2c2(#391), second commit — "Add the zero-out-internal-field logic to PCP too".On its surface, this could be fixed by adding a
HandleScopeto~WallProfiler, but it unfortunately doesn't fully fix the problem, because we can invoke~WallProfilerthrough theWeakCallbackofNan::ObjectWrap. V8 docs sayThis is something we clearly violate here. Turns out V8 release build doesn't actually police this today, but where the holders die in the same GC nothing orders their weak callbacks before the profiler's, so
Get()can materialize aLocalto an already-condemned object and write its internal field.And in the end, this code isn't needed: a PCP holder is reachable only through that profiler's
cpedKey_. AnotherWallProfilerin the same isolate uses a different key object. The slot dies with the only thing that could have read it so it doesn't matter whether it's zero'd out or not. So this restores 5.17.0 behavior for PCP.Extending the tests so they catch this
test-worker-threads.tsalready terminates 100 workers, butworker2.tsnever set a sample context soliveContextPtrHead_was always empty and the faulting walk never ran. I've now extended it so that it establishes one sample context around the listener registration, so the frame holding it stays reachable until termination.Jira: PROF-15797