Skip to content

bind: make _gopy_clear_go_tls opt-in, off by default#398

Merged
b-long merged 1 commit into
go-python:masterfrom
satarsa:fix-395-clear-go-tls-opt-in
Jul 24, 2026
Merged

bind: make _gopy_clear_go_tls opt-in, off by default#398
b-long merged 1 commit into
go-python:masterfrom
satarsa:fix-395-clear-go-tls-opt-in

Conversation

@satarsa

@satarsa satarsa commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

The _gopy_clear_go_tls() call added for issue #370 runs unconditionally before every CGo entry and performs a hardcoded TLS store (movq $0, %fs:-8 on linux/amd64, movq $0, %gs:0x30 on darwin/amd64). On glibc + CPython 3.12+ that offset overlaps the TLS slot CPython uses for the current thread state (_Py_tss_tstate), so the store nulls it and the interpreter segfaults on the first call: in the common single-extension case, which structurally cannot have the cross-runtime collision the clear was meant to guard against.

This PR makes the clear opt-in via a new -clear-go-tls build flag (default false): the default build is safe, and the previous behavior stays one flag away.

Fixes #395.

The two problems with the unconditional clear

  1. It runs for a single extension too. That is the common case and the one that cannot have a cross-runtime TLS collision, so there the clear is pure downside.
  2. The address is wrong for c-shared under glibc. In a normal Go binary the runtime lays out its TLS so the g pointer sits at %fs:-8. In a c-shared library loaded by glibc the loader owns the static TLS block and the g pointer is reached through a linker-assigned TLS variable (runtime.tls_g), not a fixed -8. So movq $0, %fs:-8 does not clear Go's g at all: it zeroes whatever thread-local happens to sit there, which on CPython 3.12+ is _Py_tss_tstate.

PR #393 already shipped the mechanism that actually isolates the runtimes: each extension is loaded without RTLD_GLOBAL, and gopy links each .so with a version script exporting only PyInit_*. Both keep every Go runtime symbol (including the g-pointer TLS) local to its own .so, so there is nothing shared to collide over. The raw-TLS clear is therefore redundant, and with the wrong offset actively harmful.

What this PR changes

  • Adds a bind.ClearGoTLS option and a -clear-go-tls build flag (default false), mirroring the existing -no-make flag across cmd_build.go, cmd_gen.go, cmd_pkg.go, cmd_exe.go.
  • Gates the single _gopy_clear_go_tls() emission site in bind/gen_func.go on that flag.
  • Leaves the C helper and its pybindgen registration in place, so -clear-go-tls restores today's behavior exactly.
  • Leaves the RTLD_GLOBAL handling untouched and unconditional.

6 files, +33/-8. gofmt, go vet ./..., and go test ./bind/ are clean.

Verification

Downstream (the crashing project). Gentoo, glibc, CPython 3.14.6, go 1.26.4,
gcc 16. Rebuilding slidge-whatsapp's extension:

  • Default (clear off): the generated wrapper contains zero _gopy_clear_go_tls() sites and the first call succeeds (exit 0).
  • gopy build -clear-go-tls: the call reappears (one per CGo entry) and the crash returns (SIGSEGV, exit 139).

So the flag is a faithful toggle, and off-by-default removes the crash with no post-build patching.

Multi-extension case, clear off. Two independent gopy extensions (two Go runtimes) loaded in one interpreter, 200000 interleaved cross-calls on the same OS thread with periodic gc.collect() (the condition #370 describes): 0 wrong results, no crash, no corruption. And each .so exports exactly one global dynamic symbol (PyInit_*) and zero runtime.* symbols, so the g pointer is per-runtime-private regardless of RTLD_GLOBAL.

I do not have the original #370 reproducer, so I cannot claim to have re-run that exact case; the opt-in keeps the old behavior available for anyone who needs it.

Follow-up (optional)

Happy to add regression tests, here or in a follow-up:

  1. A codegen assertion (layout-independent): default flags produce a wrapper without _gopy_clear_go_tls(), -clear-go-tls produces one with it.
  2. A two-extension _examples coexistence example, cross-called under gc.collect(), asserting a clean run with the clear off.

The unconditional _gopy_clear_go_tls() call (issue go-python#370) performs a
hardcoded TLS store (movq $0, %fs:-8 on linux/amd64). On glibc + CPython
3.12+ that offset overlaps CPython's current-thread-state TLS slot, so the
store nulls it and the interpreter segfaults on the first CGo entry in the
common single-extension case (issue go-python#395).

Add a -clear-go-tls build flag (default false) and gate the single call
site on it. The C helper and its pybindgen registration are left in place
so opt-in restores the previous behavior exactly. RTLD_GLOBAL-local
loading, which is what actually isolates each runtime's goroutine-pointer
TLS, is untouched and remains unconditional.

Fixes go-python#395

@b-long b-long left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@satarsa Thanks so much for this PR! 🙇

I'm happy to merge this—it's definitely a great improvement for gopy. While reviewing, the most straightforward test I could think of was adding Python 3.12.x to the build matrix. (We clearly need to expand our test coverage, but that’s a separate topic for later.)

Would you mind taking a look at PR #399? If you check the order of my commits, you can see where tests pass and fail. By cherry-picking and then reverting your patch, it seems to confirm just how valuable your work is for our Python 3.12 support.

I also ran into a bug while working on that PR and think I’ve patched it, but I’d love to get your thoughts on the fix if you have a moment.

Thanks again!

@b-long
b-long merged commit f0b546c into go-python:master Jul 24, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#370's _gopy_clear_go_tls (movq $0, %fs:-8) hard-crashes CPython 3.12+ in a shipping extension

2 participants