UN-3996 [FIX] Stop WebSockets exhausting the Gunicorn thread pool - #2241
Merged
Conversation
…thread pool The Socket.IO WSGI app runs with async_mode="threading", so every open WebSocket holds one gthread pool thread for that connection's lifetime rather than for a single request. With --threads 2, two browser tabs landing on the same worker left it with no threads for HTTP, and the worker kept accepting connections it could never read. Make workers, threads and worker-connections environment-configurable and raise the default thread ceiling to 512, matching the value already used by the cloud chart. The pool spawns threads lazily, so the higher ceiling costs nothing at idle. Drive --log-level from DEFAULT_LOG_LEVEL instead of hardcoding debug, and state --worker-class gthread explicitly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kirtimanmishrazipstack
changed the title
kirtimanmishrazipstack
marked this pull request as ready for review
August 13, 2026 08:29
Contributor
|
| Filename | Overview |
|---|---|
| backend/entrypoint.sh | Makes worker and thread counts environment-driven, raises the thread default to 512, and explicitly configures the gthread worker class. |
| backend/sample.env | Exposes the new Gunicorn worker and thread settings with defaults matching the entrypoint. |
Reviews (6): Last reviewed commit: "Merge branch 'main' into UN-4001-gunicor..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
kirtimanmishrazipstack
changed the title
Contributor
Author
|
@greptile-apps please re-review. The one open finding ( |
kirtimanmishrazipstack
changed the title
ritwik-g
reviewed
Aug 13, 2026
ritwik-g
reviewed
Aug 13, 2026
Revert --log-level back to the hardcoded debug so this change is scoped to the thread pool alone. Reusing DEFAULT_LOG_LEVEL conflated Django's application log level with Gunicorn's, which are independent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
They were commented out, unlike the other 110 settings in the file. Leaving them commented hid the knob, which was the original complaint — there was no discoverable way to change the thread count. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1000 is already Gunicorn's default for worker_connections, so passing it changed nothing. Making it configurable only added a way to set it below GUNICORN_THREADS, which silently disables HTTP keep-alive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
kirtimanmishrazipstack
requested review from
chandrasekharan-zipstack and
ritwik-g
and removed request for
ritwik-g
August 17, 2026 08:04
|
Contributor
Unstract test resultsPer-group results
Critical paths
|
ritwik-g
approved these changes
Aug 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
backend/entrypoint.sh: Gunicorn workers and threads are now env-driven, with the thread default raised from2to512.backend/entrypoint.sh:--worker-class gthreadis stated explicitly.backend/sample.env: ships both variables set to their defaults, so the knob is visible in the file every deployment copies.Why
async_mode="threading", so every open WebSocket holds one gthread pool thread for that connection's lifetime rather than for a single request.--threads 2, two logged-in browser tabs landing on the same worker left it with zero threads for HTTP, and it kept accepting connections it could never read — so requests stalled silently instead of failing over.How
frontend/src/index.jsx:64mounts oneSocketProviderper tab →backend/backend/wsgi.py:42wraps Django in the Socket.IO app (async_mode="threading",utils/log_events.py:29-31) →engineio/socket.py:222runs the socket loop inline on a pool thread sized by--threads(gunicorn/workers/gthread.py:98).512matches the value already running in the cloud chart and is free at idle, since--threadsis aThreadPoolExecutorceiling and CPython only starts a thread onsubmit()when none is idle.--worker-class gthreadwas already in effect —config.py:109-110coercessynctogthreadwheneverthreads > 1— so stating it is a no-op that just removes the ambiguity.worker-connectionsis deliberately left alone.1000is already Gunicorn's default (config.py:732), so passing it explicitly would change nothing, and making it settable only adds a way to configure it belowGUNICORN_THREADS, which drivesmax_keepalivednegative (gthread.py:73) and force-closes every response (gthread.py:330).Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)
No — the change is confined to Gunicorn CLI arguments, both variables fall back to the values already in use, and no application code is touched. The only behavioural change is that a worker can now serve more than two concurrent connections.
Database Migrations
Env Config
GUNICORN_WORKERS(default2, unchanged from before).GUNICORN_THREADS(default512, was a hardcoded2) — must exceed concurrent browser tabs per worker.backend/sample.envand reach the container through the existingenv_file: ../backend/.envon the backend service. The entrypoint keeps a:-fallback per variable, so an.envthat predates this PR starts with the same values.Relevant Docs
Related Issues or PRs
--threads 512, so no cloud change is needed.4to2as part of a local-memory cleanup. The line it changed carried a# NOTE updated socket threadscomment, added in the same commit that introduced the Socket.IO threading mode.Dependencies Versions
Notes on Testing
bash -npasses; argument array expanded and inspected with no env set and with both variables overridden.Measured locally with
backend/.venv(Python 3.12.9) —--threadsis a ceiling, not an allocation, so a high default is free until the connections exist:CPU with 512 threads parked: 0.005% of one core over 3s.
VmSizegrows ~12.5 GB at 512 live threads (glibc reserves an 8 MB stack per thread fromRLIMIT_STACK), but that is reserved address space rather than RSS.Repro for reviewers: with
--threads 2, open two logged-in tabs and loopGET /api/v1/health; one worker stops responding whiless -tanp | grep :8000shows ESTABLISHED sockets with non-zeroRecv-Qand an empty LISTEN backlog.Screenshots
N/A — no user-facing surface.
Checklist
I have read and understood the Contribution Guidelines.