Skip to content

hono-bun: size the Postgres pool to the process count - #1251

Open
MDA2AV wants to merge 1 commit into
mainfrom
fix/hono-bun-pg-pool
Open

hono-bun: size the Postgres pool to the process count#1251
MDA2AV wants to merge 1 commit into
mainfrom
fix/hono-bun-pg-pool

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Aug 21, 2026

Copy link
Copy Markdown
Owner

site/static/logs/crud/4096/hono-bun.log came out of #1242 at 54MB / 1.9M
lines
, where every other log in that directory is under 11KB. 72,031 of those
lines are Postgres answering sorry, too many clients already.

It is not cosmetic: the published crud-4096 carries status_5xx=23308.
Every other JS entry publishes 0 there, and hono-bun's 218k rps trails
bun/node/deno at 310–347k. It cleared the 5% error gate, so it was published
anyway.

Cause

The pool was a fixed max: 8 per process. entrypoint.sh starts one process per
CPU, and the crud profile hands the container the cpuset 1-31,65-95 — so
nproc reports 62 on the benchmark box:

62 processes x max 8 = 496 connections
postgres         max_connections=256, minus superuser_reserved_connections

The comment above it claimed "64 cores × 8 = 512 to match aspnet-minimal's
Npgsql pool for fair cross-framework comparison"
, which is over the limit on its
own terms. Both the pool and the SQLite block predate #1242 (March/April); crud
was already in meta.json, so what that PR changed was only that the profile got
re-run and wrote this log.

Fix

The pool divides DATABASE_MAX_CONN by the process count, which entrypoint.sh
now exports, leaving headroom for superuser_reserved_connections.

Verified on a 32-CPU box, driving /crud/items/:id at 1024 connections:

connections held too many clients
before 256 asked (32 × 8) yes on the bench box (62 × 8 = 496)
after 226 (32 × 7, +2) 0

Two other things from the same log

  • No pool.on("error") handler. node-pg re-emits connection failures as
    'error' events; with nothing attached, each printed a full stack trace, which
    is what turned a connection problem into a 54MB artifact. Every sibling entry
    attaches one.
  • Unguarded SQLite open — 186 SQLITE_CANTOPEN stack traces (62 processes ×
    3 attempts). The harness mounts /data/dataset.json and /data/static only,
    never /data/benchmark.db, so the file is absent on every benchmarked profile.
    Guarded with existsSync, matching express and ultimate-express.

Validation

62 passed, 0 failed.

Worth a separate look

hono-bun is not alone in publishing 5xx — 31 profile-runs across the results
set do
, and it is not the worst. The top of that list:

php-fpm                baseline-h2-1024   5xx=1462506   rps=294864
symfony-spawn-franken  baseline-h2-256    5xx=381252    rps=28243
roadrunner             crud-4096          5xx=86954     rps=189233
vanilla-epoll          crud-4096          5xx=66317     rps=317372
vanilla-io_uring       crud-4096          5xx=63843     rps=338676

All under the 5% gate, all published as if clean.

🤖 Generated with Claude Code

The crud run left a 54MB log - site/static/logs/crud/4096/hono-bun.log, 1.9M
lines - and published crud-4096 with status_5xx=23308. Every other JS entry
publishes 0 there, and hono-bun's 218k rps trails bun/node/deno at 310-347k.
72,031 of those log lines are Postgres answering "sorry, too many clients
already".

The pool was a fixed max of 8 per process. entrypoint.sh starts one process per
CPU, and the crud profile hands the container the cpuset 1-31,65-95, so nproc
reports 62 on the benchmark box: 62 x 8 = 496 connections asked of a Postgres
running max_connections=256, which also reserves a few for the superuser. The
comment claimed "64 cores x 8 = 512 to match aspnet-minimal's Npgsql pool",
which was over the limit on its own terms.

The pool now divides DATABASE_MAX_CONN by the process count, which entrypoint.sh
exports, and leaves headroom for superuser_reserved_connections. Verified on a
32-CPU box: 226 connections held under crud load where the old code asked for
256, and nothing refused.

Two other things came out of the same log:

  - node-pg re-emits connection failures as 'error' events. With no handler
    attached each one printed a stack trace, which is what turned a connection
    problem into a 54MB artifact. Every sibling entry attaches one.

  - The SQLite open was unguarded, so all 62 processes printed three
    SQLITE_CANTOPEN stack traces each - 186 in that log. The harness mounts
    /data/dataset.json and /data/static only and never /data/benchmark.db, so
    the file is absent on every benchmarked profile. Guarded with existsSync,
    matching express and ultimate-express.

Validation: 62 passed, 0 failed.
MDA2AV added a commit that referenced this pull request Aug 22, 2026
…ssing profiles

  fastapi, uvicorn, fastpysgi-wsgi, mq-bridge-py   crud
  flask                                            static-tls, crud
  bottle                                           json-comp, crud

crud runs cache-aside on Redis everywhere, 200ms TTL, explicit delete on update.
The raw ASGI and WSGI entries key their dispatch on the path up to the last
slash, so /crud/items lands on "/crud/" and /crud/items/<id> on "/crud/items/",
and their method guards had to admit PUT.

bottle had no response compression at all, so json-comp is negotiated per
request in the handler - gzip level 1, nothing without Accept-Encoding.

Two pre-existing problems fixed along the way:

  - mq-bridge-py forks one serving process per core and each opened a pool of
    the whole DATABASE_MAX_CONN. The crud profile hands the container a 62-CPU
    cpuset, so that asked for 62 x 256 connections of a Postgres running
    max_connections=256. The budget is now divided by the worker count with
    headroom for superuser_reserved_connections. This is the same bug #1251
    fixed on hono-bun, and crud is the profile that would have exposed it.

  - flask answered /pipeline and /baseline11 with text/html. Flask defaults a
    bare body to text/html and neither handler set a content type, so all three
    Content-Type checks failed; the profiles want text/plain.

tags is a JSONB column, so it arrives as text unless a codec is set.

Validation: fastapi 58/0, uvicorn 58/0, fastpysgi-wsgi 58/0, mq-bridge-py 74/0,
flask 56/0, bottle 56/0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant