python frameworks: add the missing HTTP/1.1 profiles - #1254
Merged
Conversation
static, static-tls, json-tls, async-db, crud, api-4 and api-16. Each carried four endpoints; api-4 and api-16 rotate over /baseline11, /json and /async-db, so wiring Postgres brings all three profiles with it. json-tls and static-tls come from the launcher fastapi already uses: it spawns uvicorn twice, the second on port+1 with the cert and key. litestar and django ran uvicorn straight from CMD and now go through the same launcher. Static bodies are read from disk on every request - Starlette's StaticFiles, litestar's static router, and FileResponse for django, all of which stream the handle rather than holding the bytes. The encoding is left to the gzip middleware each entry already mounts. Postgres goes through asyncpg. uvicorn forks one worker per core and each gets its own pool, so DATABASE_MAX_CONN is divided by the worker count with headroom left for superuser_reserved_connections. tags is a JSONB column, so it arrives as text unless a codec is set. crud runs cache-aside on Redis with a 200ms TTL and an explicit delete on update. django wires both lazily on first use: its ASGI application has no startup hook that runs inside the worker's event loop. Starlette 1.x takes a lifespan rather than on_startup. Validation: 58 passed, 0 failed on all three.
Owner
Author
|
/benchmark-multiple --save |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
…ssing profiles
fastapi, uvicorn, fastpysgi-wsgi, mq-bridge-py crud
flask static-tls, crud
bottle json-comp, crud
crud runs cache-aside on Redis everywhere, 200ms TTL, explicit delete on update.
The raw ASGI and WSGI entries key their dispatch on the path up to the last
slash, so /crud/items lands on "/crud/" and /crud/items/<id> on "/crud/items/",
and their method guards had to admit PUT.
bottle had no response compression at all, so json-comp is negotiated per
request in the handler - gzip level 1, nothing without Accept-Encoding.
Two pre-existing problems fixed along the way:
- mq-bridge-py forks one serving process per core and each opened a pool of
the whole DATABASE_MAX_CONN. The crud profile hands the container a 62-CPU
cpuset, so that asked for 62 x 256 connections of a Postgres running
max_connections=256. The budget is now divided by the worker count with
headroom for superuser_reserved_connections. This is the same bug #1251
fixed on hono-bun, and crud is the profile that would have exposed it.
- flask answered /pipeline and /baseline11 with text/html. Flask defaults a
bare body to text/html and neither handler set a content type, so all three
Content-Type checks failed; the profiles want text/plain.
tags is a JSONB column, so it arrives as text unless a codec is set.
Validation: fastapi 58/0, uvicorn 58/0, fastpysgi-wsgi 58/0, mq-bridge-py 74/0,
flask 56/0, bottle 56/0.
Contributor
|
👋 Heads up! This PR modifies the following frameworks:
|
aiohttp static, static-tls, json-tls, async-db, crud, api-4, api-16 (6 -> 13) sanic static, async-db, crud, api-4, api-16 (6 -> 11) api-4 and api-16 rotate over /baseline11, /json and /async-db, so wiring Postgres brings all three profiles with it. aiohttp gets json-tls and static-tls from a second TCPSite on 8081 over the same runner. Every forked worker binds it exactly as they all bind 8080, so the listener is spread rather than parked on one process. sanic does not get them. It serves one port per app.run(); the documented multi-port form - repeated app.prepare() then Sanic.serve() - starts without error but the 8081 listener never answers, with or without a per-listener worker count. Its own TLS helper also raises KeyError on the arena's self-signed certificate, which has no subjectAltName, so a pre-built ssl.SSLContext is needed to get that far at all. Rather than ship a listener that does not serve, the two profiles are left unsubscribed. Static bodies are read from disk on every request - FileResponse for aiohttp, file_stream for sanic - and the encoding is left to the compression each entry already does. Postgres goes through asyncpg, one pool per worker with DATABASE_MAX_CONN divided by the worker count and headroom left for superuser_reserved_connections. crud runs cache-aside on Redis, 200ms TTL, explicit delete on update. tags is a JSONB column, so it arrives as text unless a codec is set. Validation: aiohttp 58/0, sanic 47/0.
api-4 and api-16 rotate over /baseline11, /json and /async-db, so wiring
Postgres brings all three. asyncpg, one pool per listener process, opened lazily
because launcher.py starts a process per port and there is no startup hook that
runs inside each one's event loop. The budget is divided by the worker count
with headroom for superuser_reserved_connections.
crud is deliberately left out. The collection route works, but a handler
declared with a path parameter does not also receive `receive`/`send`, so
/crud/items/{id} can neither read a PUT body nor write its own response - it
answered with an empty body. Shipping half of crud would fail validation, so it
is not subscribed.
Worth noting for anyone extending this entry: handlers that take `send` must
write the response themselves; a returned Response is silently ignored.
Validation: 66 passed, 1 failed - up from 63 passed, 1 failed on the unmodified
entry. The failure is [frag: POST, chunked], 1 of 140 offsets, and reproduces
identically without these changes.
Contributor
Benchmark ResultsFrameworks: 3 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 30,135 | 5959.3% | 5.1GiB | -1.4% | +112.5% |
| baseline | 4096 | 29,206 | 5923.5% | 6.0GiB | -1.2% | +93.5% |
| pipelined | 512 | 68,657 | 6247.9% | 5.0GiB | -2.8% | +108.3% |
| pipelined | 4096 | 55,483 | 6168.8% | 5.7GiB | -2.7% | +83.9% |
| limited-conn | 512 | 29,304 | 6016.6% | 5.1GiB | -1.8% | +104.0% |
| limited-conn | 4096 | 29,247 | 5941.7% | 6.0GiB | -1.4% | +93.5% |
| json | 4096 | 26,856 | 6004.9% | 5.9GiB | -1.2% | +90.3% |
| json-comp | 512 | 24,724 | 6033.8% | 5.4GiB | -1.4% | +100.0% |
| json-comp | 4096 | 24,224 | 5994.1% | 7.9GiB | -1.3% | +68.1% |
| json-comp | 16384 | 21,979 | 5836.9% | 12.2GiB | -0.5% | +29.8% |
| json-tls | 4096 | 25,017 | 6217.7% | 6.0GiB | NEW | NEW |
| upload | 32 | 640 | 2147.7% | 5.0GiB | -2.4% | +108.3% |
| upload | 256 | 643 | 6367.8% | 5.6GiB | -2.0% | +93.1% |
| api-4 | 256 | 2,354 | 367.9% | 440MiB | NEW | NEW |
| api-16 | 1024 | 7,580 | 1555.0% | 1.5GiB | NEW | NEW |
| static | 1024 | 13,459 | 7127.0% | 6.8GiB | NEW | NEW |
| static | 4096 | 12,731 | 6821.8% | 9.6GiB | NEW | NEW |
| static | 6800 | 12,372 | 6691.6% | 11.4GiB | NEW | NEW |
| static-tls | 1024 | 12,937 | 7023.3% | 7.1GiB | NEW | NEW |
| static-tls | 4096 | 12,103 | 6780.0% | 9.8GiB | NEW | NEW |
| static-tls | 6800 | 11,557 | 6586.9% | 11.8GiB | NEW | NEW |
| async-db | 1024 | 3,874 | 1428.2% | 5.3GiB | NEW | NEW |
| crud | 4096 | 17,342 | 5404.5% | 5.9GiB | NEW | NEW |
✅ litestar
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 314,484 | 6504.1% | 4.4GiB | -1.4% | +131.6% |
| baseline | 4096 | 316,185 | 6385.9% | 4.4GiB | -2.0% | +131.6% |
| pipelined | 512 | 379,038 | 6534.2% | 4.4GiB | -1.4% | +131.6% |
| pipelined | 4096 | 373,405 | 6530.3% | 4.6GiB | -2.8% | +119.0% |
| limited-conn | 512 | 288,035 | 6513.4% | 5.4GiB | -2.0% | +80.0% |
| limited-conn | 4096 | 290,401 | 6511.4% | 5.4GiB | -2.2% | +80.0% |
| json | 4096 | 268,753 | 6502.8% | 4.7GiB | -2.9% | +104.3% |
| json-comp | 512 | 133,682 | 6459.3% | 4.6GiB | -2.0% | +119.0% |
| json-comp | 4096 | 134,515 | 6359.7% | 4.6GiB | -2.1% | +119.0% |
| json-comp | 16384 | 132,367 | 6461.3% | 4.6GiB | -0.5% | +100.0% |
| json-tls | 4096 | 218,456 | 6489.1% | 4.9GiB | NEW | NEW |
| upload | 32 | 2,111 | 2546.4% | 4.4GiB | -0.7% | +131.6% |
| upload | 256 | 1,534 | 6448.5% | 4.6GiB | -1.3% | +109.1% |
| api-4 | 256 | 12,227 | 401.3% | 449MiB | NEW | NEW |
| api-16 | 1024 | 42,849 | 1611.3% | 1.5GiB | NEW | NEW |
| static | 1024 | 16,737 | 6397.4% | 5.7GiB | NEW | NEW |
| static | 4096 | 16,195 | 6420.1% | 6.9GiB | NEW | NEW |
| static | 6800 | 15,776 | 6407.4% | 8.0GiB | NEW | NEW |
| static-tls | 1024 | 16,565 | 6460.7% | 5.8GiB | NEW | NEW |
| static-tls | 4096 | 15,370 | 6237.4% | 8.9GiB | NEW | NEW |
| static-tls | 6800 | 14,820 | 6420.0% | 9.9GiB | NEW | NEW |
| async-db | 1024 | 96,581 | 6570.9% | 4.6GiB | NEW | NEW |
| crud | 4096 | 37,725 | 2553.4% | 4.6GiB | NEW | NEW |
✅ starlette
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 395,579 | 6533.6% | 3.5GiB | -3.6% | +150.0% |
| baseline | 4096 | 399,413 | 6399.8% | 3.5GiB | -3.2% | +133.3% |
| pipelined | 512 | 492,899 | 6546.5% | 3.5GiB | -2.5% | +150.0% |
| pipelined | 4096 | 487,988 | 6431.9% | 3.7GiB | -2.1% | +131.2% |
| limited-conn | 512 | 356,521 | 6518.4% | 4.6GiB | -1.2% | +76.9% |
| limited-conn | 4096 | 361,088 | 6329.6% | 4.6GiB | -1.4% | +76.9% |
| json | 4096 | 225,181 | 6382.3% | 3.8GiB | -1.5% | +111.1% |
| json-comp | 512 | 129,899 | 6416.6% | 3.7GiB | -1.5% | +131.2% |
| json-comp | 4096 | 130,650 | 6338.4% | 3.7GiB | -0.9% | +131.2% |
| json-comp | 16384 | 126,853 | 6070.5% | 3.7GiB | -0.9% | +117.6% |
| json-tls | 4096 | 185,089 | 6241.8% | 3.8GiB | NEW | NEW |
| upload | 32 | 2,132 | 2361.9% | 3.5GiB | ~0% | +133.3% |
| upload | 256 | 1,549 | 6335.5% | 3.8GiB | -0.6% | +123.5% |
| api-4 | 256 | 10,118 | 401.0% | 366MiB | NEW | NEW |
| api-16 | 1024 | 37,044 | 1609.6% | 1.2GiB | NEW | NEW |
| static | 1024 | 23,668 | 6747.8% | 4.0GiB | NEW | NEW |
| static | 4096 | 22,619 | 6630.8% | 4.8GiB | NEW | NEW |
| static | 6800 | 22,112 | 6583.5% | 5.0GiB | NEW | NEW |
| static-tls | 1024 | 22,998 | 6652.6% | 4.1GiB | NEW | NEW |
| static-tls | 4096 | 21,297 | 6530.9% | 5.0GiB | NEW | NEW |
| static-tls | 6800 | 20,546 | 6397.3% | 7.0GiB | NEW | NEW |
| async-db | 1024 | 88,740 | 6476.7% | 3.7GiB | NEW | NEW |
| crud | 4096 | 41,200 | 2564.2% | 3.6GiB | NEW | NEW |
Owner
Author
|
/benchmark-multiple -f aiohttp,sanic,blackbull,flask,bottle,fastapi,uvicorn,fastpysgi-wsgi,mq-bridge-py --save |
Contributor
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Contributor
Benchmark ResultsFrameworks: 9 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 568,388 | 6509.5% | 536MiB | -0.4% | +25.2% |
| baseline | 4096 | 580,906 | 6331.2% | 581MiB | -1.7% | +18.3% |
| pipelined | 512 | 1,181,574 | 6398.7% | 510MiB | +1.4% | +24.1% |
| pipelined | 4096 | 1,047,760 | 6388.5% | 636MiB | -0.6% | +0.6% |
| limited-conn | 512 | 486,962 | 6134.6% | 527MiB | +0.5% | +24.0% |
| limited-conn | 4096 | 506,418 | 6318.6% | 535MiB | -0.5% | +13.3% |
| json | 4096 | 290,502 | 6366.6% | 568MiB | +0.4% | +18.8% |
| json-comp | 512 | 101,155 | 6191.0% | 870MiB | +0.1% | +11.0% |
| json-comp | 4096 | 101,995 | 6349.6% | 1.6GiB | ~0% | ~0% |
| json-comp | 16384 | 100,526 | 6132.5% | 2.0GiB | +2.3% | -13.0% |
| json-tls | 4096 | 237,427 | 6391.7% | 931MiB | NEW | NEW |
| upload | 32 | 1,608 | 2154.9% | 683MiB | -2.7% | +11.2% |
| upload | 256 | 1,229 | 5404.0% | 1.1GiB | -1.0% | ~0% |
| api-4 | 256 | 12,165 | 398.0% | 90MiB | NEW | NEW |
| api-16 | 1024 | 45,852 | 1585.4% | 217MiB | NEW | NEW |
| static | 1024 | 93,454 | 6323.2% | 762MiB | NEW | NEW |
| static | 4096 | 89,973 | 6313.0% | 1.0GiB | NEW | NEW |
| static | 6800 | 87,840 | 6257.9% | 1.2GiB | NEW | NEW |
| static-tls | 1024 | 82,483 | 6252.2% | 892MiB | NEW | NEW |
| static-tls | 4096 | 79,947 | 6237.6% | 1.5GiB | NEW | NEW |
| static-tls | 6800 | 73,015 | 6175.0% | 2.0GiB | NEW | NEW |
| async-db | 1024 | 100,980 | 6114.3% | 684MiB | NEW | NEW |
| crud | 4096 | 41,824 | 2001.2% | 796MiB | NEW | NEW |
✅ sanic
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 591,772 | 6515.0% | 1.9GiB | -0.2% | +18.7% |
| baseline | 4096 | 599,731 | 6529.4% | 1.9GiB | -2.0% | +18.7% |
| pipelined | 512 | 1,443,942 | 6406.9% | 1.9GiB | -1.2% | +18.7% |
| pipelined | 4096 | 1,386,665 | 6375.9% | 1.9GiB | +1.1% | +18.7% |
| limited-conn | 512 | 509,868 | 6588.9% | 1.9GiB | ~0% | +18.7% |
| limited-conn | 4096 | 519,587 | 6381.8% | 1.9GiB | -0.7% | +18.7% |
| json | 4096 | 341,560 | 6283.8% | 1.9GiB | -0.8% | +18.7% |
| json-comp | 512 | 171,580 | 6462.7% | 1.9GiB | -0.4% | +18.7% |
| json-comp | 4096 | 171,366 | 6141.0% | 1.9GiB | -0.9% | +18.7% |
| json-comp | 16384 | 168,848 | 6062.7% | 2.0GiB | -0.3% | +17.6% |
| upload | 32 | 1,813 | 2595.2% | 1.9GiB | ~0% | +18.7% |
| upload | 256 | 1,232 | 6452.3% | 2.1GiB | -1.0% | +16.7% |
| api-4 | 256 | 13,541 | 400.5% | 204MiB | NEW | NEW |
| api-16 | 1024 | 50,763 | 1608.7% | 568MiB | NEW | NEW |
| static | 1024 | 18,785 | 6259.9% | 2.0GiB | NEW | NEW |
| static | 4096 | 17,739 | 6041.7% | 2.3GiB | NEW | NEW |
| static | 6800 | 17,594 | 5856.7% | 2.4GiB | NEW | NEW |
| async-db | 1024 | 108,506 | 6551.2% | 1.9GiB | NEW | NEW |
| crud | 4096 | 42,121 | 1952.2% | 2.0GiB | NEW | NEW |
✅ blackbull
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 292,704 | 6495.8% | 1.4GiB | +1.8% | ~0% |
| baseline | 4096 | 289,534 | 6288.7% | 1.4GiB | +2.0% | -6.7% |
| pipelined | 512 | 383,534 | 6512.1% | 1.4GiB | +0.2% | ~0% |
| pipelined | 4096 | 380,213 | 6336.5% | 1.5GiB | +1.6% | -6.3% |
| limited-conn | 512 | 247,809 | 6460.9% | 1.6GiB | +1.0% | ~0% |
| limited-conn | 4096 | 245,915 | 6307.2% | 2.0GiB | -0.2% | +5.3% |
| json | 4096 | 182,405 | 6324.5% | 1.8GiB | +0.1% | ~0% |
| json-comp | 512 | 101,379 | 6387.4% | 1.6GiB | +0.6% | ~0% |
| json-comp | 4096 | 101,151 | 6242.5% | 1.7GiB | +1.3% | ~0% |
| json-comp | 16384 | 99,451 | 6103.1% | 1.9GiB | +1.5% | +5.6% |
| json-tls | 4096 | 169,667 | 6247.8% | 2.7GiB | +0.7% | ~0% |
| upload | 32 | 1,027 | 2526.9% | 2.7GiB | -0.9% | -6.9% |
| upload | 256 | 624 | 6280.0% | 7.3GiB | -3.4% | -19.8% |
| api-4 | 256 | 7,689 | 399.9% | 411MiB | NEW | NEW |
| api-16 | 1024 | 26,494 | 1627.8% | 1.1GiB | NEW | NEW |
| static | 1024 | 198,224 | 6431.0% | 1.4GiB | +2.0% | ~0% |
| static | 4096 | 189,464 | 6310.3% | 1.5GiB | +1.4% | ~0% |
| static | 6800 | 189,090 | 6323.7% | 1.5GiB | +1.9% | -6.3% |
| static-tls | 1024 | 170,421 | 6439.1% | 1.8GiB | +0.6% | ~0% |
| static-tls | 4096 | 162,397 | 6317.9% | 3.0GiB | -1.3% | ~0% |
| static-tls | 6800 | 156,701 | 6276.1% | 4.1GiB | -2.7% | ~0% |
| async-db | 1024 | 30,951 | 2404.0% | 2.4GiB | NEW | NEW |
| baseline-h2 | 256 | 186,119 | 6349.1% | 1.6GiB | +0.4% | ~0% |
| baseline-h2 | 1024 | 180,004 | 6429.6% | 1.9GiB | +2.1% | ~0% |
| static-h2 | 256 | 105,595 | 6358.7% | 1.6GiB | -0.3% | ~0% |
| static-h2 | 1024 | 103,663 | 6434.8% | 1.9GiB | +1.2% | -5.0% |
| echo-ws | 512 | 669,184 | 6547.7% | 1.4GiB | +0.8% | ~0% |
| echo-ws | 4096 | 709,419 | 6626.9% | 1.6GiB | +3.8% | ~0% |
| echo-ws | 16384 | 658,141 | 6173.8% | 1.9GiB | +2.0% | -9.5% |
✅ flask
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 119,984 | 5741.6% | 3.3GiB | +4.2% | +3.1% |
| baseline | 4096 | 84,858 | 3092.1% | 3.2GiB | +9.8% | ~0% |
| pipelined | 512 | 118,792 | 5419.1% | 3.3GiB | +2.0% | +3.1% |
| pipelined | 4096 | 102,036 | 3333.2% | 3.2GiB | +31.3% | ~0% |
| limited-conn | 512 | 117,681 | 5793.8% | 3.3GiB | +3.2% | +3.1% |
| limited-conn | 4096 | 104,623 | 4410.7% | 3.3GiB | -2.6% | +3.1% |
| json | 4096 | 100,754 | 5778.7% | 3.2GiB | +22.7% | ~0% |
| json-comp | 512 | 70,573 | 6399.7% | 3.2GiB | +2.4% | ~0% |
| json-comp | 4096 | 65,853 | 5144.0% | 3.2GiB | -0.6% | ~0% |
| json-comp | 16384 | 55,928 | 4125.7% | 3.2GiB | +1.5% | ~0% |
| json-tls | 4096 | 6,497 | 1725.3% | 3.3GiB | -4.9% | ~0% |
| upload | 32 | 1,297 | 3454.3% | 3.3GiB | +3.4% | ~0% |
| upload | 256 | 1,474 | 6319.1% | 3.4GiB | +4.2% | +3.0% |
| api-4 | 256 | 5,649 | 380.3% | 264MiB | -10.7% | -1.5% |
| api-16 | 1024 | 20,059 | 1480.7% | 897MiB | +17.3% | -2.1% |
| static | 1024 | 67,055 | 4718.0% | 3.2GiB | +31.3% | ~0% |
| static | 4096 | 63,304 | 4038.0% | 3.2GiB | +29.3% | ~0% |
| static | 6800 | 41,967 | 2726.7% | 3.2GiB | -7.9% | ~0% |
| static-tls | 1024 | 9,246 | 3191.2% | 3.3GiB | NEW | NEW |
| static-tls | 4096 | 8,326 | 2710.2% | 3.3GiB | NEW | NEW |
| static-tls | 6800 | 6,093 | 2156.7% | 3.3GiB | NEW | NEW |
| async-db | 1024 | 47,347 | 5332.0% | 3.3GiB | +11.7% | ~0% |
| crud | 4096 | 42,209 | 4001.7% | 3.4GiB | NEW | NEW |
✅ bottle
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 114,700 | 4011.0% | 2.5GiB | -25.6% | ~0% |
| baseline | 4096 | 109,539 | 2997.5% | 2.5GiB | +1.5% | ~0% |
| pipelined | 512 | 119,728 | 3841.3% | 2.5GiB | -22.5% | ~0% |
| pipelined | 4096 | 117,430 | 2946.7% | 2.5GiB | -5.5% | ~0% |
| limited-conn | 512 | 119,417 | 4190.4% | 2.5GiB | -21.6% | ~0% |
| limited-conn | 4096 | 124,815 | 4429.1% | 2.5GiB | +4.6% | ~0% |
| json | 4096 | 112,849 | 4489.1% | 2.6GiB | +6.6% | +4.0% |
| json-comp | 512 | 101,347 | 6327.1% | 2.7GiB | NEW | NEW |
| json-comp | 4096 | 93,267 | 4927.0% | 2.7GiB | NEW | NEW |
| json-comp | 16384 | 68,458 | 3240.5% | 2.6GiB | NEW | NEW |
| json-tls | 4096 | 5,935 | 1479.2% | 2.7GiB | -17.0% | +8.0% |
| upload | 32 | 926 | 3460.5% | 4.1GiB | +5.9% | +5.1% |
| upload | 256 | 993 | 5880.7% | 4.1GiB | +6.4% | ~0% |
| api-4 | 256 | 7,004 | 377.0% | 225MiB | -26.6% | +3.7% |
| api-16 | 1024 | 25,288 | 1440.8% | 744MiB | -1.1% | +4.5% |
| static | 1024 | 81,253 | 4148.5% | 2.5GiB | -0.6% | ~0% |
| static | 4096 | 80,774 | 3937.5% | 2.5GiB | +15.6% | ~0% |
| static | 6800 | 68,276 | 2873.8% | 2.5GiB | +7.0% | ~0% |
| static-tls | 1024 | 9,352 | 2896.8% | 2.5GiB | +2.7% | -3.8% |
| static-tls | 4096 | 7,955 | 2352.1% | 2.5GiB | -7.1% | ~0% |
| static-tls | 6800 | 7,840 | 2259.5% | 2.5GiB | -1.1% | ~0% |
| async-db | 1024 | 55,704 | 5164.3% | 2.6GiB | +2.6% | ~0% |
| crud | 4096 | 42,411 | 3197.9% | 2.7GiB | NEW | NEW |
✅ fastapi
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 300,308 | 6522.4% | 4.8GiB | -6.3% | +6.7% |
| baseline | 4096 | 300,032 | 6420.8% | 4.9GiB | -4.6% | +6.5% |
| pipelined | 512 | 344,488 | 6515.4% | 4.9GiB | -5.5% | +8.9% |
| pipelined | 4096 | 338,598 | 6516.7% | 5.1GiB | -5.9% | +8.5% |
| limited-conn | 512 | 275,247 | 6472.9% | 6.1GiB | -3.1% | +7.0% |
| limited-conn | 4096 | 276,844 | 6375.2% | 6.1GiB | -3.5% | +5.2% |
| json | 4096 | 153,735 | 6465.7% | 5.1GiB | -2.2% | +6.2% |
| json-comp | 512 | 102,045 | 6402.9% | 5.1GiB | +8.9% | +8.5% |
| json-comp | 4096 | 102,024 | 6272.3% | 5.1GiB | +8.6% | +8.5% |
| json-comp | 16384 | 98,111 | 6453.0% | 5.3GiB | +6.4% | +8.2% |
| json-tls | 4096 | 133,843 | 6206.3% | 5.1GiB | -1.6% | +6.2% |
| upload | 32 | 2,103 | 2532.4% | 4.9GiB | -1.0% | +6.5% |
| upload | 256 | 1,562 | 6477.8% | 5.1GiB | +1.7% | +6.2% |
| api-4 | 256 | 8,351 | 401.0% | 512MiB | -1.2% | +3.6% |
| api-16 | 1024 | 30,307 | 1609.5% | 1.8GiB | +0.3% | +5.9% |
| static | 1024 | 23,488 | 6717.4% | 5.4GiB | +6.5% | +1.9% |
| static | 4096 | 22,193 | 6809.5% | 6.1GiB | +6.1% | -10.3% |
| static | 6800 | 21,867 | 6799.4% | 6.5GiB | +7.6% | -17.7% |
| static-tls | 1024 | 22,486 | 6724.4% | 5.8GiB | -3.9% | +11.5% |
| static-tls | 4096 | 20,809 | 6686.1% | 7.0GiB | -2.8% | +14.8% |
| static-tls | 6800 | 20,078 | 6704.9% | 8.0GiB | -0.2% | +3.9% |
| async-db | 1024 | 72,394 | 5999.7% | 5.2GiB | -5.2% | +8.3% |
| crud | 4096 | 41,838 | 2711.1% | 5.0GiB | NEW | NEW |
✅ uvicorn
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 646,675 | 6632.2% | 3.2GiB | -6.5% | ~0% |
| baseline | 4096 | 674,231 | 6430.7% | 3.3GiB | -7.3% | +3.1% |
| pipelined | 512 | 803,308 | 6675.9% | 3.2GiB | +0.7% | ~0% |
| pipelined | 4096 | 813,948 | 6500.6% | 3.4GiB | +1.8% | ~0% |
| limited-conn | 512 | 545,434 | 6567.6% | 4.9GiB | -4.4% | ~0% |
| limited-conn | 4096 | 566,251 | 6414.6% | 5.0GiB | -3.7% | ~0% |
| json | 4096 | 473,340 | 6348.4% | 3.8GiB | +1.7% | ~0% |
| json-comp | 512 | 273,984 | 6457.2% | 3.6GiB | +1.1% | ~0% |
| json-comp | 4096 | 277,383 | 6340.9% | 3.6GiB | +2.2% | ~0% |
| json-comp | 16384 | 269,838 | 6110.5% | 3.6GiB | +2.3% | ~0% |
| json-tls | 4096 | 342,025 | 6287.0% | 3.5GiB | +1.5% | ~0% |
| upload | 32 | 1,674 | 2722.8% | 3.3GiB | +0.4% | +3.1% |
| upload | 256 | 1,187 | 6329.8% | 3.5GiB | +1.1% | +2.9% |
| api-4 | 256 | 17,898 | 400.2% | 397MiB | -27.8% | -11.2% |
| api-16 | 1024 | 65,798 | 1612.1% | 1.3GiB | +5.1% | ~0% |
| static | 1024 | 655,898 | 6579.1% | 3.2GiB | +2.3% | ~0% |
| static | 4096 | 628,280 | 6501.0% | 3.3GiB | +2.5% | ~0% |
| static | 6800 | 580,297 | 6405.5% | 3.4GiB | +1.8% | ~0% |
| static-tls | 1024 | 402,681 | 6481.1% | 3.4GiB | -1.1% | ~0% |
| static-tls | 4096 | 344,785 | 6360.1% | 3.9GiB | ~0% | ~0% |
| static-tls | 6800 | 318,833 | 6201.3% | 4.3GiB | -0.6% | ~0% |
| async-db | 1024 | 126,817 | 6115.6% | 3.5GiB | +95.6% | +2.9% |
| crud | 4096 | 42,455 | 1984.4% | 3.7GiB | NEW | NEW |
✅ fastpysgi-wsgi
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 1,404,276 | 6815.9% | 363MiB | +1.7% | -1.1% |
| baseline | 4096 | 1,451,335 | 6279.4% | 852MiB | +3.6% | +0.7% |
| pipelined | 512 | 3,181,300 | 6698.5% | 351MiB | -7.8% | +0.9% |
| pipelined | 4096 | 2,769,939 | 6470.6% | 851MiB | +3.3% | +0.7% |
| limited-conn | 512 | 1,079,618 | 6238.9% | 405MiB | +3.4% | -0.2% |
| limited-conn | 4096 | 1,194,424 | 6676.9% | 506MiB | +2.7% | -0.2% |
| json | 4096 | 760,527 | 6293.8% | 846MiB | +3.8% | +0.5% |
| json-comp | 512 | 366,870 | 5935.2% | 533MiB | +2.7% | -10.1% |
| json-comp | 4096 | 379,027 | 6270.2% | 795MiB | +3.3% | +1.0% |
| json-comp | 16384 | 380,300 | 6264.8% | 934MiB | +2.5% | -2.9% |
| json-tls | 4096 | 608,966 | 6492.6% | 657MiB | +1.8% | -0.3% |
| upload | 32 | 1,865 | 1825.8% | 2.1GiB | -1.9% | ~0% |
| upload | 256 | 1,456 | 4997.8% | 4.0GiB | -1.2% | -9.1% |
| api-4 | 256 | 15,640 | 338.0% | 87MiB | -21.3% | -1.1% |
| api-16 | 1024 | 56,595 | 1260.9% | 249MiB | +2.7% | +1.6% |
| static | 1024 | 1,361,554 | 6528.8% | 419MiB | +2.0% | +0.5% |
| static | 4096 | 1,353,741 | 6395.8% | 870MiB | +3.4% | +1.5% |
| static | 6800 | 1,333,554 | 6376.2% | 1.2GiB | +3.3% | ~0% |
| static-tls | 1024 | 850,165 | 6543.8% | 700MiB | -0.7% | +28.0% |
| static-tls | 4096 | 555,719 | 6481.5% | 1.9GiB | ~0% | ~0% |
| static-tls | 6800 | 519,771 | 6400.9% | 1.9GiB | +1.1% | -62.7% |
| async-db | 1024 | 73,203 | 4381.3% | 854MiB | +4.0% | -3.6% |
| crud | 4096 | 44,823 | 2262.7% | 1.9GiB | NEW | NEW |
✅ mq-bridge-py
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline | 512 | 808,248 | 6650.2% | 2.9GiB | -3.8% | ~0% |
| baseline | 4096 | 1,175,117 | 6616.8% | 3.6GiB | -2.8% | ~0% |
| pipelined | 512 | 1,545,133 | 6494.6% | 2.8GiB | -2.4% | ~0% |
| pipelined | 4096 | 2,621,213 | 6507.8% | 3.2GiB | -1.3% | ~0% |
| limited-conn | 512 | 662,149 | 6610.2% | 3.3GiB | -1.3% | -2.9% |
| limited-conn | 4096 | 938,568 | 6567.8% | 5.1GiB | -1.6% | -1.9% |
| json | 4096 | 326,864 | 6516.0% | 3.6GiB | -1.4% | ~0% |
| json-comp | 512 | 58,835 | 6430.6% | 3.3GiB | -1.7% | ~0% |
| json-comp | 4096 | 95,670 | 6461.3% | 4.8GiB | -3.2% | ~0% |
| json-comp | 16384 | 130,684 | 6263.0% | 6.3GiB | +10.4% | -1.6% |
| json-tls | 4096 | 330,284 | 6692.6% | 4.1GiB | -0.5% | ~0% |
| upload | 32 | 1,112 | 3217.4% | 18.8GiB | ~0% | -13.0% |
| upload | 256 | 1,090 | 6311.2% | 32.6GiB | -0.3% | +9.8% |
| api-4 | 256 | 8,748 | 401.5% | 2.9GiB | -0.3% | ~0% |
| api-16 | 1024 | 35,639 | 1726.3% | 3.6GiB | -0.3% | ~0% |
| static | 1024 | 342,458 | 6630.8% | 3.7GiB | ~0% | ~0% |
| static | 4096 | 372,816 | 6543.2% | 7.4GiB | +0.2% | +1.4% |
| static | 6800 | 355,921 | 6499.9% | 10.3GiB | ~0% | +1.0% |
| static-tls | 1024 | 298,592 | 6627.7% | 3.8GiB | ~0% | ~0% |
| static-tls | 4096 | 314,673 | 6501.2% | 7.5GiB | -0.4% | ~0% |
| static-tls | 6800 | 301,999 | 6536.4% | 9.0GiB | ~0% | -15.1% |
| async-db | 1024 | 77,262 | 5373.2% | 5.2GiB | -0.3% | ~0% |
| crud | 4096 | 41,256 | 3188.4% | 5.9GiB | NEW | NEW |
| baseline-h2 | 256 | 1,011,821 | 6603.7% | 13.2GiB | +0.2% | -1.5% |
| baseline-h2 | 1024 | 918,646 | 6560.7% | 20.8GiB | -1.1% | -1.0% |
| static-h2 | 256 | 234,833 | 6595.5% | 7.1GiB | ~0% | ~0% |
| static-h2 | 1024 | 266,878 | 6562.7% | 9.6GiB | +0.4% | +1.1% |
| baseline-h2c | 256 | 975,434 | 6655.8% | 11.0GiB | +1.1% | -13.4% |
| baseline-h2c | 1024 | 909,931 | 6669.6% | 19.7GiB | +0.6% | -0.5% |
| baseline-h2c | 4096 | 866,580 | 6530.3% | 26.0GiB | -0.7% | -0.8% |
| json-h2c | 1024 | 323,223 | 6623.4% | 10.1GiB | +0.8% | ~0% |
| json-h2c | 4096 | 314,368 | 6506.8% | 15.4GiB | ~0% | -0.6% |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Python sweep. Twelve entries gain profiles; three are deliberately left
short and are explained below.
pyronovawas already complete.How the pieces were done
json-tls / static-tls come from the launcher
fastapialready used — uvicornspawned twice, the second on port+1 with the cert and key. litestar and django
ran uvicorn straight from
CMDand now share it. aiohttp uses a secondTCPSiteon the same runner, bound by every forked worker.static bodies are read from disk on every request: Starlette's
StaticFiles,litestar's static router,
FileResponsefor django and aiohttp,file_streamfor sanic. The encoding is left to the compression each entry already does.
Postgres is asyncpg (psycopg where the entry already used it), one pool per
worker with
DATABASE_MAX_CONNdivided by the worker count and headroom leftfor
superuser_reserved_connections. crud runs cache-aside on Redis, 200msTTL, explicit delete on update.
Pre-existing bugs fixed along the way
the whole
DATABASE_MAX_CONN. On the crud profile's 62-CPU cpuset that asksfor 62 × 256 connections of a Postgres running
max_connections=256— thesame bug hono-bun: size the Postgres pool to the process count #1251 fixed on hono-bun, and crud is exactly the profile that would
have exposed it.
/pipelineand/baseline11withtext/html. Flaskdefaults a bare body to that and neither handler set a content type, so all
three Content-Type checks failed.
Left short, on purpose
json-tls/static-tls. It serves one port perapp.run(); thedocumented multi-port form (repeated
app.prepare()thenSanic.serve())starts without error but 8081 never answers. Its own TLS helper also raises
KeyError: 'subjectAltName'on the arena's self-signed certificate, so apre-built
ssl.SSLContextis needed to get that far at all.crud. The collection route works, but a handler declaredwith a path parameter does not also receive
receive/send, so/crud/items/{id}can neither read a PUT body nor write a response.(static, static-tls and TCP fragmentation, all answering empty), so there was
no clean baseline to add crud against. Needs its own investigation.
Also worth a look
uvicorn,fastpysgi-asgiandfastpysgi-wsgiall preload static file bodiesinto a
STATIC_FILESdict at startup and serve from memory. The static profilesforbid that in every mode, including
engine, which all three are. I havenot touched their static path — all three already publish
staticandstatic-tls, and fixing it would move those numbers — but it is the sameviolation class as go-fasthttp in #1253 and swoole in #1244.
Validation
starlette, litestar, django, aiohttp, fastapi, uvicorn, fastpysgi-wsgi 58/0;
mq-bridge-py 74/0; flask, bottle 56/0; sanic 47/0; blackbull 66/1,
where the one failure is
[frag: POST, chunked]and reproduces identically at63/1 on the unmodified entry.
Still open
robyn(7 profiles),slimewebandbjoern(3 each), plus the three notedabove.
🤖 Generated with Claude Code