Skip to content

[trillium.rs] update dependencies and use websocket message coalescing - #1243

Open
jbr wants to merge 6 commits into
MDA2AV:mainfrom
jbr:trillium-websocket-feed
Open

[trillium.rs] update dependencies and use websocket message coalescing#1243
jbr wants to merge 6 commits into
MDA2AV:mainfrom
jbr:trillium-websocket-feed

Conversation

@jbr

@jbr jbr commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Description

This PR updates trillium and trillium-tuned to use websocket message coalescing, as well as updates trillium dependencies to pick up upstream improvements.


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,... Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test> Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp-11-kestrel

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@jbr

jbr commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark-multiple --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@MDA2AV

MDA2AV commented Aug 21, 2026

Copy link
Copy Markdown
Owner

/benchmark-multiple --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@MDA2AV
MDA2AV force-pushed the trillium-websocket-feed branch from 6de28cf to f6ee0f9 Compare August 21, 2026 23:15
Repository owner deleted a comment from github-actions Bot Aug 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Frameworks: 2 | Test: all tests

trillium

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 604,570 5905.4% 139MiB -0.4% +3.7%
baseline 4096 585,828 6019.5% 221MiB -11.7% +10.5%
pipelined 512 2,667,538 6689.8% 112MiB +3.3% -10.4%
pipelined 4096 2,804,885 6557.4% 162MiB +1.7% -30.8%
limited-conn 512 475,874 2422.4% 211MiB -5.2% -2.3%
limited-conn 4096 485,401 2382.9% 214MiB +0.4% -4.5%
json 4096 566,570 4534.5% 311MiB +5.7% +4.0%
json-comp 512 121,004 6323.4% 784MiB +2.1% +5.0%
json-comp 4096 111,002 6264.8% 814MiB -7.4% +5.3%
json-comp 16384 114,623 6206.0% 769MiB +1.7% +1.9%
json-tls 4096 554,254 6412.6% 415MiB -9.5% +0.2%
upload 32 2,689 1930.0% 107MiB +1.6% -1.8%
upload 256 3,044 3119.5% 204MiB -0.3% ~0%
api-4 256 57,519 400.2% 64MiB +3.5% -5.9%
api-16 1024 185,207 1510.0% 141MiB +0.8% ~0%
static 1024 24,316 3633.5% 375MiB +18.2% -23.8%
static 4096 20,477 2543.4% 432MiB -13.8% -4.6%
static 6800 22,993 3918.0% 445MiB +15.6% -30.1%
static-tls 1024 25,032 3153.9% 355MiB -1.1% -3.0%
static-tls 4096 18,341 3611.5% 545MiB -20.1% +9.7%
static-tls 6800 18,043 3587.6% 818MiB -7.3% +13.3%
async-db 1024 216,440 3178.6% 240MiB +0.6% -10.8%
crud 4096 462,400 4122.3% 338MiB -0.5% -1.5%
fortunes 1024 219,169 5388.8% 305MiB -0.2% +1.0%
baseline-h2 256 1,132,063 5451.9% 1.7GiB -13.5% ~0%
baseline-h2 1024 1,049,629 5660.9% 5.8GiB -16.0% -12.1%
static-h2 256 22,034 3558.8% 1.2GiB +5.1% ~0%
static-h2 1024 18,354 4127.3% 2.2GiB -1.3% ~0%
baseline-h3 64 441,602 2535.1% 250MiB -1.6% +7.3%
static-h3 64 18,344 3972.1% 600MiB -1.4% -7.0%
gateway-64 512 82,849 5328.2% 2.8GiB -2.4% ~0%
gateway-64 1024 82,282 5008.9% 5.1GiB +3.6% -1.9%
gateway-h3 64 82,810 4120.5% 933MiB -1.0% -1.2%
gateway-h3 256 61,590 2839.6% 2.3GiB -0.9% ~0%
unary-grpc 256 673,280 5459.5% 635MiB -17.1% +10.1%
unary-grpc 1024 646,600 5695.7% 2.0GiB -15.6% +5.3%
unary-grpc-tls 256 690,836 5457.4% 567MiB -16.3% -35.5%
unary-grpc-tls 1024 722,016 5508.5% 2.1GiB +3.6% +5.0%
stream-grpc 64 2,137,000 6227.2% 573MiB -3.5% -10.7%
stream-grpc-tls 64 2,292,000 6210.9% 674MiB +1.6% +12.7%
echo-ws 512 438,837 3086.0% 351MiB -1.0% +2.9%
echo-ws 4096 353,276 3904.4% 550MiB -0.2% -17.5%
echo-ws 16384 355,709 3427.3% 733MiB +2.1% -40.3%
echo-ws-pipeline 512 9,128,259 3634.2% 269MiB +97.9% -18.5%
echo-ws-pipeline 4096 8,051,852 4028.8% 922MiB +61.9% +48.0%
echo-ws-pipeline 16384 7,719,356 3346.8% 832MiB +60.5% -32.3%

trillium-tuned

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 602,812 6384.0% 223MiB +2.1% -14.2%
baseline 4096 637,119 6204.6% 418MiB +3.0% +13.6%
pipelined 512 2,718,400 6697.9% 173MiB -0.8% -19.9%
pipelined 4096 2,633,349 6344.2% 390MiB -6.6% +22.3%
limited-conn 512 584,893 5569.1% 235MiB +11.4% -10.6%
limited-conn 4096 600,639 6229.7% 506MiB +18.5% +35.7%
json 4096 599,620 6112.1% 528MiB +16.7% +36.8%
json-comp 512 109,212 5791.2% 720MiB +7309.2% -81.0%
json-comp 4096 112,238 6282.1% 1.0GiB +7614.0% -75.0%
json-comp 16384 101,450 6465.5% 1.4GiB +6969.7% -65.0%
json-tls 4096 556,725 6317.6% 491MiB -4.3% +2.9%
upload 32 3,026 1640.9% 200MiB +0.6% -19.7%
upload 256 3,107 3155.7% 228MiB +1.2% -17.1%
api-4 256 55,313 393.6% 78MiB -8.9% +5.4%
api-16 1024 190,886 1545.9% 177MiB +9.9% +14.9%
static 1024 627,681 6510.0% 196MiB +44830.6% -95.2%
static 4096 609,839 6381.7% 374MiB +43678.8% -90.6%
static 6800 533,451 6178.2% 498MiB +38167.6% -87.8%
static-tls 1024 597,328 6432.6% 348MiB +3.6% -3.6%
static-tls 4096 517,615 6237.9% 566MiB -0.6% +2.9%
static-tls 6800 571,420 6356.3% 746MiB +9.2% -0.4%
async-db 1024 237,577 3568.7% 270MiB -16.7% -7.5%
crud 4096 503,038 4368.2% 486MiB -1.5% +23.7%
fortunes 1024 169,345 4528.2% 380MiB NEW NEW
baseline-h2 256 1,182,087 5462.0% 1.2GiB -46.1% +30.0%
baseline-h2 1024 1,278,369 5609.8% 3.8GiB -35.1% -20.8%
static-h2 256 618,493 5668.7% 826MiB NEW -84.2%
static-h2 1024 575,832 5849.9% 2.6GiB NEW -48.0%
baseline-h3 64 456,153 2433.5% 328MiB +277.5% +46.4%
static-h3 64 248,130 2177.6% 591MiB +477073.1% +94.4%
gateway-64 512 76,505 5726.0% 2.9GiB NEW NEW
gateway-64 1024 75,413 5580.5% 5.3GiB NEW NEW
gateway-h3 64 83,609 4041.0% 968MiB NEW NEW
gateway-h3 256 60,906 2804.4% 2.4GiB NEW NEW
unary-grpc 256 738,569 5485.0% 966MiB NEW NEW
unary-grpc 1024 718,670 5747.7% 3.1GiB NEW NEW
unary-grpc-tls 256 768,687 5415.1% 1.2GiB NEW NEW
unary-grpc-tls 1024 779,719 5824.0% 3.8GiB NEW NEW
stream-grpc 64 2,085,000 3725.3% 624MiB NEW NEW
stream-grpc-tls 64 2,086,000 3728.2% 594MiB NEW NEW
echo-ws 512 1,644,124 6557.0% 352MiB -3.1% -2.8%
echo-ws 4096 425,725 6503.7% 895MiB +1.8% -4.6%
echo-ws 16384 406,426 6296.7% 3.0GiB +1.6% +11.1%
echo-ws-pipeline 512 14,857,936 6589.3% 354MiB +212.3% -6.1%
echo-ws-pipeline 4096 11,000,723 6315.9% 963MiB +136.0% +6.1%
echo-ws-pipeline 16384 6,936,953 6299.3% 3.0GiB +64.1% +11.1%

@jbr

jbr commented Aug 22, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark-multiple --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Frameworks: 2 | Test: all tests

trillium

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 663,746 5983.3% 139MiB +9.3% +3.7%
baseline 4096 604,856 5671.8% 194MiB -8.8% -3.0%
pipelined 512 2,633,650 6705.8% 110MiB +2.0% -12.0%
pipelined 4096 2,862,768 6568.3% 181MiB +3.8% -22.6%
limited-conn 512 501,795 2371.2% 205MiB ~0% -5.1%
limited-conn 4096 478,485 2515.1% 210MiB -1.0% -6.2%
json 4096 585,350 4740.9% 297MiB +9.2% -0.7%
json-comp 512 114,989 6097.1% 734MiB -3.0% -1.7%
json-comp 4096 115,249 6261.8% 765MiB -3.8% -1.0%
json-comp 16384 112,553 6165.7% 790MiB -0.2% +4.6%
json-tls 4096 540,273 6335.5% 393MiB -11.8% -5.1%
upload 32 2,653 1969.0% 110MiB +0.2% +0.9%
upload 256 3,059 3310.0% 192MiB +0.2% -5.9%
api-4 256 57,476 400.3% 72MiB +3.4% +5.9%
api-16 1024 185,011 1512.3% 130MiB +0.7% -7.8%
static 1024 20,058 4017.5% 366MiB -2.5% -25.6%
static 4096 24,536 3799.4% 390MiB +3.3% -13.9%
static 6800 20,380 2922.5% 519MiB +2.5% -18.5%
static-tls 1024 21,880 3340.2% 392MiB -13.6% +7.1%
static-tls 4096 22,090 4318.5% 516MiB -3.7% +3.8%
static-tls 6800 19,908 2713.5% 667MiB +2.2% -7.6%
async-db 1024 216,324 3214.6% 245MiB +0.5% -8.9%
crud 4096 468,423 4157.1% 332MiB +0.8% -3.2%
fortunes 1024 216,461 5180.3% 312MiB -1.5% +3.3%
baseline-h2 256 1,141,901 5415.7% 1.1GiB -12.8% -35.3%
baseline-h2 1024 1,236,165 5497.8% 2.9GiB -1.1% -56.1%
static-h2 256 21,933 3494.4% 1.1GiB +4.6% -8.3%
static-h2 1024 20,274 3218.0% 1.8GiB +9.0% -18.2%
baseline-h3 64 448,335 2470.2% 232MiB -0.1% -0.4%
static-h3 64 18,523 4198.8% 622MiB -0.4% -3.6%
gateway-64 512 83,000 5070.4% 2.7GiB -2.2% -3.6%
gateway-64 1024 77,432 5247.3% 5.2GiB -2.5% ~0%
gateway-h3 64 85,684 3922.0% 940MiB +2.4% -0.4%
gateway-h3 256 62,329 2768.9% 2.3GiB +0.3% ~0%
unary-grpc 256 714,218 5431.8% 466MiB -12.1% -19.2%
unary-grpc 1024 705,580 5638.3% 1.4GiB -7.9% -26.3%
unary-grpc-tls 256 760,233 5386.3% 415MiB -7.9% -52.8%
unary-grpc-tls 1024 621,288 5665.6% 1.4GiB -10.8% -30.0%
stream-grpc 64 2,171,000 6162.3% 592MiB -2.0% -7.8%
stream-grpc-tls 64 1,835,000 6234.2% 629MiB -18.7% +5.2%
echo-ws 512 435,266 3092.2% 348MiB -1.8% +2.1%
echo-ws 4096 356,484 3944.7% 835MiB +0.7% +25.2%
echo-ws 16384 354,753 3359.7% 832MiB +1.8% -32.3%
echo-ws-pipeline 512 9,074,489 3820.9% 338MiB +96.7% +2.4%
echo-ws-pipeline 4096 7,915,168 4620.0% 616MiB +59.2% -1.1%
echo-ws-pipeline 16384 8,006,288 4102.5% 973MiB +66.5% -20.8%

trillium-tuned

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 660,476 6315.9% 222MiB +11.8% -14.6%
baseline 4096 584,296 6393.8% 412MiB -5.6% +12.0%
pipelined 512 2,723,929 6629.9% 176MiB -0.6% -18.5%
pipelined 4096 2,666,520 6348.3% 388MiB -5.4% +21.6%
limited-conn 512 523,016 5558.6% 231MiB -0.4% -12.2%
limited-conn 4096 570,150 6047.9% 410MiB +12.5% +9.9%
json 4096 594,303 6301.3% 529MiB +15.6% +37.0%
json-comp 512 117,125 5740.6% 759MiB +7846.1% -80.0%
json-comp 4096 115,432 6283.3% 1.0GiB +7833.5% -75.0%
json-comp 16384 99,652 6098.3% 1.5GiB +6844.4% -62.5%
json-tls 4096 587,268 6442.4% 489MiB +0.9% +2.5%
upload 32 3,014 1605.7% 199MiB +0.2% -20.1%
upload 256 3,122 3179.8% 228MiB +1.7% -17.1%
api-4 256 56,113 394.5% 80MiB -7.6% +8.1%
api-16 1024 186,681 1495.3% 168MiB +7.4% +9.1%
static 1024 551,114 6496.4% 194MiB +39349.8% -95.3%
static 4096 545,528 6498.4% 368MiB +39062.1% -90.8%
static 6800 592,435 6299.4% 539MiB +42398.9% -86.8%
static-tls 1024 582,611 6440.8% 356MiB +1.0% -1.4%
static-tls 4096 562,235 6248.5% 548MiB +8.0% -0.4%
static-tls 6800 506,932 6090.2% 723MiB -3.1% -3.5%
async-db 1024 235,680 3504.0% 270MiB -17.4% -7.5%
crud 4096 503,441 4412.4% 488MiB -1.5% +24.2%
fortunes 1024 167,908 4475.4% 384MiB NEW NEW
baseline-h2 256 1,369,900 5264.6% 824MiB -37.5% -12.8%
baseline-h2 1024 1,378,086 5552.6% 2.3GiB -30.0% -52.1%
static-h2 256 658,768 5705.2% 706MiB NEW -86.5%
static-h2 1024 611,706 5859.4% 1.9GiB NEW -62.0%
baseline-h3 64 459,735 2440.7% 333MiB +280.5% +48.7%
static-h3 64 266,107 2411.5% 614MiB +511644.2% +102.0%
gateway-64 512 76,083 6005.8% 2.9GiB NEW NEW
gateway-64 1024 72,048 5403.5% 5.5GiB NEW NEW
gateway-h3 64 82,249 4120.1% 960MiB NEW NEW
gateway-h3 256 60,591 2925.3% 2.4GiB NEW NEW
unary-grpc 256 666,662 5395.2% 616MiB NEW NEW
unary-grpc 1024 735,534 5726.3% 2.3GiB NEW NEW
unary-grpc-tls 256 706,694 5306.0% 798MiB NEW NEW
unary-grpc-tls 1024 713,320 5825.6% 1.9GiB NEW NEW
stream-grpc 64 1,914,000 3901.7% 588MiB NEW NEW
stream-grpc-tls 64 1,899,000 3904.3% 577MiB NEW NEW
echo-ws 512 1,638,670 6623.0% 346MiB -3.4% -4.4%
echo-ws 4096 425,887 6517.1% 908MiB +1.8% -3.2%
echo-ws 16384 405,613 6286.8% 2.9GiB +1.4% +7.4%
echo-ws-pipeline 512 15,457,392 6681.0% 353MiB +224.9% -6.4%
echo-ws-pipeline 4096 10,985,401 6596.6% 926MiB +135.7% +2.0%
echo-ws-pipeline 16384 6,907,075 6562.1% 2.9GiB +63.4% +7.4%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants