Docs: Node comparison report#7346
Open
sudo-shashank wants to merge 17 commits into
Open
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdded a Node Comparison documentation section with Forest versus Lotus feature, RPC performance, and snapshot generation comparisons. ChangesNode comparison documentation
Estimated code review effort: 1 (Trivial) | ~5 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/docs/users/reports/node_comparison.md (1)
1-132: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRun Prettier before merging.
Docs Check / Checkcurrently fails because this file is not formatted according to Prettier. Run Prettier with--write, then rerun the documentation checks.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/docs/users/reports/node_comparison.md` around lines 1 - 132, Format the Node Comparison documentation with Prettier using write mode, then rerun the documentation checks and ensure the file passes without formatting errors.Source: Pipeline failures
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/docs/users/reports/node_comparison.md`:
- Around line 121-123: Update the snapshot compression note to distinguish Lotus
export output in this benchmark from Lotus snapshots generally. Remove the claim
that Lotus snapshots are uncompressed, and align the wording with
snapshot_service.md by stating that snapshots are zstd-compressed and readable
by both Forest and Lotus.
- Line 11: Update the version references in the node comparison documentation so
the environment versions and snapshot benchmark versions are aligned, or
explicitly label the snapshot figures as a separate historical benchmark with
Forest 0.33.6 and Lotus 1.36.0. Ensure the 7–13x conclusion clearly refers to
the correct releases.
---
Outside diff comments:
In `@docs/docs/users/reports/node_comparison.md`:
- Around line 1-132: Format the Node Comparison documentation with Prettier
using write mode, then rerun the documentation checks and ensure the file passes
without formatting errors.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 66f9a67b-da56-4636-9488-086ade311903
⛔ Files ignored due to path filters (16)
docs/static/img/reports/node_comparison/average-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/average-batch-size.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/build-info.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/cpu.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/in-flight-requests.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/memory.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/p50-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/p50-per-flow-latency.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/p95-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/p95-per-flow-latency.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/p99-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/p99-per-flow-latency.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/request-count-batch-aware.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/request-count-http.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/snapshot-export-comparison.pngis excluded by!**/*.pngdocs/static/img/reports/node_comparison/wall-time-per-http-request.pngis excluded by!**/*.png
📒 Files selected for processing (2)
docs/docs/users/reports/node_comparison.mdpackage.json
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
filecoin-project/lotus(manual)
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
docs/docs/users/reports/node_comparison.md (1)
120-129: 🗄️ Data Integrity & Integration | 🟠 MajorScope the snapshot compression claim to the measured export.
“Lotus snapshots are also not compressed” overgeneralizes the benchmark. State whether each measured export is compressed or uncompressed, and avoid presenting the disk-space difference as an intrinsic Lotus-versus-Forest property. If this benchmark measured uncompressed Lotus export output, say so explicitly.
This follows the linked Lotus findings that compressed CAR inputs are supported.
Also applies to: 136-136
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/docs/users/reports/node_comparison.md` around lines 120 - 129, Update the note in the snapshot comparison to explicitly state that the measured Lotus exports produced uncompressed output, and clarify that the disk-space difference applies only to this benchmark rather than being an intrinsic implementation property. Retain the distinction that Forest compresses snapshots during export and that both implementations can consume compressed snapshots.Source: Linked repositories
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/docs/users/reports/node_comparison.md`:
- Line 8: Expand the node comparison report beyond performance, resource usage,
and snapshots by adding a compact comparison section covering implementation
language, supported features, and other defining properties requested by issue
`#5197`. Alternatively, explicitly narrow the report’s stated scope to exclude
those comparisons.
- Line 120: Update the snapshot benchmark paragraph and its associated table to
align the cited Lotus and Forest versions with the environment versions, or
explicitly label the results as historical benchmarks. Ensure the “7–13x”
performance takeaway preserves and clearly attributes the actual benchmark
versions, including the matching references around the repeated section.
---
Duplicate comments:
In `@docs/docs/users/reports/node_comparison.md`:
- Around line 120-129: Update the note in the snapshot comparison to explicitly
state that the measured Lotus exports produced uncompressed output, and clarify
that the disk-space difference applies only to this benchmark rather than being
an intrinsic implementation property. Retain the distinction that Forest
compresses snapshots during export and that both implementations can consume
compressed snapshots.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 47426223-1377-42db-982b-f94b7b6496cd
⛔ Files ignored due to path filters (1)
docs/static/img/reports/node_comparison/p50-per-flow-latency.pngis excluded by!**/*.png
📒 Files selected for processing (1)
docs/docs/users/reports/node_comparison.md
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
filecoin-project/lotus(manual)
sudo-shashank
marked this pull request as ready for review
July 13, 2026 13:16
sudo-shashank
requested review from
LesnyRumcajs and
hanabi1224
and removed request for
a team
July 13, 2026 13:16
LesnyRumcajs
requested changes
Jul 14, 2026
LesnyRumcajs
left a comment
LesnyRumcajs
left a comment
Member
There was a problem hiding this comment.
Please re-read the task description. It's fine to shill Forest's performance but this was not the gap in documentation we want to close.
Let's:
- make it less marketing-aggressive. There are certain individuals in the ecosystem that do that already (especially with the help of their sycophant AI companions) and we shouldn't follow their example. Just showing the numbers and explaining them in boring, academic voice is fine.
- split to two reports - RPC and snapshot generation.
- this does not close the linked issue - a separate page must be made to address it - with actual feature objective comparison etc., good and bad points. All in all, a new user must be able to decide whether to use Forest or Lotus after reading it.
Contributor
Author
Got it. Thanks for the clarification and feedback here I'll make the changes. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted filessee 10 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
sudo-shashank
force-pushed
the
shashank/node-comparison-report
branch
from
July 15, 2026 01:18
18d1b13 to
68b060e
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (1)
docs/docs/users/reports/node_comparison/index.md (1)
8-12: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winMake the overview neutral and version-scoped. The opening still uses broad claims like “substantially faster,” “a fraction of the memory,” and “new protocol features typically land first.” Move the measured performance comparison into the linked reports and add the Forest/Lotus versions and test conditions here so this page reads as an objective summary.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/docs/users/reports/node_comparison/index.md` around lines 8 - 12, Revise the overview in the Forest/Lotus comparison document to remove broad performance and feature-timing claims, including “substantially faster,” “a fraction of the memory,” and “new protocol features typically land first.” Keep the summary neutral and add the specific Forest and Lotus versions plus the test conditions, linking measured performance details to the comparison reports.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/docs/users/reports/node_comparison/index.md`:
- Around line 13-26: Expand the Feature comparison section in the node
comparison page to include compact, version-scoped resource requirements plus
explicit pros and cons for both Forest and Lotus. Preserve the existing
capability table, and ensure the added sections clearly support choosing between
the implementations without requiring separate reports.
In `@docs/docs/users/reports/node_comparison/rpc_comparison.md`:
- Around line 8-12: Soften the attribution claim in the introductory comparison
text to describe only the differences observed in the captured traces, rather
than asserting they are caused by the nodes. Update the “Comparable load”
reference as needed to avoid implying that request rates, batch sizes, and
concurrency alone establish equivalent workload.
- Around line 94-105: Clarify the CPU and memory metric definitions in the
“Resource usage” section: specify whether percentages represent process, host,
or cgroup usage, and state whether they are averages or peaks plus the sampling
or observation window used. Preserve the existing Forest/Lotus comparisons and
cache note.
- Around line 18-24: Add concise methodology notes to the Per-flow latency and
amortized latency sections, documenting the aggregation window, interval
duration, sample count, and whether each chart series represents a full-window
percentile, interval percentile, or interval maximum. Keep the existing latency
findings and chart references unchanged.
In `@docs/docs/users/reports/node_comparison/snapshot_comparison.md`:
- Around line 8-24: Expand the snapshot comparison documentation around the
Forest and Lotus benchmark table to include chain height, hardware, disk type,
exact export commands and flags, and compression settings. Explicitly define
whether “Required disk space” represents peak working space or final snapshot
size, while preserving the existing benchmark results and comparisons.
- Around line 26-30: Revise the compatibility statement in the snapshot
comparison documentation to scope it explicitly to the tested Forest 0.33.6 and
Lotus 1.36.0 basic snapshot export path. Remove broad claims covering every CAR
or snapshot variant and compressed-input interoperability, and ensure the
surrounding compression note does not imply untested compatibility.
---
Nitpick comments:
In `@docs/docs/users/reports/node_comparison/index.md`:
- Around line 8-12: Revise the overview in the Forest/Lotus comparison document
to remove broad performance and feature-timing claims, including “substantially
faster,” “a fraction of the memory,” and “new protocol features typically land
first.” Keep the summary neutral and add the specific Forest and Lotus versions
plus the test conditions, linking measured performance details to the comparison
reports.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: a202a1c7-7965-4c3d-957f-7d56423a0a3e
⛔ Files ignored due to path filters (16)
docs/static/img/reports/rpc_comparison/average-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/average-batch-size.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/build-info.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/cpu.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/in-flight-requests.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/memory.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/p50-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/p50-per-flow-latency.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/p95-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/p95-per-flow-latency.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/p99-amortized-latency.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/p99-per-flow-latency.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/request-count-batch-aware.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/request-count-http.pngis excluded by!**/*.pngdocs/static/img/reports/rpc_comparison/wall-time-per-http-request.pngis excluded by!**/*.pngdocs/static/img/reports/snapshot_comparison/snapshot-export-comparison.pngis excluded by!**/*.png
📒 Files selected for processing (4)
docs/docs/users/reports/node_comparison/_category_.jsondocs/docs/users/reports/node_comparison/index.mddocs/docs/users/reports/node_comparison/rpc_comparison.mddocs/docs/users/reports/node_comparison/snapshot_comparison.md
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
filecoin-project/lotus(manual)
LesnyRumcajs
requested changes
Jul 15, 2026
sudo-shashank
enabled auto-merge
July 17, 2026 07:00
| | Storage provider / sealing (`lotus-miner`, `lotus-worker`) | No | Yes | | ||
| | Block production / mining | No (untested) | Yes | | ||
|
|
||
| Forest and Lotus both expose a Lotus-compatible Filecoin JSON-RPC API (requests and responses use the same JSON format). Forest serves three API versions: `/rpc/v0` (deprecated, legacy Lotus-compatible methods), `/rpc/v1` (stable and recommended for production), and `/rpc/v2` (experimental, still being rolled out). Forest does not aim for 100% Lotus API parity: it implements the methods needed for chain validation, RPC serving, and snapshots, but not storage-provider or sealing methods. For the full, per-version list of supported methods, see the [JSON-RPC overview](../../reference/json-rpc/overview.md) and [methods reference](../../reference/json-rpc/methods.mdx). |
Member
There was a problem hiding this comment.
Suggested change
| Forest and Lotus both expose a Lotus-compatible Filecoin JSON-RPC API (requests and responses use the same JSON format). Forest serves three API versions: `/rpc/v0` (deprecated, legacy Lotus-compatible methods), `/rpc/v1` (stable and recommended for production), and `/rpc/v2` (experimental, still being rolled out). Forest does not aim for 100% Lotus API parity: it implements the methods needed for chain validation, RPC serving, and snapshots, but not storage-provider or sealing methods. For the full, per-version list of supported methods, see the [JSON-RPC overview](../../reference/json-rpc/overview.md) and [methods reference](../../reference/json-rpc/methods.mdx). | |
| Forest and Lotus both expose a compatible Filecoin JSON-RPC API (requests and responses use the same JSON format). Forest serves three API versions: `/rpc/v0` (deprecated, legacy Lotus-compatible methods), `/rpc/v1` (stable and recommended for production), and `/rpc/v2` (experimental, still being rolled out). For the full, per-version list of supported methods, see the [JSON-RPC overview](../../reference/json-rpc/overview.md) and [methods reference](../../reference/json-rpc/methods.mdx). |
Also, maybe mention https://docs.forest.chainsafe.io/reports/api_conformance/report_2026-07-01
| ## Feature comparison | ||
|
|
||
| | Capability | Forest | Lotus | | ||
| | ---------------------------------------------------------- | ------------- | ----- | |
Member
There was a problem hiding this comment.
Let's add some features that Forest has that Lotus doesn't like eth tracing methods, archival lite/diff snapshots generation, auto-download snapshot on startup, single binary for all networks.
Member
There was a problem hiding this comment.
I'm not sure if Lotus has healthcheck endpoint - something to check out
| | Bootstrap node | Yes | Yes | | ||
| | F3 (Fast Finality) participation | Yes | Yes | | ||
| | Storage provider / sealing (`lotus-miner`, `lotus-worker`) | No | Yes | | ||
| | Block production / mining | No (untested) | Yes | |
Member
There was a problem hiding this comment.
Suggested change
| | Block production / mining | No (untested) | Yes | | |
| | Block production / mining | No (untested) | Yes | | |
| | Written in Rust 🦀 | Yes | No | |
;)
Member
There was a problem hiding this comment.
I don't think batch size is interesting for readers. The only thing that's important in that regard is the req/s and the fact that both Forest and Lotus received roughly same volume of traffic.
Member
There was a problem hiding this comment.
No need for a screenshot of this, just write down the versions...
|
|
||
| Both implementations produce compatible snapshots and can consume snapshots produced by the other. | ||
|
|
||
| :::note |
Member
There was a problem hiding this comment.
also worth noting that Forest snapshots contain an index that currently only Forest supports that allow it to use the snapshot directly as a blockstore.
Comment on lines
+18
to
+24
| Forest `0.33.6` finished the export in 34 minutes using 32 GiB of RAM. Lotus `1.36.0` took 450 minutes (about `13x` slower) with 128 GiB, and 232 minutes (about `7x` slower) even with 256 GiB. Forest also needs less than half the disk space for the resulting snapshot. The table below compares a basic snapshot export across implementations. | ||
|
|
||
| | | Required disk space [GiB] | RAM [GiB] | Export duration [minutes] | | ||
| | ------ | ------------------------- | --------- | ------------------------- | | ||
| | Forest | 200 | 32 | 34 | | ||
| | Lotus | 450 | 128 | 450 | | ||
| | Lotus | 450 | 256 | 232 | |
Member
There was a problem hiding this comment.
Suggested change
| Forest `0.33.6` finished the export in 34 minutes using 32 GiB of RAM. Lotus `1.36.0` took 450 minutes (about `13x` slower) with 128 GiB, and 232 minutes (about `7x` slower) even with 256 GiB. Forest also needs less than half the disk space for the resulting snapshot. The table below compares a basic snapshot export across implementations. | |
| | | Required disk space [GiB] | RAM [GiB] | Export duration [minutes] | | |
| | ------ | ------------------------- | --------- | ------------------------- | | |
| | Forest | 200 | 32 | 34 | | |
| | Lotus | 450 | 128 | 450 | | |
| | Lotus | 450 | 256 | 232 | |
it's already in thescreenshot, why the repetition?
LesnyRumcajs
left a comment
LesnyRumcajs
left a comment
Member
There was a problem hiding this comment.
It makes sense to run any descriptions through some Grammarly to tweak up the language.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
Changes introduced in this pull request:
Node Comparisionsection underReports.Reference issue to close (if applicable)
Closes #5197
Other information and links
Change checklist
Outside contributions
Summary by CodeRabbit