diff --git a/README.md b/README.md index eec6c59a..0d0964fd 100644 --- a/README.md +++ b/README.md @@ -6,10 +6,12 @@ [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE) [![Go Reference](https://pkg.go.dev/badge/github.com/labstack/fanout.svg)](https://pkg.go.dev/github.com/labstack/fanout) -Fanout ingests OpenTelemetry data, stores it as Parquet, queries it with DuckDB, -and puts an AI agent in front of it — in one Go process with no external -dependencies to operate. Point a collector at it, open the browser, and ask -questions about your telemetry in plain language. +Fanout ingests OpenTelemetry data, durably publishes it as atomic Parquet +batches, and puts an AI agent in front of it — in one Go process with no +external dependencies to operate. Persistent trace indexes serve targeted +reads; embedded DuckDB handles SQL, broad scans, and rebuildable rollups. Point +an SDK or Collector at it, open the browser, and ask questions about your +telemetry in plain language. There is no separate ingester, query service, metadata database, object store, or dashboard server to deploy. One binary, one data directory. @@ -31,26 +33,34 @@ browser client, an in-process agent, and any external MCP host all reach the same typed observability contract rather than issuing raw SQL. -Parquet is the telemetry source of truth, DuckDB query state is disposable, -and SQLite is reserved for transactional product state. Native compaction +Parquet is authoritative telemetry, DuckDB query state is rebuildable, and +SQLite is reserved for transactional product state. Native compaction prepares replacements while reads continue and briefly gates readers only for the crash-safe namespace swap: ![Fanout persistence](docs/diagrams/persistence.svg) Application state (users, sessions, dashboards, alert rules, agent threads) -lives in that separate SQLite database and never sits on the telemetry write -path. There is no Iceberg, DuckLake, external catalog, or telemetry server -database. +lives in the control SQLite database and never sits on the telemetry write +path. The published Parquet directories are self-describing, so startup can +discover the authoritative batch set directly from the filesystem. ## Performance The independent [Fanout Bench](https://github.com/labstack/fanout-bench) project measures authenticated ingest and optional dashboard read load against your hardware. It uses the official OpenTelemetry generator and publishes raw, -reproducible evidence separately from the production binary. Ingest, DuckDB -queries, and native Parquet maintenance have separate coordination paths but -still compete for the same CPU, memory bandwidth, filesystem cache, and disk. +reproducible evidence separately from the production binary. Ingest, indexed +reads, DuckDB analytics, and native Parquet maintenance have separate +coordination paths but still compete for the same CPU, memory bandwidth, +filesystem cache, and disk. + +The current publication candidate is **296,196 accepted OpenTelemetry items per +second** sustained for five minutes with traces, logs, and metrics arriving +together on a machine with eight logical CPUs and 15.6 GiB of memory. It is a +single run, and the benchmark harness that produced it carried uncommitted +local changes, so it is not Fanout's official headline yet. The [performance methodology](https://fanout.run/explanation/performance) +shows the signal breakdown, quality gates, limitations, and publication bar. ## How it compares @@ -61,7 +71,7 @@ what separates it from its neighbours. | If you use | Where Fanout differs | | --- | --- | | **Grafana with Loki, Tempo, and Mimir** | That stack keeps a service and a query language per signal, plus object storage underneath. Fanout keeps one process, one data directory, and one typed contract across all three signals, at the cost of the horizontal scale those components are built for. | -| **SigNoz** | Both are OTLP-native and self-hosted. SigNoz composes a collector, ClickHouse, and query services; Fanout compiles ingest, indexed storage, DuckDB queries, alerting, and the browser client into one binary, with open Parquet on local disk instead of a database cluster. | +| **SigNoz** | Both are OTLP-native and self-hosted. SigNoz composes a collector, ClickHouse, and query services; Fanout compiles ingest, authoritative Parquet storage, indexed trace reads, DuckDB analytics, alerting, and the browser client into one binary. | | **Jaeger** | Jaeger covers traces and expects a storage backend you run separately. Fanout ingests traces, logs, and metrics into the same store, with nothing else to deploy. | | **Prometheus with Grafana** | Prometheus pulls metrics and is excellent at them. Fanout accepts pushed OTLP for all three signals and is built around investigating a specific incident rather than maintaining long-range metric series. | | **Datadog**, **Honeycomb**, **Grafana Cloud** | Those are managed services: someone else runs the storage, the scaling, and the upgrades, and your telemetry leaves your network to get there. Fanout is a binary you run, on data that stays on your disk. | diff --git a/docs/diagrams/architecture.d2 b/docs/diagrams/architecture.d2 index d2b34aaa..8a8ca36f 100644 --- a/docs/diagrams/architecture.d2 +++ b/docs/diagrams/architecture.d2 @@ -45,7 +45,7 @@ fanout: "fanout — one Go process" { store: Storage { style.fill: transparent telemetry: "Parquet batches + trace indexes\nstorage.data_dir/telemetry" {shape: cylinder} - qstate: "Query catalog\nstorage.data_dir/query" {shape: cylinder} + qstate: "DuckDB query state\nstorage.data_dir/query" {shape: cylinder} control: "Control SQLite\nstorage.data_dir/control" {shape: cylinder} } diff --git a/docs/diagrams/architecture.svg b/docs/diagrams/architecture.svg index 6f054670..47ec1819 100644 --- a/docs/diagrams/architecture.svg +++ b/docs/diagrams/architecture.svg @@ -1,23 +1,23 @@ -Clientsfanout — one Go processStorageAnthropic or OpenAIOTLP collector or SDKBrowserExternal MCP hostHTTP :7520routes + auth middlewareIngestOTLP gRPC :4317 + HTTP :4318Embedded React assetsAgent runtimemodel + tool loopMCP server5 typed + 4 dashboard toolsTyped observability contractTelemetry commit workersatomic Parquet batchesQuery kernelDuckDB + rollupsStorage maintenanceretention + native compactionAlert enginerule evaluation + webhooksParquet batches + trace indexesstorage.data_dir/telemetryQuery catalogstorage.data_dir/queryControl SQLitestorage.data_dir/control serves embedded assetsAG-UI streamin-memory transporttyped HTTP APIevaluates rollupsOTLP/gRPC or OTLP/HTTPHTTPS/mcp — OAuthindexed traces + SQL scansatomic replacementusers, sessions, settings, dashboards, threadsrules and fired alertsHTTPS + .d2-1952141514 .fill-N1{fill:#0A0F25;} + .d2-1952141514 .fill-N2{fill:#676C7E;} + .d2-1952141514 .fill-N3{fill:#9499AB;} + .d2-1952141514 .fill-N4{fill:#CFD2DD;} + .d2-1952141514 .fill-N5{fill:#DEE1EB;} + .d2-1952141514 .fill-N6{fill:#EEF1F8;} + .d2-1952141514 .fill-N7{fill:#FFFFFF;} + .d2-1952141514 .fill-B1{fill:#000536;} + .d2-1952141514 .fill-B2{fill:#0F66B7;} + .d2-1952141514 .fill-B3{fill:#4393DD;} + .d2-1952141514 .fill-B4{fill:#87BFF3;} + .d2-1952141514 .fill-B5{fill:#BCDDFB;} + .d2-1952141514 .fill-B6{fill:#E5F3FF;} + .d2-1952141514 .fill-AA2{fill:#7639C5;} + .d2-1952141514 .fill-AA4{fill:#C1A2F3;} + .d2-1952141514 .fill-AA5{fill:#DACEFB;} + .d2-1952141514 .fill-AB4{fill:#EA99C6;} + .d2-1952141514 .fill-AB5{fill:#FFDEF1;} + .d2-1952141514 .stroke-N1{stroke:#0A0F25;} + .d2-1952141514 .stroke-N2{stroke:#676C7E;} + .d2-1952141514 .stroke-N3{stroke:#9499AB;} + .d2-1952141514 .stroke-N4{stroke:#CFD2DD;} + .d2-1952141514 .stroke-N5{stroke:#DEE1EB;} + .d2-1952141514 .stroke-N6{stroke:#EEF1F8;} + .d2-1952141514 .stroke-N7{stroke:#FFFFFF;} + .d2-1952141514 .stroke-B1{stroke:#000536;} + .d2-1952141514 .stroke-B2{stroke:#0F66B7;} + .d2-1952141514 .stroke-B3{stroke:#4393DD;} + .d2-1952141514 .stroke-B4{stroke:#87BFF3;} + .d2-1952141514 .stroke-B5{stroke:#BCDDFB;} + .d2-1952141514 .stroke-B6{stroke:#E5F3FF;} + .d2-1952141514 .stroke-AA2{stroke:#7639C5;} + .d2-1952141514 .stroke-AA4{stroke:#C1A2F3;} + .d2-1952141514 .stroke-AA5{stroke:#DACEFB;} + .d2-1952141514 .stroke-AB4{stroke:#EA99C6;} + .d2-1952141514 .stroke-AB5{stroke:#FFDEF1;} + .d2-1952141514 .background-color-N1{background-color:#0A0F25;} + .d2-1952141514 .background-color-N2{background-color:#676C7E;} + .d2-1952141514 .background-color-N3{background-color:#9499AB;} + .d2-1952141514 .background-color-N4{background-color:#CFD2DD;} + .d2-1952141514 .background-color-N5{background-color:#DEE1EB;} + .d2-1952141514 .background-color-N6{background-color:#EEF1F8;} + .d2-1952141514 .background-color-N7{background-color:#FFFFFF;} + .d2-1952141514 .background-color-B1{background-color:#000536;} + .d2-1952141514 .background-color-B2{background-color:#0F66B7;} + .d2-1952141514 .background-color-B3{background-color:#4393DD;} + .d2-1952141514 .background-color-B4{background-color:#87BFF3;} + .d2-1952141514 .background-color-B5{background-color:#BCDDFB;} + .d2-1952141514 .background-color-B6{background-color:#E5F3FF;} + .d2-1952141514 .background-color-AA2{background-color:#7639C5;} + .d2-1952141514 .background-color-AA4{background-color:#C1A2F3;} + .d2-1952141514 .background-color-AA5{background-color:#DACEFB;} + .d2-1952141514 .background-color-AB4{background-color:#EA99C6;} + .d2-1952141514 .background-color-AB5{background-color:#FFDEF1;} + .d2-1952141514 .color-N1{color:#0A0F25;} + .d2-1952141514 .color-N2{color:#676C7E;} + .d2-1952141514 .color-N3{color:#9499AB;} + .d2-1952141514 .color-N4{color:#CFD2DD;} + .d2-1952141514 .color-N5{color:#DEE1EB;} + .d2-1952141514 .color-N6{color:#EEF1F8;} + .d2-1952141514 .color-N7{color:#FFFFFF;} + .d2-1952141514 .color-B1{color:#000536;} + .d2-1952141514 .color-B2{color:#0F66B7;} + .d2-1952141514 .color-B3{color:#4393DD;} + .d2-1952141514 .color-B4{color:#87BFF3;} + .d2-1952141514 .color-B5{color:#BCDDFB;} + .d2-1952141514 .color-B6{color:#E5F3FF;} + .d2-1952141514 .color-AA2{color:#7639C5;} + .d2-1952141514 .color-AA4{color:#C1A2F3;} + .d2-1952141514 .color-AA5{color:#DACEFB;} + .d2-1952141514 .color-AB4{color:#EA99C6;} + .d2-1952141514 .color-AB5{color:#FFDEF1;}.appendix text.text{fill:#0A0F25}.md{--color-fg-default:#0A0F25;--color-fg-muted:#676C7E;--color-fg-subtle:#9499AB;--color-canvas-default:#FFFFFF;--color-canvas-subtle:#EEF1F8;--color-border-default:#000536;--color-border-muted:#0F66B7;--color-neutral-muted:#EEF1F8;--color-accent-fg:#0F66B7;--color-accent-emphasis:#0F66B7;--color-attention-subtle:#676C7E;--color-danger-fg:red;}.sketch-overlay-B1{fill:url(#streaks-darker-d2-1952141514);mix-blend-mode:lighten}.sketch-overlay-B2{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-B3{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-B4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-B5{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-B6{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-AA2{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-AA4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-AA5{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-AB4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-AB5{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-N1{fill:url(#streaks-darker-d2-1952141514);mix-blend-mode:lighten}.sketch-overlay-N2{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-N3{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-N4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-N5{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-N6{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-N7{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.light-code{display: block}.dark-code{display: none}]]>Clientsfanout — one Go processStorageAnthropic or OpenAIOTLP collector or SDKBrowserExternal MCP hostHTTP :7520routes + auth middlewareIngestOTLP gRPC :4317 + HTTP :4318Embedded React assetsAgent runtimemodel + tool loopMCP server5 typed + 4 dashboard toolsTyped observability contractTelemetry commit workersatomic Parquet batchesQuery kernelDuckDB + rollupsStorage maintenanceretention + native compactionAlert enginerule evaluation + webhooksParquet batches + trace indexesstorage.data_dir/telemetryDuckDB query statestorage.data_dir/queryControl SQLitestorage.data_dir/control serves embedded assetsAG-UI streamin-memory transporttyped HTTP APIevaluates rollupsOTLP/gRPC or OTLP/HTTPHTTPS/mcp — OAuthindexed traces + SQL scansatomic replacementusers, sessions, settings, dashboards, threadsrules and fired alertsHTTPS diff --git a/docs/diagrams/persistence.d2 b/docs/diagrams/persistence.d2 index 33472831..d5b508b5 100644 --- a/docs/diagrams/persistence.d2 +++ b/docs/diagrams/persistence.d2 @@ -29,7 +29,7 @@ writers: "Durable write path" { authority: "Authoritative telemetry\nimmutable *.batch directories" { shape: cylinder - tooltip: Ordinary Parquet files are the source of truth; the filesystem is the catalog. + tooltip: Ordinary Parquet files are authoritative; published batch directories are discovered directly from the filesystem. } readers: Readers { diff --git a/docs/diagrams/persistence.svg b/docs/diagrams/persistence.svg index e1c636dc..50c353fc 100644 --- a/docs/diagrams/persistence.svg +++ b/docs/diagrams/persistence.svg @@ -1,26 +1,26 @@ -Authenticated OTLP requestsspans, logs, metricsDurable write pathAuthoritative telemetryimmutable *.batch directoriesOrdinary Parquet files are the source of truthReadersBounded maintenanceRebuildable query statestorage.data_dir/queryDuckDB rollups and temporary spill. Not authoritative telemetry.Bounded group admissionconfigured row target + 20 ms windowUp to four commit workersindependent batches encode in parallelStaging batch directorymetadata + signal Parquet + trace.fidx for spansDurable publicationfsync → atomic rename → fsync parentthe filesystem is the catalogIndexed trace readbinary-search .fidx → Parquet rowsDuckDB SQLfilters, ordering, broad scansDuckDB rollupsrebuildable analytical cacheRetention + compaction admissionbounded batch and row groupsNative Parquet mergeprepared outside the reader gateCrash-safe namespace swapmarker + bounded renames + fsync acknowledge only after publicationOrdinary Parquet files are the source of truth - + .d2-1108726992 .fill-N1{fill:#0A0F25;} + .d2-1108726992 .fill-N2{fill:#676C7E;} + .d2-1108726992 .fill-N3{fill:#9499AB;} + .d2-1108726992 .fill-N4{fill:#CFD2DD;} + .d2-1108726992 .fill-N5{fill:#DEE1EB;} + .d2-1108726992 .fill-N6{fill:#EEF1F8;} + .d2-1108726992 .fill-N7{fill:#FFFFFF;} + .d2-1108726992 .fill-B1{fill:#000536;} + .d2-1108726992 .fill-B2{fill:#0F66B7;} + .d2-1108726992 .fill-B3{fill:#4393DD;} + .d2-1108726992 .fill-B4{fill:#87BFF3;} + .d2-1108726992 .fill-B5{fill:#BCDDFB;} + .d2-1108726992 .fill-B6{fill:#E5F3FF;} + .d2-1108726992 .fill-AA2{fill:#7639C5;} + .d2-1108726992 .fill-AA4{fill:#C1A2F3;} + .d2-1108726992 .fill-AA5{fill:#DACEFB;} + .d2-1108726992 .fill-AB4{fill:#EA99C6;} + .d2-1108726992 .fill-AB5{fill:#FFDEF1;} + .d2-1108726992 .stroke-N1{stroke:#0A0F25;} + .d2-1108726992 .stroke-N2{stroke:#676C7E;} + .d2-1108726992 .stroke-N3{stroke:#9499AB;} + .d2-1108726992 .stroke-N4{stroke:#CFD2DD;} + .d2-1108726992 .stroke-N5{stroke:#DEE1EB;} + .d2-1108726992 .stroke-N6{stroke:#EEF1F8;} + .d2-1108726992 .stroke-N7{stroke:#FFFFFF;} + .d2-1108726992 .stroke-B1{stroke:#000536;} + .d2-1108726992 .stroke-B2{stroke:#0F66B7;} + .d2-1108726992 .stroke-B3{stroke:#4393DD;} + .d2-1108726992 .stroke-B4{stroke:#87BFF3;} + .d2-1108726992 .stroke-B5{stroke:#BCDDFB;} + .d2-1108726992 .stroke-B6{stroke:#E5F3FF;} + .d2-1108726992 .stroke-AA2{stroke:#7639C5;} + .d2-1108726992 .stroke-AA4{stroke:#C1A2F3;} + .d2-1108726992 .stroke-AA5{stroke:#DACEFB;} + .d2-1108726992 .stroke-AB4{stroke:#EA99C6;} + .d2-1108726992 .stroke-AB5{stroke:#FFDEF1;} + .d2-1108726992 .background-color-N1{background-color:#0A0F25;} + .d2-1108726992 .background-color-N2{background-color:#676C7E;} + .d2-1108726992 .background-color-N3{background-color:#9499AB;} + .d2-1108726992 .background-color-N4{background-color:#CFD2DD;} + .d2-1108726992 .background-color-N5{background-color:#DEE1EB;} + .d2-1108726992 .background-color-N6{background-color:#EEF1F8;} + .d2-1108726992 .background-color-N7{background-color:#FFFFFF;} + .d2-1108726992 .background-color-B1{background-color:#000536;} + .d2-1108726992 .background-color-B2{background-color:#0F66B7;} + .d2-1108726992 .background-color-B3{background-color:#4393DD;} + .d2-1108726992 .background-color-B4{background-color:#87BFF3;} + .d2-1108726992 .background-color-B5{background-color:#BCDDFB;} + .d2-1108726992 .background-color-B6{background-color:#E5F3FF;} + .d2-1108726992 .background-color-AA2{background-color:#7639C5;} + .d2-1108726992 .background-color-AA4{background-color:#C1A2F3;} + .d2-1108726992 .background-color-AA5{background-color:#DACEFB;} + .d2-1108726992 .background-color-AB4{background-color:#EA99C6;} + .d2-1108726992 .background-color-AB5{background-color:#FFDEF1;} + .d2-1108726992 .color-N1{color:#0A0F25;} + .d2-1108726992 .color-N2{color:#676C7E;} + .d2-1108726992 .color-N3{color:#9499AB;} + .d2-1108726992 .color-N4{color:#CFD2DD;} + .d2-1108726992 .color-N5{color:#DEE1EB;} + .d2-1108726992 .color-N6{color:#EEF1F8;} + .d2-1108726992 .color-N7{color:#FFFFFF;} + .d2-1108726992 .color-B1{color:#000536;} + .d2-1108726992 .color-B2{color:#0F66B7;} + .d2-1108726992 .color-B3{color:#4393DD;} + .d2-1108726992 .color-B4{color:#87BFF3;} + .d2-1108726992 .color-B5{color:#BCDDFB;} + .d2-1108726992 .color-B6{color:#E5F3FF;} + .d2-1108726992 .color-AA2{color:#7639C5;} + .d2-1108726992 .color-AA4{color:#C1A2F3;} + .d2-1108726992 .color-AA5{color:#DACEFB;} + .d2-1108726992 .color-AB4{color:#EA99C6;} + .d2-1108726992 .color-AB5{color:#FFDEF1;}.appendix text.text{fill:#0A0F25}.md{--color-fg-default:#0A0F25;--color-fg-muted:#676C7E;--color-fg-subtle:#9499AB;--color-canvas-default:#FFFFFF;--color-canvas-subtle:#EEF1F8;--color-border-default:#000536;--color-border-muted:#0F66B7;--color-neutral-muted:#EEF1F8;--color-accent-fg:#0F66B7;--color-accent-emphasis:#0F66B7;--color-attention-subtle:#676C7E;--color-danger-fg:red;}.sketch-overlay-B1{fill:url(#streaks-darker-d2-1108726992);mix-blend-mode:lighten}.sketch-overlay-B2{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-B3{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-B4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-B5{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-B6{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-AA2{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-AA4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-AA5{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-AB4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-AB5{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-N1{fill:url(#streaks-darker-d2-1108726992);mix-blend-mode:lighten}.sketch-overlay-N2{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-N3{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-N4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-N5{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-N6{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-N7{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.light-code{display: block}.dark-code{display: none}]]>Authenticated OTLP requestsspans, logs, metricsDurable write pathAuthoritative telemetryimmutable *.batch directoriesOrdinary Parquet files are authoritativeReadersBounded maintenanceRebuildable query statestorage.data_dir/queryDuckDB rollups and temporary spill. Not authoritative telemetry.Bounded group admissionconfigured row target + 20 ms windowUp to four commit workersindependent batches encode in parallelStaging batch directorymetadata + signal Parquet + trace.fidx for spansDurable publicationfsync → atomic rename → fsync parentpublished batch directories are discovered directly from the filesystemIndexed trace readbinary-search .fidx → Parquet rowsDuckDB SQLfilters, ordering, broad scansDuckDB rollupsrebuildable analytical cacheRetention + compaction admissionbounded batch and row groupsNative Parquet mergeprepared outside the reader gateCrash-safe namespace swapmarker + bounded renames + fsync acknowledge only after publicationOrdinary Parquet files are authoritative + - + DuckDB rollups and temporary spill. Not authoritative telemetry. - + - + - + diff --git a/internal/config/config.go b/internal/config/config.go index c854a606..721937c2 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -39,8 +39,8 @@ type Config struct { // RollupSkipToLatest, set once at boot, advances every rollup watermark to the // current max ingested timestamp so existing data is treated as already-rolled-up // instead of aggregated as a backlog. Stands up a large pre-seeded historical - // dataset (benchmarks, restores) without a multi-minute first-rollup catch-up that - // holds the write gate and starves ingest. Off in normal operation. + // dataset (benchmarks, restores) without a multi-minute first-rollup catch-up + // that monopolizes DuckDB write capacity. Off in normal operation. RollupSkipToLatest bool `koanf:"storage.rollup_skip_to_latest" env:"FANOUT_ROLLUP_SKIP_TO_LATEST" default:"false"` DefaultNamespace string `koanf:"ingest.default_namespace" env:"FANOUT_DEFAULT_NAMESPACE" default:"default"` // PprofEnabled exposes Go's net/http/pprof handlers at /debug/pprof/* for diff --git a/internal/query/duck.go b/internal/query/duck.go index 02c6bdff..0198bf80 100644 --- a/internal/query/duck.go +++ b/internal/query/duck.go @@ -263,9 +263,9 @@ func NewDuck(ctx context.Context, cfg config.Config, repository *telemetrystore. // skipRollupToLatest advances every rollup watermark to the current max ingested // timestamp, so existing data is treated as already-processed rather than // aggregated as a backlog. This avoids a multi-minute first-rollup catch-up -// (a wide-start_time backlog holds the write gate and starves ingest) when standing up -// a large pre-seeded historical dataset. Runs once at boot before RunRollups, so -// taking the write gate here is uncontended. +// that monopolizes DuckDB write capacity when standing up a large pre-seeded +// historical dataset. Runs once at boot before RunRollups, so taking the write +// gate here is uncontended. func (d *Duck) skipRollupToLatest(ctx context.Context) error { unlock := d.writeGate.Lock(writegate.WriteRollupSkip) defer unlock() diff --git a/site/public/diagrams/architecture.svg b/site/public/diagrams/architecture.svg index 6f054670..47ec1819 100644 --- a/site/public/diagrams/architecture.svg +++ b/site/public/diagrams/architecture.svg @@ -1,23 +1,23 @@ -Clientsfanout — one Go processStorageAnthropic or OpenAIOTLP collector or SDKBrowserExternal MCP hostHTTP :7520routes + auth middlewareIngestOTLP gRPC :4317 + HTTP :4318Embedded React assetsAgent runtimemodel + tool loopMCP server5 typed + 4 dashboard toolsTyped observability contractTelemetry commit workersatomic Parquet batchesQuery kernelDuckDB + rollupsStorage maintenanceretention + native compactionAlert enginerule evaluation + webhooksParquet batches + trace indexesstorage.data_dir/telemetryQuery catalogstorage.data_dir/queryControl SQLitestorage.data_dir/control serves embedded assetsAG-UI streamin-memory transporttyped HTTP APIevaluates rollupsOTLP/gRPC or OTLP/HTTPHTTPS/mcp — OAuthindexed traces + SQL scansatomic replacementusers, sessions, settings, dashboards, threadsrules and fired alertsHTTPS + .d2-1952141514 .fill-N1{fill:#0A0F25;} + .d2-1952141514 .fill-N2{fill:#676C7E;} + .d2-1952141514 .fill-N3{fill:#9499AB;} + .d2-1952141514 .fill-N4{fill:#CFD2DD;} + .d2-1952141514 .fill-N5{fill:#DEE1EB;} + .d2-1952141514 .fill-N6{fill:#EEF1F8;} + .d2-1952141514 .fill-N7{fill:#FFFFFF;} + .d2-1952141514 .fill-B1{fill:#000536;} + .d2-1952141514 .fill-B2{fill:#0F66B7;} + .d2-1952141514 .fill-B3{fill:#4393DD;} + .d2-1952141514 .fill-B4{fill:#87BFF3;} + .d2-1952141514 .fill-B5{fill:#BCDDFB;} + .d2-1952141514 .fill-B6{fill:#E5F3FF;} + .d2-1952141514 .fill-AA2{fill:#7639C5;} + .d2-1952141514 .fill-AA4{fill:#C1A2F3;} + .d2-1952141514 .fill-AA5{fill:#DACEFB;} + .d2-1952141514 .fill-AB4{fill:#EA99C6;} + .d2-1952141514 .fill-AB5{fill:#FFDEF1;} + .d2-1952141514 .stroke-N1{stroke:#0A0F25;} + .d2-1952141514 .stroke-N2{stroke:#676C7E;} + .d2-1952141514 .stroke-N3{stroke:#9499AB;} + .d2-1952141514 .stroke-N4{stroke:#CFD2DD;} + .d2-1952141514 .stroke-N5{stroke:#DEE1EB;} + .d2-1952141514 .stroke-N6{stroke:#EEF1F8;} + .d2-1952141514 .stroke-N7{stroke:#FFFFFF;} + .d2-1952141514 .stroke-B1{stroke:#000536;} + .d2-1952141514 .stroke-B2{stroke:#0F66B7;} + .d2-1952141514 .stroke-B3{stroke:#4393DD;} + .d2-1952141514 .stroke-B4{stroke:#87BFF3;} + .d2-1952141514 .stroke-B5{stroke:#BCDDFB;} + .d2-1952141514 .stroke-B6{stroke:#E5F3FF;} + .d2-1952141514 .stroke-AA2{stroke:#7639C5;} + .d2-1952141514 .stroke-AA4{stroke:#C1A2F3;} + .d2-1952141514 .stroke-AA5{stroke:#DACEFB;} + .d2-1952141514 .stroke-AB4{stroke:#EA99C6;} + .d2-1952141514 .stroke-AB5{stroke:#FFDEF1;} + .d2-1952141514 .background-color-N1{background-color:#0A0F25;} + .d2-1952141514 .background-color-N2{background-color:#676C7E;} + .d2-1952141514 .background-color-N3{background-color:#9499AB;} + .d2-1952141514 .background-color-N4{background-color:#CFD2DD;} + .d2-1952141514 .background-color-N5{background-color:#DEE1EB;} + .d2-1952141514 .background-color-N6{background-color:#EEF1F8;} + .d2-1952141514 .background-color-N7{background-color:#FFFFFF;} + .d2-1952141514 .background-color-B1{background-color:#000536;} + .d2-1952141514 .background-color-B2{background-color:#0F66B7;} + .d2-1952141514 .background-color-B3{background-color:#4393DD;} + .d2-1952141514 .background-color-B4{background-color:#87BFF3;} + .d2-1952141514 .background-color-B5{background-color:#BCDDFB;} + .d2-1952141514 .background-color-B6{background-color:#E5F3FF;} + .d2-1952141514 .background-color-AA2{background-color:#7639C5;} + .d2-1952141514 .background-color-AA4{background-color:#C1A2F3;} + .d2-1952141514 .background-color-AA5{background-color:#DACEFB;} + .d2-1952141514 .background-color-AB4{background-color:#EA99C6;} + .d2-1952141514 .background-color-AB5{background-color:#FFDEF1;} + .d2-1952141514 .color-N1{color:#0A0F25;} + .d2-1952141514 .color-N2{color:#676C7E;} + .d2-1952141514 .color-N3{color:#9499AB;} + .d2-1952141514 .color-N4{color:#CFD2DD;} + .d2-1952141514 .color-N5{color:#DEE1EB;} + .d2-1952141514 .color-N6{color:#EEF1F8;} + .d2-1952141514 .color-N7{color:#FFFFFF;} + .d2-1952141514 .color-B1{color:#000536;} + .d2-1952141514 .color-B2{color:#0F66B7;} + .d2-1952141514 .color-B3{color:#4393DD;} + .d2-1952141514 .color-B4{color:#87BFF3;} + .d2-1952141514 .color-B5{color:#BCDDFB;} + .d2-1952141514 .color-B6{color:#E5F3FF;} + .d2-1952141514 .color-AA2{color:#7639C5;} + .d2-1952141514 .color-AA4{color:#C1A2F3;} + .d2-1952141514 .color-AA5{color:#DACEFB;} + .d2-1952141514 .color-AB4{color:#EA99C6;} + .d2-1952141514 .color-AB5{color:#FFDEF1;}.appendix text.text{fill:#0A0F25}.md{--color-fg-default:#0A0F25;--color-fg-muted:#676C7E;--color-fg-subtle:#9499AB;--color-canvas-default:#FFFFFF;--color-canvas-subtle:#EEF1F8;--color-border-default:#000536;--color-border-muted:#0F66B7;--color-neutral-muted:#EEF1F8;--color-accent-fg:#0F66B7;--color-accent-emphasis:#0F66B7;--color-attention-subtle:#676C7E;--color-danger-fg:red;}.sketch-overlay-B1{fill:url(#streaks-darker-d2-1952141514);mix-blend-mode:lighten}.sketch-overlay-B2{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-B3{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-B4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-B5{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-B6{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-AA2{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-AA4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-AA5{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-AB4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-AB5{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-N1{fill:url(#streaks-darker-d2-1952141514);mix-blend-mode:lighten}.sketch-overlay-N2{fill:url(#streaks-dark-d2-1952141514);mix-blend-mode:overlay}.sketch-overlay-N3{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-N4{fill:url(#streaks-normal-d2-1952141514);mix-blend-mode:color-burn}.sketch-overlay-N5{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-N6{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.sketch-overlay-N7{fill:url(#streaks-bright-d2-1952141514);mix-blend-mode:darken}.light-code{display: block}.dark-code{display: none}]]>Clientsfanout — one Go processStorageAnthropic or OpenAIOTLP collector or SDKBrowserExternal MCP hostHTTP :7520routes + auth middlewareIngestOTLP gRPC :4317 + HTTP :4318Embedded React assetsAgent runtimemodel + tool loopMCP server5 typed + 4 dashboard toolsTyped observability contractTelemetry commit workersatomic Parquet batchesQuery kernelDuckDB + rollupsStorage maintenanceretention + native compactionAlert enginerule evaluation + webhooksParquet batches + trace indexesstorage.data_dir/telemetryDuckDB query statestorage.data_dir/queryControl SQLitestorage.data_dir/control serves embedded assetsAG-UI streamin-memory transporttyped HTTP APIevaluates rollupsOTLP/gRPC or OTLP/HTTPHTTPS/mcp — OAuthindexed traces + SQL scansatomic replacementusers, sessions, settings, dashboards, threadsrules and fired alertsHTTPS diff --git a/site/public/diagrams/persistence.svg b/site/public/diagrams/persistence.svg index e1c636dc..50c353fc 100644 --- a/site/public/diagrams/persistence.svg +++ b/site/public/diagrams/persistence.svg @@ -1,26 +1,26 @@ -Authenticated OTLP requestsspans, logs, metricsDurable write pathAuthoritative telemetryimmutable *.batch directoriesOrdinary Parquet files are the source of truthReadersBounded maintenanceRebuildable query statestorage.data_dir/queryDuckDB rollups and temporary spill. Not authoritative telemetry.Bounded group admissionconfigured row target + 20 ms windowUp to four commit workersindependent batches encode in parallelStaging batch directorymetadata + signal Parquet + trace.fidx for spansDurable publicationfsync → atomic rename → fsync parentthe filesystem is the catalogIndexed trace readbinary-search .fidx → Parquet rowsDuckDB SQLfilters, ordering, broad scansDuckDB rollupsrebuildable analytical cacheRetention + compaction admissionbounded batch and row groupsNative Parquet mergeprepared outside the reader gateCrash-safe namespace swapmarker + bounded renames + fsync acknowledge only after publicationOrdinary Parquet files are the source of truth - + .d2-1108726992 .fill-N1{fill:#0A0F25;} + .d2-1108726992 .fill-N2{fill:#676C7E;} + .d2-1108726992 .fill-N3{fill:#9499AB;} + .d2-1108726992 .fill-N4{fill:#CFD2DD;} + .d2-1108726992 .fill-N5{fill:#DEE1EB;} + .d2-1108726992 .fill-N6{fill:#EEF1F8;} + .d2-1108726992 .fill-N7{fill:#FFFFFF;} + .d2-1108726992 .fill-B1{fill:#000536;} + .d2-1108726992 .fill-B2{fill:#0F66B7;} + .d2-1108726992 .fill-B3{fill:#4393DD;} + .d2-1108726992 .fill-B4{fill:#87BFF3;} + .d2-1108726992 .fill-B5{fill:#BCDDFB;} + .d2-1108726992 .fill-B6{fill:#E5F3FF;} + .d2-1108726992 .fill-AA2{fill:#7639C5;} + .d2-1108726992 .fill-AA4{fill:#C1A2F3;} + .d2-1108726992 .fill-AA5{fill:#DACEFB;} + .d2-1108726992 .fill-AB4{fill:#EA99C6;} + .d2-1108726992 .fill-AB5{fill:#FFDEF1;} + .d2-1108726992 .stroke-N1{stroke:#0A0F25;} + .d2-1108726992 .stroke-N2{stroke:#676C7E;} + .d2-1108726992 .stroke-N3{stroke:#9499AB;} + .d2-1108726992 .stroke-N4{stroke:#CFD2DD;} + .d2-1108726992 .stroke-N5{stroke:#DEE1EB;} + .d2-1108726992 .stroke-N6{stroke:#EEF1F8;} + .d2-1108726992 .stroke-N7{stroke:#FFFFFF;} + .d2-1108726992 .stroke-B1{stroke:#000536;} + .d2-1108726992 .stroke-B2{stroke:#0F66B7;} + .d2-1108726992 .stroke-B3{stroke:#4393DD;} + .d2-1108726992 .stroke-B4{stroke:#87BFF3;} + .d2-1108726992 .stroke-B5{stroke:#BCDDFB;} + .d2-1108726992 .stroke-B6{stroke:#E5F3FF;} + .d2-1108726992 .stroke-AA2{stroke:#7639C5;} + .d2-1108726992 .stroke-AA4{stroke:#C1A2F3;} + .d2-1108726992 .stroke-AA5{stroke:#DACEFB;} + .d2-1108726992 .stroke-AB4{stroke:#EA99C6;} + .d2-1108726992 .stroke-AB5{stroke:#FFDEF1;} + .d2-1108726992 .background-color-N1{background-color:#0A0F25;} + .d2-1108726992 .background-color-N2{background-color:#676C7E;} + .d2-1108726992 .background-color-N3{background-color:#9499AB;} + .d2-1108726992 .background-color-N4{background-color:#CFD2DD;} + .d2-1108726992 .background-color-N5{background-color:#DEE1EB;} + .d2-1108726992 .background-color-N6{background-color:#EEF1F8;} + .d2-1108726992 .background-color-N7{background-color:#FFFFFF;} + .d2-1108726992 .background-color-B1{background-color:#000536;} + .d2-1108726992 .background-color-B2{background-color:#0F66B7;} + .d2-1108726992 .background-color-B3{background-color:#4393DD;} + .d2-1108726992 .background-color-B4{background-color:#87BFF3;} + .d2-1108726992 .background-color-B5{background-color:#BCDDFB;} + .d2-1108726992 .background-color-B6{background-color:#E5F3FF;} + .d2-1108726992 .background-color-AA2{background-color:#7639C5;} + .d2-1108726992 .background-color-AA4{background-color:#C1A2F3;} + .d2-1108726992 .background-color-AA5{background-color:#DACEFB;} + .d2-1108726992 .background-color-AB4{background-color:#EA99C6;} + .d2-1108726992 .background-color-AB5{background-color:#FFDEF1;} + .d2-1108726992 .color-N1{color:#0A0F25;} + .d2-1108726992 .color-N2{color:#676C7E;} + .d2-1108726992 .color-N3{color:#9499AB;} + .d2-1108726992 .color-N4{color:#CFD2DD;} + .d2-1108726992 .color-N5{color:#DEE1EB;} + .d2-1108726992 .color-N6{color:#EEF1F8;} + .d2-1108726992 .color-N7{color:#FFFFFF;} + .d2-1108726992 .color-B1{color:#000536;} + .d2-1108726992 .color-B2{color:#0F66B7;} + .d2-1108726992 .color-B3{color:#4393DD;} + .d2-1108726992 .color-B4{color:#87BFF3;} + .d2-1108726992 .color-B5{color:#BCDDFB;} + .d2-1108726992 .color-B6{color:#E5F3FF;} + .d2-1108726992 .color-AA2{color:#7639C5;} + .d2-1108726992 .color-AA4{color:#C1A2F3;} + .d2-1108726992 .color-AA5{color:#DACEFB;} + .d2-1108726992 .color-AB4{color:#EA99C6;} + .d2-1108726992 .color-AB5{color:#FFDEF1;}.appendix text.text{fill:#0A0F25}.md{--color-fg-default:#0A0F25;--color-fg-muted:#676C7E;--color-fg-subtle:#9499AB;--color-canvas-default:#FFFFFF;--color-canvas-subtle:#EEF1F8;--color-border-default:#000536;--color-border-muted:#0F66B7;--color-neutral-muted:#EEF1F8;--color-accent-fg:#0F66B7;--color-accent-emphasis:#0F66B7;--color-attention-subtle:#676C7E;--color-danger-fg:red;}.sketch-overlay-B1{fill:url(#streaks-darker-d2-1108726992);mix-blend-mode:lighten}.sketch-overlay-B2{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-B3{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-B4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-B5{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-B6{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-AA2{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-AA4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-AA5{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-AB4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-AB5{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-N1{fill:url(#streaks-darker-d2-1108726992);mix-blend-mode:lighten}.sketch-overlay-N2{fill:url(#streaks-dark-d2-1108726992);mix-blend-mode:overlay}.sketch-overlay-N3{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-N4{fill:url(#streaks-normal-d2-1108726992);mix-blend-mode:color-burn}.sketch-overlay-N5{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-N6{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.sketch-overlay-N7{fill:url(#streaks-bright-d2-1108726992);mix-blend-mode:darken}.light-code{display: block}.dark-code{display: none}]]>Authenticated OTLP requestsspans, logs, metricsDurable write pathAuthoritative telemetryimmutable *.batch directoriesOrdinary Parquet files are authoritativeReadersBounded maintenanceRebuildable query statestorage.data_dir/queryDuckDB rollups and temporary spill. Not authoritative telemetry.Bounded group admissionconfigured row target + 20 ms windowUp to four commit workersindependent batches encode in parallelStaging batch directorymetadata + signal Parquet + trace.fidx for spansDurable publicationfsync → atomic rename → fsync parentpublished batch directories are discovered directly from the filesystemIndexed trace readbinary-search .fidx → Parquet rowsDuckDB SQLfilters, ordering, broad scansDuckDB rollupsrebuildable analytical cacheRetention + compaction admissionbounded batch and row groupsNative Parquet mergeprepared outside the reader gateCrash-safe namespace swapmarker + bounded renames + fsync acknowledge only after publicationOrdinary Parquet files are authoritative + - + DuckDB rollups and temporary spill. Not authoritative telemetry. - + - + - + diff --git a/site/src/components/landing/Benchmark.astro b/site/src/components/landing/Benchmark.astro new file mode 100644 index 00000000..be1514ba --- /dev/null +++ b/site/src/components/landing/Benchmark.astro @@ -0,0 +1,643 @@ +--- +/* A publication-candidate benchmark rendered as an instrument, not a claim card. + * + * Every number in this component comes from the same fanout-bench report and is + * also published in /explanation/performance, so no figure lives only in + * markup. The status line is deliberately impossible to miss: this branch + * previews the presentation before the required clean, repeated run replaces + * the candidate. Green remains a health/status colour, matching the product; + * throughput uses the site's violet interaction accent instead. + * + * The per-signal rates below are three separate saturation runs, not a + * breakdown of the sustained three-signal figure — they sum to roughly twice + * it, so the strip says so in text rather than leaving the reader to guess. + */ +const signals = [ + { name: "Traces", unit: "spans/s", value: "220,324" }, + { name: "Logs", unit: "records/s", value: "187,303" }, + { name: "Metrics", unit: "points/s", value: "209,638" }, +]; + +const gates = ["server drops 0", "no restarts", "Parquet verified"]; +--- + +
+
+ + Publication candidate + measured on one machine · methodology +
+ +
+
+

Sustained three-signal ingest

+

+ 296,196 items/s +

+

+ Fanout's accepted-row counter over five minutes—not the load + generator's requested rate. +

+ + +
+ +
+
+ Accepted throughput + items/s +
+ + Accepted three-signal throughput by generator process count + Throughput rose from 38,853 items per second with one generator process per signal to a 355,938 short peak with sixteen. The five-minute sustained rate at sixteen processes, the headline figure, was 296,196. The horizontal axis doubles at every step, so its points are evenly spaced. + + + 400k + 300k + 200k + 100k + 0 + + + + + + + + + + + + + + 1 + 2 + 4 + 8 + 16 + + generator processes per signal · doubling, evenly spaced + 355,938/s short peak + 296,196/s sustained over five minutes + +
+
+ +
+

+ Each signal was also saturated on its own. These three runs are separate + measurements, not a breakdown of the sustained three-signal rate above. +

+
+ {signals.map((signal) => ( +
+ {signal.name} alone + {signal.value} + {signal.unit} +
+ ))} +
+ Whole suite + 153.8M + items accepted, cumulative +
+
+
+ +
+
+ {gates.map((gate) => ( + {gate} + ))} +
+

v2026.8.11 · 8 logical CPUs · 15.6 GiB · same-host generators · OTLP/gRPC

+
+
+ + + + diff --git a/site/src/components/landing/Contrast.astro b/site/src/components/landing/Contrast.astro index c24d642d..bf7fd21d 100644 --- a/site/src/components/landing/Contrast.astro +++ b/site/src/components/landing/Contrast.astro @@ -22,8 +22,8 @@ const assembled = [ const single = [ { icon: "plugs-connected", part: "OTLP in over gRPC and HTTP" }, - { icon: "hard-drives", part: "Parquet on local disk" }, - { icon: "magnifying-glass", part: "DuckDB, in the same process" }, + { icon: "hard-drives", part: "Authoritative Parquet on local disk" }, + { icon: "magnifying-glass", part: "Indexed traces and DuckDB analytics" }, { icon: "chart-line", part: "The browser client, served by the binary" }, { icon: "bell", part: "The alert engine, on a fixed interval" }, ]; diff --git a/site/src/components/landing/Hero.astro b/site/src/components/landing/Hero.astro index d228a09d..9b31eb70 100644 --- a/site/src/components/landing/Hero.astro +++ b/site/src/components/landing/Hero.astro @@ -50,8 +50,9 @@ const { title = data.title, tagline, actions = [] } = data.hero ?? {}; ) }

- One Go process: OTLP in over gRPC or HTTP, Parquet on disk, DuckDB for - queries. No collector to deploy alongside it and no database to operate. + One Go process: OTLP in over gRPC or HTTP, authoritative Parquet on disk, + indexed trace reads, and embedded DuckDB analytics. No database to + operate.

diff --git a/site/src/components/landing/Pipeline.astro b/site/src/components/landing/Pipeline.astro index 32fa544f..c4f8ff04 100644 --- a/site/src/components/landing/Pipeline.astro +++ b/site/src/components/landing/Pipeline.astro @@ -39,8 +39,9 @@ const outPath = (y: number) => Traces, logs and metrics arrive over OTLP into a single Fanout process - that writes Parquet and queries it with DuckDB, and the answers leave to a - browser, the HTTP API, or an agent over MCP. + that publishes authoritative Parquet with trace indexes and uses DuckDB + for analytics, and the answers leave to a browser, the HTTP API, or an + agent over MCP. @@ -82,8 +83,8 @@ const outPath = (y: number) => stroke-width="1.5" > one process - Parquet on disk - DuckDB in-process + Parquet + trace indexes + DuckDB analytics OTLP diff --git a/site/src/content/docs/explanation/performance.mdx b/site/src/content/docs/explanation/performance.mdx index 9dbffb77..9512674c 100644 --- a/site/src/content/docs/explanation/performance.mdx +++ b/site/src/content/docs/explanation/performance.mdx @@ -1,53 +1,103 @@ --- title: Performance -description: Why Fanout publishes no throughput headline, and how to measure your own hardware instead. -summary: There is no official throughput number. The bundled driver measures your hardware, and the standard a published result would have to meet. +description: Fanout's benchmark candidate, its limits, and how to reproduce the measurement on your hardware. +summary: A qualified preview of Fanout's measured three-signal ingest, plus the evidence required before it becomes a published result. read_when: - - You are looking for a benchmark figure and cannot find one. + - You want to understand the performance figure on the landing page. - You need to size an instance for your own load. status: preview --- -Fanout does not publish a throughput headline. That is a decision, not an -omission, and it is worth stating plainly because the absence otherwise reads as -something to be embarrassed about. +Fanout has a publication candidate, not an official throughput claim yet. The +distinction is visible beside the number because provenance is part of a +benchmark result, not a footnote added after it. + +## Candidate result + +On a machine with eight logical CPUs and 15.6 GiB of memory, Fanout v2026.8.11 +accepted **296,196 OpenTelemetry items per second** for five minutes. The load +contained traces, logs and metrics at the same time, and Fanout and the load +generators shared the host. + +| Workload | Fanout accepted/s | Offered/s | Average Fanout CPU | Peak RSS | +|---|---:|---:|---:|---:| +| Traces, spans | 220,324 | 320,000 | 2.45 cores | 1.2 GiB | +| Logs, records | 187,303 | 320,000 | 3.04 cores | 1.3 GiB | +| Metrics, data points | 209,638 | 320,000 | 1.69 cores | 1.1 GiB | +| Three-signal short peak | 355,938 | 960,000 | 4.57 cores | 3.0 GiB | +| Three-signal sustained | 296,196 | 960,000 | 4.90 cores | 9.4 GiB | + +Accepted throughput is the change in Fanout's own +`fanout_ingest_rows_total` counter over the measured phase. It is not the rate +requested or reported by the generator. Across the complete saturation suite, +Fanout accepted 153.8 million items. + +The run passed its functional quality gates: every generator exited +successfully, Fanout reported zero dropped rows and no restart, and `fanout +repair verify` validated the authoritative Parquet store. + +### Generator load, not a Fanout scaling curve + +The chart on the landing page plots the same run's generator sweep: one, two, +four, eight and sixteen `telemetrygen` processes per signal, doubling at every +step. Accepted throughput rose from 38,853 items per second at one process per +signal to the 355,938 short peak at sixteen. That is a 9.2x gain for a 16x +increase in generators, and the shape is the point—below sixteen processes the +load generators, not Fanout, set the rate. Read it as evidence that the +saturating configuration was reached, not as a scaling law for Fanout itself. +The intermediate process counts are plotted from the same run and their exact +values ship with the raw artifacts. + +:::caution[Why this is not published evidence yet] +This was one repetition, and the benchmark harness it ran from carried +uncommitted local changes, so no commit identifies the code that produced it. +The result is a useful design and capacity preview, but it will not become the +official headline until the clean harness produces at least three passing +repetitions and the raw artifacts are published together. +::: -## The number that used to be here +## What this number does—and does not—measure -An earlier two-vCPU report quoted specific ingest and query figures. It was -withdrawn before the first public release, for reasons that had nothing to do -with the numbers being unflattering: +This is a **three-signal ingest** result: spans, log records and metric data +points were written concurrently. It is not a read/write result. Calling it a +mixed workload would imply that dashboard queries ran beside ingest, which +they did not. -- Its raw JSON reports and the exact benchmark-driver commit were not retained. -- Its ingest run predated mandatory ingest authentication, so it measured a path - the current release does not have. +The official read-under-write result will separately report requested, +completed, failed and shed query rates plus p50, p95 and p99 latency. Until that +result passes, Fanout makes no claim about interactive query capacity at the +headline write rate. -The prose was detailed. Detail is not provenance — a result nobody can reproduce -from the artifacts that produced it is an anecdote with decimal places. +The generators shared the Fanout host, so generator CPU and memory contention +can make the observed result lower than an isolated-server ceiling. Synthetic +`telemetrygen` payloads are also highly compressible, so this run is not a +storage-sizing formula for production data. -## What a published result would have to carry +## What the official result must carry -Before any figure goes back on this page, it has to ship with: +Before the candidate becomes Fanout's official headline, it has to ship with: -- the Fanout image digest and the benchmark-driver commit; +- the Fanout release artifact and checksum plus the benchmark-driver commit; - complete machine, network, container and Fanout configuration; -- the raw `ingest.json` and `mixed.json` reports from that same run; +- the raw `report.json`, `report.md` and `samples.jsonl` from that same run; - an authenticated OTLP path matching the current release; -- repeated runs, or an explicit note that it is a single run; -- separate ingest-only and mixed read/write results. +- at least three publish-preset repetitions and their variability; +- separate ingest-only and read-under-write results; +- the query completion rate and latency distribution beside any read claim; +- limitations stated next to the result rather than hidden at the bottom. -## Measure your own instead +## Reproduce it on your hardware The independent [Fanout Bench](https://github.com/labstack/fanout-bench) project drives the official OpenTelemetry `telemetrygen` client, counts rows accepted by Fanout, and emits raw JSON plus a publication-ready report. It can add authenticated dashboard reads on top, which is the part that matters. -Parquet ingest and DuckDB reads no longer serialize through one write gate, but -they still compete for the same CPU, memory bandwidth, filesystem cache, and -disk. Maintenance adds native Parquet merge work and short reader-exclusive -namespace changes. An ingest-only number therefore still overstates what an -instance does while anyone is looking at it. +Parquet ingest, indexed trace reads, and DuckDB analytics have separate +coordination paths, but they still compete for the same CPU, memory bandwidth, +filesystem cache, and disk. Maintenance adds native Parquet merge work and +short reader-exclusive namespace changes. An ingest-only number therefore +still overstates what an instance does while anyone is looking at it. Create a short-lived ingest token through first-admin setup, configure Fanout with a dedicated `FANOUT_METRICS_TOKEN`, then expose that same metrics token to @@ -64,8 +114,19 @@ fanout-bench run \ --same-host ``` -The harness repository documents signal-specific saturation, sustained mixed -load, read-under-write latency, machine identity, and publication rules. +The harness repository documents signal-specific saturation, sustained +three-signal load, read-under-write latency, machine identity, and publication +rules. For a publication-quality run, use the `publish` preset and retain all +three repetitions: + +```sh +fanout-bench run \ + --host your-benchmark-host \ + --fanout-version v2026.8.11 \ + --preset publish \ + --repeat 3 \ + --output results/fanout-v2026.8.11 +``` :::caution[A result describes the machine it ran on] Benchmark output describes the hardware and dataset you tested, not an @@ -75,7 +136,7 @@ Fanout is sensitive to all three — [storage](/explanation/storage-model) explains why. ::: -## What to watch instead of a headline +## What to watch in production For capacity, the useful signals are the ones an instance emits about itself: telemetry drops, sustained query latency, readiness failures, and free disk. diff --git a/site/src/content/docs/explanation/storage-model.mdx b/site/src/content/docs/explanation/storage-model.mdx index 3d71ac76..06a91559 100644 --- a/site/src/content/docs/explanation/storage-model.mdx +++ b/site/src/content/docs/explanation/storage-model.mdx @@ -8,20 +8,19 @@ read_when: status: preview --- -Fanout uses a small embedded storage stack, not a data lake. Telemetry is retained -as ordinary Parquet files and queried by DuckDB in the same process. Application -state—users, sessions, dashboards, alert rules, and agent history—lives in SQLite. +Fanout retains telemetry as ordinary Parquet files in self-contained atomic +batch directories. Persistent sidecars index traces, and DuckDB handles SQL, +broad scans, and rebuildable analytical rollups in the same process. +Application state—users, sessions, dashboards, alert rules, and agent +history—lives in SQLite. | Component | Purpose | |---|---| | Atomic Parquet batches | Authoritative retained spans, logs, and metrics | | Trace sidecar | Persistent row-range index for targeted trace reads from Parquet | -| DuckDB | SQL, log filtering, aggregation, and broad analytical reads | +| DuckDB | SQL, log filtering, broad analytical reads, and rebuildable rollups | | SQLite | Transactional application and identity state | -There is no Iceberg, DuckLake, external catalog, or server database in the -telemetry path. -
-
+
-## Three signals in, one process, answers back out. +## Nearly 300,000 OpenTelemetry items a second. One binary. -Fanout is where your telemetry lands, not something you put in front of the -thing it lands in. Traces, logs and metrics arrive over standard OTLP from SDKs -or a Collector you already run, and everything after that happens inside one -binary. - - +In a five-minute publication-candidate run, Fanout accepted **296,196 spans, +log records and metric data points per second** on a machine with eight logical +CPUs and 15.6 GiB of memory. +Fanout reported zero dropped rows, the process did not restart, and the +resulting Parquet store passed verification. The load generators shared the +machine, so this is an observed whole-host result—not an isolated +storage-engine ceiling. -Nothing in the middle crosses a network, and nothing else has to be running for -it to answer. The data directory it writes is the whole of its state. + - +It is labeled a candidate because it is a single run: the published headline +waits on a clean harness repeating it three times. +[Read the methodology and every qualification](/explanation/performance), or +[run the same open-source harness](https://github.com/labstack/fanout-bench) on +your own host.
@@ -101,6 +106,24 @@ integration layer to debug at three in the morning.
+
+ +## Three signals in, one process, answers back out. + +Fanout is where your telemetry lands, not something you put in front of the +thing it lands in. Traces, logs and metrics arrive over standard OTLP from SDKs +or a Collector you already run, and everything after that happens inside one +binary. + + + +Nothing in the middle crosses a network, and nothing else has to be running for +it to answer. The data directory it writes is the whole of its state. + + + +
+
## What Fanout will not do. @@ -112,7 +135,7 @@ None of them is a roadmap item phrased as a virtue. - **Not multi-tenant:** namespaces scope queries, not access. Anyone who can query the instance can query every namespace in it; an instance is one trust domain. - **No horizontal scale:** ingest, query and maintenance share a process and a machine. There is no scaling one without the others, because they are the same thing. -- **No published throughput figure:** the previous benchmark was withdrawn because its raw reports and driver commit were not retained. [Measure your own hardware](/explanation/performance) instead. +- **No universal capacity number:** the benchmark above describes one release, host and workload. Your payload cardinality, query mix and disk decide the capacity of your instance. [Measure your own hardware](/explanation/performance). - **No migration paths:** Fanout is pre-release. A renamed setting is renamed, not aliased, which is why an unknown variable stops the process rather than being ignored. diff --git a/site/src/content/docs/reference/data-layout.mdx b/site/src/content/docs/reference/data-layout.mdx index 2b547e8a..0a07111a 100644 --- a/site/src/content/docs/reference/data-layout.mdx +++ b/site/src/content/docs/reference/data-layout.mdx @@ -62,7 +62,7 @@ as much as byte count for query latency, which is what the compaction pass exist bound. [Tune retention](/guides/tune-retention) covers both. :::caution[Do not edit these files] -Fanout owns this directory. Editing the SQLite databases directly — deleting a -user row, for instance — cascades in ways the application would have handled and -a manual edit will not. +Fanout owns this directory. Editing control SQLite, DuckDB query state, or +telemetry batches directly can violate application and storage invariants. Use +the documented commands and APIs instead. ::: diff --git a/site/src/content/docs/reference/settings/storage.mdx b/site/src/content/docs/reference/settings/storage.mdx index 87809fdf..b40862ef 100644 --- a/site/src/content/docs/reference/settings/storage.mdx +++ b/site/src/content/docs/reference/settings/storage.mdx @@ -47,4 +47,4 @@ Controls Parquet retention and compaction, and query-cache checkpointing. ### `storage.rollup_skip_to_latest` -RollupSkipToLatest, set once at boot, advances every rollup watermark to the current max ingested timestamp so existing data is treated as already-rolled-up instead of aggregated as a backlog. Stands up a large pre-seeded historical dataset (benchmarks, restores) without a multi-minute first-rollup catch-up that holds the write gate and starves ingest. Off in normal operation. +RollupSkipToLatest, set once at boot, advances every rollup watermark to the current max ingested timestamp so existing data is treated as already-rolled-up instead of aggregated as a backlog. Stands up a large pre-seeded historical dataset (benchmarks, restores) without a multi-minute first-rollup catch-up that monopolizes DuckDB write capacity. Off in normal operation. diff --git a/site/src/content/docs/start/send-telemetry.mdx b/site/src/content/docs/start/send-telemetry.mdx index d2c4244d..04ff2dc8 100644 --- a/site/src/content/docs/start/send-telemetry.mdx +++ b/site/src/content/docs/start/send-telemetry.mdx @@ -18,18 +18,12 @@ Fanout accepts traces, logs and metrics over OTLP on two listeners: gRPC on The raw ingest token is shown once, immediately after you create the first administrator. It is `fo_`-prefixed, and Fanout stores only its hash. -Send it as a bearer token: +Send it as a bearer token; this is the only accepted ingest credential form: ```text Authorization: Bearer fo_ ``` -:::caution[The `x-fanout-ingest-token` header was removed] -Earlier builds also accepted `x-fanout-ingest-token`. That header is no longer -accepted on either listener — an exporter still sending it is rejected as -unauthenticated. `Authorization: Bearer` is the only form. -::: - Two things are worth knowing before you debug a rejection: - **Browser sessions and MCP tokens are not ingest credentials.** They are diff --git a/site/src/content/docs/start/what-fanout-is.mdx b/site/src/content/docs/start/what-fanout-is.mdx index e572f721..0468d837 100644 --- a/site/src/content/docs/start/what-fanout-is.mdx +++ b/site/src/content/docs/start/what-fanout-is.mdx @@ -1,6 +1,6 @@ --- title: What Fanout is -description: One Go process that ingests OpenTelemetry, stores it as Parquet, queries it with DuckDB, and answers questions about it. +description: One Go process that ingests OpenTelemetry, publishes authoritative Parquet, and answers questions through indexed reads and embedded analytics. summary: The scope of the product in one page — what the binary contains, what it deliberately does not, and who it is for. read_when: - You are deciding whether Fanout fits before installing anything. @@ -9,9 +9,9 @@ status: preview --- Fanout is a single self-hosted binary for OpenTelemetry. One Go process accepts -OTLP over gRPC and HTTP, writes telemetry to disk as Parquet, answers queries -with DuckDB, evaluates alert rules, serves a chat investigator and MCP tools, -and hosts the browser client. +OTLP over gRPC and HTTP, publishes atomic Parquet batches with persistent trace +indexes, runs DuckDB analytics and rollups, evaluates alert rules, serves a chat +investigator and MCP tools, and hosts the browser client.
diff --git a/site/src/content/docs/status/capabilities.mdx b/site/src/content/docs/status/capabilities.mdx index 3dfad479..1e9ac349 100644 --- a/site/src/content/docs/status/capabilities.mdx +++ b/site/src/content/docs/status/capabilities.mdx @@ -28,11 +28,10 @@ inline, so something reading the index knows before it fetches the page. Every page states its status explicitly rather than relying on the default. That is deliberate: `shipped` is the schema's default, so a page that says nothing asserts stability by silence — which is the opposite of what a -pre-release product should do. Most pages here are `preview`, and the reasons -are concrete rather than cautious. In the week this documentation was written, -the ingest header was replaced, `FANOUT_MCP_PUBLIC_URL` was removed outright, -namespace query scoping inverted, and `viewer` gained the ability to run the -agent. Pages describing those surfaces are marked accordingly. +pre-release product should do. Most pages here are `preview` because their +user-facing contracts are still being refined. The marker describes the +current stability of a capability; it does not imply an unlisted compatibility +surface. `shipped` is reserved for what has not moved and is not expected to: what Fanout is, the release and install mechanics, and backing up the data @@ -43,7 +42,7 @@ directory. | Capability | Status | Notes | |---|---|---| | OTLP ingest over gRPC and HTTP | shipped | Both listeners, one ingest token. | -| Parquet storage and DuckDB queries | shipped | Local data directory; no external database. | +| Parquet storage, trace indexes, and DuckDB analytics | shipped | Authoritative local Parquet; rebuildable DuckDB query state. | | Browser client | preview | Ships in the binary; surfaces still moving. | | Chat investigator | preview | Requires an AI provider key; disabled and logged without one. | | MCP tools | preview | Served at `/mcp`; the tool set is not frozen. | @@ -54,7 +53,7 @@ directory. | OIDC sign-in | shipped | Provider flow, with group and domain mapping. | | Generated settings reference | shipped | `cmd/fanout-docgen` writes it; `just check` fails when it drifts from the type. | | Release checksums and provenance | shipped | `SHA256SUMS` plus GitHub build attestations on every archive. | -| Published throughput figure | planned | Withdrawn before release; the standard a new one must meet is on [Performance](/explanation/performance). | +| Publication benchmark | preview | A qualified candidate is visible on [Performance](/explanation/performance); repeatable public artifacts are still required. | | Signatures and SBOM | planned | Deliberately deferred until something consumes them. | | Guides for every operational task | preview | The query surface — what the browser client actually shows — has no page yet. | | Namespace scoping on alert rules | planned | The field is stored but evaluation ignores it; scope with `service`. | diff --git a/site/src/pages/llms.txt.ts b/site/src/pages/llms.txt.ts index 55858515..5da3603f 100644 --- a/site/src/pages/llms.txt.ts +++ b/site/src/pages/llms.txt.ts @@ -44,10 +44,9 @@ export const GET: APIRoute = async ({ site }) => { "# Fanout", "", "One self-hosted binary for OpenTelemetry. The same Go process accepts OTLP", - "over gRPC and HTTP, writes telemetry to disk as Parquet, answers queries with", - "DuckDB, evaluates alert rules, serves a chat investigator and MCP tools, and", - "hosts the browser client. There is no collector to deploy alongside it and no", - "database to operate.", + "over gRPC and HTTP, publishes authoritative Parquet batches with persistent", + "trace indexes, runs embedded DuckDB analytics and rollups, evaluates alert", + "rules, serves a chat investigator and MCP tools, and hosts the browser client.", "", "> Use this file as a map of the Fanout documentation. Fetch any page as", "> Markdown by appending `.md` to its URL. Fanout is pre-release: entries",