Skip to content

GS: Failure-safe full refresh and large-result transfer #402

Description

@BorisTyshkevich

Parent: #398
Depends on: #399, #401

Design:

  • docs/design/google-sheets/full-refresh.md
  • docs/design/google-sheets/large-results.md
  • docs/design/google-sheets/failure-recovery.md

Scope

Implement manual full refresh with streaming ClickHouse reads, canonical value conversion, hidden staging, stable-sheet publication, limits, progress, cancellation, and failure recovery.

Acceptance criteria

  • The complete query result is never required in browser memory.
  • Google writes use byte-bounded explicit ranges and RAW semantics.
  • Existing live data remains unchanged until all query and staging work succeeds.
  • Final publication preserves the managed numeric sheet ID and atomically updates state.
  • Old trailing rows/columns are cleared and formatting/metadata are refreshed.
  • Ambiguous publication is reconciled by generation/refresh ID before retry.
  • Row, cell, byte, duration, and spreadsheet-capacity limits are enforced without truncation.
  • Cancellation and closed-tab recovery leave committed data intact and clean orphan staging later.
  • Successful full refresh can initialize/reset an append checkpoint when eligible.

Non-goals

Append page selection, automatic scheduling, or in-place partial replacement when staging cannot fit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions