Skip to content

Document copy-db's blob companion directory and restore steps - #620

Draft
kriszyp wants to merge 1 commit into
mainfrom
kris/copy-db-blob-companion
Draft

Document copy-db's blob companion directory and restore steps#620
kriszyp wants to merge 1 commit into
mainfrom
kris/copy-db-blob-companion

Conversation

@kriszyp

@kriszyp kriszyp commented Aug 6, 2026

Copy link
Copy Markdown
Member

Companion to HarperFast/harper#2098 (fixes HarperFast/harper#2048).

copy-db previously left the database's file-backed blobs behind, so a copy moved to another host, or restored under a different database name, silently lost every blob. It now copies each blob root to <target>-blobs/<rootIndex>/ alongside the database file, and refuses a target that already exists rather than merging the copy into it.

Documents:

  • the blob companion directory, its <rootIndex> layout, and the restore steps (including restoring under a different database name, which is where the blob roots have to be placed by hand)
  • that the target path must not already exist
  • that copy compaction is LMDB-only — RocksDB databases compact themselves

Generated by Claude Opus 5.

copy-db now writes the database's blob roots to `<target>-blobs/<rootIndex>/`
and refuses a target that already exists (HarperFast/harper#2048). Blob files
live outside the database file and are addressed by database name, so a copy is
not restorable without them — document the layout, the restore steps for
restoring under a different name, and that copy compaction is LMDB-only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the documentation for the harper copy-db command and database compaction, adding details about file-backed blobs, target path constraints, and database engine applicability (LMDB vs. RocksDB). The review feedback suggests adding <VersionBadge> elements to properly document these behavior changes on existing surfaces, following repository guidelines.

Comment thread reference/cli/commands.md

This copies the default `data` database to a new location with compaction applied.

The database's file-backed blobs are copied to `<target-database-path>-blobs/<rootIndex>/`, since blob files live outside the database file and are addressed by database name. The copy cannot be restored without that directory — see [Database Compaction](../database/compaction.md#file-backed-blobs-travel-separately) for the restore steps. LMDB databases only; RocksDB databases compact themselves.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository's guidelines, when documenting behavior changes to an existing surface (such as the new blob copying behavior and RocksDB exclusion in copy-db), we should use the <VersionBadge type="changed" version="vX.Y.0" /> format. Please add the appropriate version badge (e.g., v5.3.0 or the target release version) to indicate this change.

Suggested change
The database's file-backed blobs are copied to `<target-database-path>-blobs/<rootIndex>/`, since blob files live outside the database file and are addressed by database name. The copy cannot be restored without that directory — see [Database Compaction](../database/compaction.md#file-backed-blobs-travel-separately) for the restore steps. LMDB databases only; RocksDB databases compact themselves.
<VersionBadge type="changed" version="v5.3.0" /> — The database's file-backed blobs are copied to <target-database-path>-blobs/<rootIndex>/, since blob files live outside the database file and are addressed by database name. The copy cannot be restored without that directory — see [Database Compaction](../database/compaction.md#file-backed-blobs-travel-separately) for the restore steps. LMDB databases only; RocksDB databases compact themselves.
References
  1. Use the format when documenting behavior changes to existing surface, as prescribed in the repository's guidelines.


### File-backed blobs travel separately

A database's file-backed blob values (`Blob` and large `Bytes` attributes) are not stored inside the database file. They live in the configured blob roots — `storage.blobPaths[n]`, or `<rootPath>/blobs/<database>` when `blobPaths` is not configured — and are addressed by **database name**, not by the path of the database file.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository's guidelines, when documenting behavior changes to an existing surface (such as the new file-backed blobs behavior), we should use the <VersionBadge type="changed" version="vX.Y.0" /> format. Please add the appropriate version badge (e.g., v5.3.0 or the target release version) to indicate this change.

Suggested change
A database's file-backed blob values (`Blob` and large `Bytes` attributes) are not stored inside the database file. They live in the configured blob roots — `storage.blobPaths[n]`, or `<rootPath>/blobs/<database>` when `blobPaths` is not configured — and are addressed by **database name**, not by the path of the database file.
<VersionBadge type="changed" version="v5.3.0" /> — A database's file-backed blob values (Blob and large Bytes attributes) are not stored inside the database file. They live in the configured blob roots — storage.blobPaths[n], or <rootPath>/blobs/<database> when blobPaths is not configured — and are addressed by **database name**, not by the path of the database file.
References
  1. Use the format when documenting behavior changes to existing surface, as prescribed in the repository's guidelines.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-620

This preview will update automatically when you push new commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

copy-db produces a silently corrupt, non-restorable copy and exits 0 — four independent channels in bin/copyDb.ts

1 participant