Skip to content

feat(search): client side fts - #22

Open
larbish wants to merge 13 commits into
mainfrom
feat/client-side-fts-search
Open

feat(search): client side fts#22
larbish wants to merge 13 commits into
mainfrom
feat/client-side-fts-search

Conversation

@larbish

@larbish larbish commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Move search to client-side full-text search: a web worker owns a browser comark-content instance (sqlite-wasm FTS5) hydrated from per-commit snapshot artifacts, replacing the search-sections endpoint that shipped a flat section list on every page hydration.

Built on comarkdown/comark-content#109 (persisted snapshot & manifest artifacts): the webhook parses once per content push and update the cached artifactfs so consumers reuse the cached version.

How it works

Client.

useSearch pins hydration to the production commit via the new /api/content/head endpoint, then a worker fetches /api/content/blob/<sha>/manifest.json and /snapshot/content.json (SHA-pinned, immutable, isr: true cached at the CDN forever) and builds the FTS index through the plugin's cache-first path.

Queries run in the worker: BM25 ranking, heading-weighted scores, snippets.

Server.

The webhook's cache warm becomes warmSnapshot(): one full parse per push, which persists the snapshot artifact into the per-SHA Runtime Cache namespace. The first request on a new SHA is a single cache read, then ISR pins the URL; old SHA URLs simply become unreachable, so search needs no purges. Head-of-branch /blob/<sha> requests reuse the shared prod instance instead of minting a duplicate preview instance. The search-sections plugin, endpoint, route rules, and per-page payload are deleted.

Debug mode

On by default in dev; in production via ?debug=search. Both threads log onto one page-relative timeline — fixed-width lines showing stage durations, decoded/wire sizes, and result counts:

[search]    +412ms  manifest              14.2ms     1.3 KB  1.0 KB on the wire
[search]   +1284ms  index:built          118.4ms             net of the snapshot fetch above
[search]   +1290ms  index:rows                                87 results · sections in the FTS table

Next steps to discuss

  • Extract the search stack into a plugin — useful for nuxt.com (migration on comark-content in progress).

  • Version-aware search — the hydration URLs are already per-SHA (/api/content/blob/<sha>/…), so searching any historical version is just pointing the worker at that commit's artifacts, which the /blob/<sha> routes already serve and self-heal on first request. Would pair with the existing version-history UI; main open question is worker lifecycle when switching versions (one instance per SHA vs. reset + rehydrate).

@vercel

vercel Bot commented Aug 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
comark-docs-playground Ready Ready Preview Aug 20, 2026 2:30pm

@pkg-pr-new

pkg-pr-new Bot commented Aug 19, 2026

Copy link
Copy Markdown
npm i https://pkg.pr.new/comarkdown/comark-docs@22

commit: 6b95a04

@larbish
larbish requested review from atinux and farnabaz August 19, 2026 16:58
@larbish
larbish marked this pull request as ready for review August 19, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants