feat(search): client side fts - #22
Open
larbish wants to merge 13 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Move search to client-side full-text search: a web worker owns a browser
comark-contentinstance (sqlite-wasm FTS5) hydrated from per-commit snapshot artifacts, replacing thesearch-sectionsendpoint that shipped a flat section list on every page hydration.Built on comarkdown/comark-content#109 (persisted snapshot & manifest artifacts): the webhook parses once per content push and update the cached artifactfs so consumers reuse the cached version.
How it works
Client.
useSearchpins hydration to the production commit via the new/api/content/headendpoint, then a worker fetches/api/content/blob/<sha>/manifest.jsonand/snapshot/content.json(SHA-pinned, immutable,isr: truecached at the CDN forever) and builds the FTS index through the plugin's cache-first path.Queries run in the worker: BM25 ranking, heading-weighted scores, snippets.
Server.
The webhook's cache warm becomes
warmSnapshot(): one full parse per push, which persists the snapshot artifact into the per-SHA Runtime Cache namespace. The first request on a new SHA is a single cache read, then ISR pins the URL; old SHA URLs simply become unreachable, so search needs no purges. Head-of-branch/blob/<sha>requests reuse the shared prod instance instead of minting a duplicate preview instance. Thesearch-sectionsplugin, endpoint, route rules, and per-page payload are deleted.Debug mode
On by default in dev; in production via
?debug=search. Both threads log onto one page-relative timeline — fixed-width lines showing stage durations, decoded/wire sizes, and result counts:Next steps to discuss
Extract the search stack into a plugin — useful for nuxt.com (migration on comark-content in progress).
Version-aware search — the hydration URLs are already per-SHA (
/api/content/blob/<sha>/…), so searching any historical version is just pointing the worker at that commit's artifacts, which the/blob/<sha>routes already serve and self-heal on first request. Would pair with the existing version-history UI; main open question is worker lifecycle when switching versions (one instance per SHA vs. reset + rehydrate).