Skip to content

feat(assistant): add the in-page AI assistant to docs.celo.org (#2250) - #2286

Open
GigaHierz wants to merge 5 commits into
mainfrom
GigaHierz/issue-2250-options
Open

feat(assistant): add the in-page AI assistant to docs.celo.org (#2250)#2286
GigaHierz wants to merge 5 commits into
mainfrom
GigaHierz/issue-2250-options

Conversation

@GigaHierz

@GigaHierz GigaHierz commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Closes the implementation half of #2250. The research, comparison table and decision are recorded on the issue.

What this adds

  • assistant.js — a loader that injects the assistant widget on every page. Mintlify includes every .js file in the content directory site-wide but doesn't support a raw <script src> in MDX, so the tag is created programmatically.
  • use-docs-with-ai.mdx — documents the Ask AI button alongside the existing MCP / llms.txt entry points.

The widget and its API live in celo-org/docs-ai-assistant, deliberately outside this repo — any .js committed here is injected into the live site.

How it works

There is no second content pipeline. The backend answers using the Mintlify-hosted MCP endpoint at docs.celo.org/mcp, which is reindexed on every docs deploy, so the assistant can never serve a stale index. Citations come free because the MCP search tool returns page URLs.

Models are Haiku 4.5 by default with a Sonnet 5 escalation when an answer cites nothing.

Verified against the live deployment

Check Result
Answer quality Grounded, 8.3s, citations resolve 200
Escalation path Sonnet 5, 4 citations
Off-topic question Refuses rather than guessing
Origin allowlist Non-docs origins get 403
Rate limiting Exactly 15 requests / 10 min per IP, then 429
submit_feedback MCP tool Denylisted — not exposed to the model
Dependencies 0 vulnerabilities

Before merging

The assistant currently runs on a temporary, rate-limited OpenRouter key for testing. Merging puts the widget on every docs page, so the sequence should be:

  1. Swap in a dedicated API key with a monthly spend cap.
  2. Point WIDGET_SRC at docs-assistant.celo.org once IT creates the CNAME (currently the Vercel URL — a one-line change).

Merging also confirms the last open question: whether content-directory JS injection works on our Mintlify plan. The pricing table and the already-injected style.css both say yes, but preview deployments are Pro-gated, so production is the only conclusive test. Open any page's console and look for the widget.

🤖 Generated with Claude Code

Trivial marker script to confirm Mintlify's content-directory JS
injection works on the current plan before building the real widget
loader.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@GigaHierz
GigaHierz marked this pull request as ready for review August 25, 2026 14:33
@GigaHierz
GigaHierz requested a review from a team as a code owner August 25, 2026 14:33
@GigaHierz
GigaHierz requested a review from palango August 25, 2026 14:53
GigaHierz and others added 2 commits August 26, 2026 09:19
Replaces the injection-test marker with the real loader. The widget is
served from the assistant deployment and answers from the docs MCP
endpoint with links back to the pages it used.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the Ask AI button to the entry-point table and a short section
covering what it answers from, that it cites its sources, and that
uncited gaps feed back into what gets written.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GigaHierz GigaHierz changed the title test(assistant): verify custom JS injection on current Mintlify plan (#2250) feat(assistant): add the in-page AI assistant to docs.celo.org (#2250) Aug 26, 2026
GigaHierz and others added 2 commits August 27, 2026 20:23
…org host

Swap the temporary Vercel preview URL for the production CNAME now that it
resolves (200, application/javascript).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@palango palango left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid research behind this, and the #2250 comparison answers most of what I'd otherwise have asked: Mintlify's own assistant ruled out on price, custom JS confirmed on Starter, and the MCP endpoint as the backend so there's no second index to go stale. Three things block merge for me, then a list of smaller fixes.

The widget breaks outside production

assistant.js injects on every host, but the API only ever allows docs.celo.org:

$ curl -s -X OPTIONS -H "Origin: http://localhost:3000" \
       -H "Access-Control-Request-Method: POST" \
       -D- -o /dev/null https://docs-ai-assistant-ten.vercel.app/api/chat
access-control-allow-origin: https://docs.celo.org
vary: Origin

widget.js is a classic script, so it loads anywhere regardless. On mint dev, which AGENTS.md tells everyone to run before opening a PR, the Ask AI button appears, accepts a question, and then the preflight fails and widget.js:184 writes the raw browser error into the chat bubble as if it were the assistant's answer. Same on any preview build, which is why no reviewer can actually try the feature this PR adds.

if (location.hostname !== 'docs.celo.org') return;

The Note promises something the widget contradicts

If the docs don't cover your question, the assistant says so instead of guessing

widget.js:181 tests the finished answer for a docs.celo.org citation and, finding none, calls offerEscalation(), which renders "No docs pages cited. / Try a more thorough answer" and re-asks with the stronger model. An uncited answer is a case the UI is built to handle, so it isn't a refusal.

Where the sentence sits is what makes this matter. It lands directly after "Follow the citations before relying on anything load-bearing, especially contract addresses and code", so it withdraws the page's only safety caution. A reader told the assistant won't guess has no reason to check the citation on an answer that invented a checksummed address. I'd describe the escalation button and drop the guarantee.

Same paragraph: "that gap is a signal we use to decide what to write next" describes a loop I can't find. The widget's only network destination is /api/chat, and there's no telemetry in the diff. If the backend logs unanswered questions, say where they land.

Land the CNAME before this merges

The body already flags docs-ai-assistant-ten.vercel.app as temporary. The trouble is that performing the swap is what creates the exposure: renaming or deleting the Vercel project releases that auto-generated hostname, and whoever registers it next serves arbitrary JS on every docs page with same-origin DOM access, including the pages that print contract addresses and RPC endpoints. There's no SRI, no crossorigin, and no CSP anywhere in the repo to fall back on (no vercel.json, no _headers, no middleware, and docs.json has no headers key).

Shipping docs-assistant.celo.org from the start avoids the window. If this has to merge first, please file the two "Before merging" steps as issues, because a squash deletes the body that currently holds them.

Smaller, all fixable here

  • AGENTS.md is now wrong about its own subject. Line 124's AI tooling row lists five entry points, and line 139 closes with "the other two entry points", a fixed count. Both need the assistant.
  • The frontmatter description still reads "the docs MCP server, llms.txt, and per-page Markdown", so the new entry point is invisible in llms.txt, which AGENTS.md:47 calls the thing an agent sees before deciding to open a page.
  • The table's URL column now holds **Ask AI** button on any page in its first row, and the table ships verbatim in llms-full.txt, so an agent reading that column for endpoints gets a button label back. Renaming the header to "Where" fixes all five rows at once.
  • No onerror on the append, while the prose promises a button in a named corner. An adblocker on *.vercel.app, a Vercel outage, or a 429 from the 15-per-10-minutes limit (shared office IPs will hit that) all end with no button, no signal, and a reader who concludes their browser is broken.
  • script.async = true does nothing here; scripts built with createElement are already async.
  • Worth setting script.dataset.apiUrl, which the widget already supports, rather than leaning on document.currentScript.

Worth follow-up issues

This is the first executable code in the repo (git ls-tree -r origin/main finds zero .js), and nothing records that anywhere. AGENTS.md §1 doesn't mention that a .js under the content root runs on every published page, CODEOWNERS has no rule for executable assets so a root .js draws the same catch-all review as prose, and CI can't see .js at all. Worth remembering submodules/developer-tooling sits under that root too.

On privacy: the Note covers accuracy but not that the question leaves the site. There's no privacy policy anywhere on docs.celo.org, GA4 landed three commits ago, and the loader sets no referrerPolicy, so the host collects IP plus exact page for every reader with no interaction at all. A line saying where questions go, plus a warning against pasting secrets next to the input, would cover it.

Finally, #2285 adds seo-schema.js with the same injection bootstrap and its own restatement of the Mintlify mechanism. Whichever of the two merges second, it would be good to document that mechanism once in AGENTS.md and have both files point at it.

Nits: Refs #2250 rather than "Closes the implementation half", since the ops boxes are still open, and AGENTS.md §9 asks for mint broken-links output in the body.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants