From d12aa17c5febf400c14b98023055052053796e95 Mon Sep 17 00:00:00 2001 From: Anne Chew Date: Thu, 20 Aug 2026 14:55:37 +0800 Subject: [PATCH] docs(AI): added skill and pipeline for AI audit Added agent, skill, and prompts to audit, review and rewrite docs to be more AI-inference-friendly. Signed-off-by: Anne Chew --- .github/copilot-instructions.md | 9 ++ .../skills/docs-ai-friendly-rewrite/SKILL.md | 150 ++++++++++++++++++ .../references/audit_prompt.md | 91 +++++++++++ .../references/consistency_check_prompt.md | 57 +++++++ .../references/rewrite_prompt.md | 83 ++++++++++ .gitignore | 3 + AGENTS.md | 83 ++++++++++ README.md | 40 ++++- .../source-code/obtain-kernel-source-git.md | 59 ++++--- prompts/audit-docs.md | 60 +++++++ prompts/consistency-check-docs.md | 51 ++++++ prompts/rewrite-docs.md | 61 +++++++ 12 files changed, 726 insertions(+), 21 deletions(-) create mode 100644 .github/skills/docs-ai-friendly-rewrite/SKILL.md create mode 100644 .github/skills/docs-ai-friendly-rewrite/references/audit_prompt.md create mode 100644 .github/skills/docs-ai-friendly-rewrite/references/consistency_check_prompt.md create mode 100644 .github/skills/docs-ai-friendly-rewrite/references/rewrite_prompt.md create mode 100644 .gitignore create mode 100644 AGENTS.md create mode 100644 prompts/audit-docs.md create mode 100644 prompts/consistency-check-docs.md create mode 100644 prompts/rewrite-docs.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 345cc2d..11f723b 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -235,3 +235,12 @@ Use short section prefixes for anchors when naming labels, for example `tutorial - Don't assume reader knowledge without brief explanations or links. - Don't link to blog posts when official docs exist. - Don't treat this repository as a mixed code-and-docs project. + +## AI-inference-friendly docs workflow + +When asked to make docs "AI-friendly," "LLM-friendly," or "inference-friendly," +use the `docs-ai-friendly-rewrite` skill (`.github/skills/`) and follow +`AGENTS.md`. That workflow keeps its own working files under +`.github/ai-friendly-audit/` (audits, an internal working term list, +reports) — separate from `docs/reference/glossary.md`, which serves a +different purpose and is not read or edited by this workflow. diff --git a/.github/skills/docs-ai-friendly-rewrite/SKILL.md b/.github/skills/docs-ai-friendly-rewrite/SKILL.md new file mode 100644 index 0000000..3be11d9 --- /dev/null +++ b/.github/skills/docs-ai-friendly-rewrite/SKILL.md @@ -0,0 +1,150 @@ +--- +name: docs-ai-friendly-rewrite +description: Rewrite existing documentation so LLMs can reason and infer from it, not just retrieve matching snippets. Use whenever asked to make docs "AI-friendly," "LLM-friendly," or "inference-friendly," to answer questions the docs don't state verbatim, or to audit/rewrite documentation for AI consumption. Applies to this repo's Sphinx/MyST/RST Diátaxis content under docs/. +--- + +# Docs → AI-Friendly (Inference-Ready) Rewrite + +Follows the open Agent Skills format (agentskills.io). This version is +tailored to this repo's conventions in `.github/copilot-instructions.md` +— read that file first; this skill builds on it, not around it. + +## The core distinction + +**Retrieval-friendly** docs (optimized for search/RAG): short, +self-contained chunks, keyword-rich, one topic per chunk. + +**Inference-friendly** docs (optimized for reasoning): explicit +definitions, explicit conditional logic ("if X then Y, unless Z"), +explicit scope/boundaries, consistent terminology across the whole +corpus, and stated rationale — so a model can correctly answer +questions the docs never spell out verbatim. + +Most teams only optimize for the first. This skill adds the second, on +top of the first — it's an audit-and-rewrite pass, not a +re-architecture, and it does not change this repo's Diátaxis structure, +formatting rules, or build tooling. + +## Repo-specific conventions this workflow uses + +- **Internal working glossary**: `.github/ai-friendly-audit/glossary.md` + — a term-consistency working file for this pipeline only. It is + separate from `docs/reference/glossary.md`, which is the reader- + facing glossary and serves a different purpose; this workflow does + not read, write, or otherwise touch that file. Spelling + exceptions/valid technical terms still go in + `docs/.custom_wordlist.txt` as normal. +- **Audit/report output**: `.github/ai-friendly-audit/` — never under + `docs/`, since anything under `docs/` is picked up by the Sphinx build. +- **Front matter**: don't add new front-matter keys. Keep the required + MyST/RST meta description front matter untouched. +- **Diátaxis category**: preserve it. A how-to rewritten for clarity is + still a how-to, not reference material. +- **Formatting**: keep all rules from `copilot-instructions.md` — sentence + case headings, active voice, blank line after headings/before lists, + section-prefixed cross-reference labels, no emoji. + +## Workflow + +### Stage 1 — Audit (per doc) +Run `references/audit_prompt.md` against each document that doesn't +already have an up-to-date audit. Returns structured findings: +undefined terms, implicit assumptions, buried conditionals, missing +scope, missing rationale. Save output to +`.github/ai-friendly-audit/.audit.md` (e.g. for +`docs/how-to/build-kernel.md`, write +`.github/ai-friendly-audit/how-to/build-kernel.audit.md`). Do every doc +before touching any rewrite. + +### Stage 2 — Build/update the internal working glossary +Merge every audit's "undefined/ambiguous terms" findings into +`.github/ai-friendly-audit/glossary.md`: canonical term → definition → +doc(s) that currently define or use it inconsistently. This is a +working file for this pipeline's own consistency checks, not +publishable content — it is not `docs/reference/glossary.md` and +nothing here gets merged into that file. Separately, if a term should +also exist in the repo's real reader-facing glossary, flag that as a +normal content suggestion for a human to action, not something this +pipeline does itself. Add new valid technical terms to +`docs/.custom_wordlist.txt` (alphabetically sorted, per existing +convention) as usual. Skip this stage only for a very small, +already-consistent batch of docs. + +### Stage 3 — Rewrite (per doc) +Run `references/rewrite_prompt.md` on each audited doc, passing in its +audit file and the current internal working glossary. Output: explicit +conditions, stated scope (in prose, not new front matter), consistent +terminology, rationale added where missing, "Related topics" section +updated if applicable. Rewrite the doc in place, preserving its +Diátaxis category and all existing formatting conventions. + +### Stage 4 — Cross-doc consistency check +Run `references/consistency_check_prompt.md` against the *rewritten* +batch. Flags contradictions, drifted duplicate content, orphaned +cross-references, and label-prefix inconsistencies (`tutorial-*`, +`how-to-*`, `exp-*`, `ref-*`). Write to +`.github/ai-friendly-audit/consistency-report.md`. Fix flagged issues, +re-run until clean. + +### Stage 5 — Validate +Two checks, both required before a doc counts as done: +1. **Quality gate**: `make spelling`, `make linkcheck`, `make lint-md`, + `make vale`, `make html` all pass. +2. **Inference test**: maintain + `.github/ai-friendly-audit/validation-questions.md` — ~15-20 + questions requiring combining or deriving from multiple facts, not + answerable by single-sentence lookup (e.g. not "what's the default + build target?" but "if a user is cross-compiling for arm64 and also + wants to enable KASAN, which steps from the how-to guides apply and + in what order?"). Run these against the rewritten docs. + +## Worked example: unstated capability + +This is the pattern most worth watching for, since it's easy to miss — +the doc has every fact needed to answer a derived question, but never +states the implication that connects them. + +**Original doc** (`docs/how-to/testing/use-lxd-for-testing.md`, excerpt): + +> Launch a container to test your build in isolation: +> ``` +> lxc launch ubuntu:noble kernel-test +> ``` +> Copy your built kernel package into the container and install it. + +**Audit finding** (Stage 1, "Unstated capabilities or implications"): +the image tag (`ubuntu:noble`) sets the container's Ubuntu series +independently of the host machine's installed series. The doc never +says this, so a reader can't infer they could run +`lxc launch ubuntu:jammy kernel-test` to test a Jammy-series kernel +while their host runs Noble. Plausible question this blocks: *"How do +I test a Jammy kernel if I only have Noble installed?"* + +**Rewritten doc** (Stage 3): adds one explicit sentence — + +> Launch a container to test your build in isolation. The image tag +> (for example `ubuntu:noble`, `ubuntu:jammy`) sets the container's +> Ubuntu series and is independent of your host machine's installed +> series — so you can test a kernel for a different series than the +> one installed on your host: +> ``` +> lxc launch ubuntu:noble kernel-test +> ``` +> Copy your built kernel package into the container and install it. + +**Validated by** (Stage 5): the question "How do I test a Kernel from +the Jammy series? I only have Noble installed" now has a directly +inferable, correct answer from this page alone — `lxc launch +ubuntu:jammy ` — without the page ever stating that +exact question-and-answer pair verbatim. + +## Notes +- Don't run Stage 3 before Stage 2 on a multi-doc corpus — you'll lock + in inconsistent terminology instead of fixing it corpus-wide. +- If renaming or moving a doc as part of a rewrite, add the redirect in + `docs/conf.py` per the existing convention — this workflow doesn't + change that requirement. +- For a single doc, Stages 1 and 3 alone (skip glossary/consistency) + are enough. +- The three reference prompts are also duplicated as plain files in the + repo's `prompts/` folder for manual/copy-paste use outside Copilot. diff --git a/.github/skills/docs-ai-friendly-rewrite/references/audit_prompt.md b/.github/skills/docs-ai-friendly-rewrite/references/audit_prompt.md new file mode 100644 index 0000000..8653d9d --- /dev/null +++ b/.github/skills/docs-ai-friendly-rewrite/references/audit_prompt.md @@ -0,0 +1,91 @@ +# Audit Prompt — Stage 1 + +Paste one document at a time as `{{DOCUMENT}}`. Note which Diátaxis +category it's in (`tutorial`, `how-to`, `explanation`, `reference`) as +`{{CATEGORY}}` — expectations differ slightly by category (see below). + +``` +You are auditing a page from a Sphinx/MyST documentation set (Diátaxis +structure) to find everything that makes it hard for an LLM to *infer* +correct answers, as opposed to just retrieve matching sentences. You are +not rewriting anything yet — only diagnosing. + +This page's Diátaxis category is: {{CATEGORY}}. Weight your findings +accordingly: undefined terms and buried conditionals matter most in +how-to and reference content; a tutorial is allowed to be procedural +and linear without much conditional branching — don't flag that as a +defect. + +Read the document below and return findings in exactly this structure: + +## Undefined or ambiguous terms +List every term, acronym, or concept used but not defined in this doc, +plus every term defined loosely enough that two readers could interpret +it differently. For each: the term, where it appears, why it's +ambiguous. + +## Implicit context +List every place the doc assumes knowledge it doesn't state — references +to "the system," "this case," "as above," a process, or a decision +without explaining what it is. Quote the phrase and say what's missing. + +## Buried conditionals +List every rule that is actually conditional ("if X then Y, unless Z") +but is written as flat narrative prose, making the logic implicit rather +than explicit. Quote the passage and restate the logic as an explicit +if/then/unless statement. + +## Missing scope or boundaries +Does the doc say what it applies to (kernel version, architecture, +Ubuntu release, environment)? List any claim made without stated scope +where scope plausibly matters. + +## Missing rationale +List statements that say what to do without saying why. Flag only cases +where the "why" would help someone (or a model) correctly generalize to +a similar-but-not-identical situation not covered by the doc. + +## Unstated capabilities or implications +List any place the doc demonstrates or references a mechanism, +parameter, flag, or option (e.g. an image tag, a config value, a +command argument) without stating a non-obvious capability that +follows from it — something true and supported, but that a reader +would have to infer rather than find stated, and likely wouldn't think +to ask about directly. For each: the mechanism as shown in the doc, the +unstated capability it implies, and a plausible question a user might +ask that the doc could answer if this were made explicit. Only flag +real, supported implications — do not invent capabilities the doc +doesn't actually support. + +Example (do not use this in an unrelated doc, it's illustrative): a +how-to shows `lxc launch ubuntu:noble test-container` as one example +command with no further comment. It never states that the image tag +sets the container's Ubuntu series independently of the host's +installed series — so a reader can't infer from this page alone that +they could run `lxc launch ubuntu:jammy test-container` to test a +Jammy-series kernel on a Noble host. That's a real capability the doc +enables but never says, and it's exactly the kind of question ("how do +I test Jammy if I only have Noble installed?") a user is likely to ask +that the model can't answer without it being explicit. + +## Retrieval-only smells +List anything written in a way that only makes sense as an isolated +search-result snippet (dangling pronouns, headers assuming the reader +just read the previous section, examples with no stated general rule +behind them). + +## Severity ranking +Rank the above findings by how much each would actually cause an LLM to +give a *wrong* answer to a plausible user question (not just an +incomplete one). Top 5 only. + +Be concrete and cite the exact text for every finding. Do not invent +issues that aren't in the text. If a category has no findings, say +"none." + +DOCUMENT ({{CATEGORY}}): +{{DOCUMENT}} +``` + +**Repo usage:** write output to +`.github/ai-friendly-audit/.audit.md`. diff --git a/.github/skills/docs-ai-friendly-rewrite/references/consistency_check_prompt.md b/.github/skills/docs-ai-friendly-rewrite/references/consistency_check_prompt.md new file mode 100644 index 0000000..42f71a7 --- /dev/null +++ b/.github/skills/docs-ai-friendly-rewrite/references/consistency_check_prompt.md @@ -0,0 +1,57 @@ +# Cross-Doc Consistency Check Prompt — Stage 4 + +Run against a batch of rewritten, related docs together. Fill in +`{{DOCS}}` as a concatenation of the docs, each labeled with its path +and Diátaxis category. + +``` +You are checking a set of Sphinx/MyST documentation pages (Diátaxis +structure) for cross-document consistency, since inconsistency between +docs is one of the main ways an LLM gives a wrong answer even when each +individual doc looks fine. + +For the documents below, find and report: + +## Contradictions +Any place where two docs state something incompatible about the same +topic (different versions, different steps, different scope for the +same feature). Quote both passages and name both docs. + +## Terminology drift +Any place the same concept is named differently across docs, or a term +defined in the internal working glossary is used with a different +meaning somewhere else. + +## Duplicated-but-diverged content +Sections covering the same ground that have drifted apart (one updated, +the other not). Flag which looks more current if determinable, +otherwise flag the pair. + +## Broken or vague cross-references +Any {ref}/{doc} or :ref:/:doc: reference that doesn't point to a valid +section-prefixed label, or a "Related topics" entry that no longer +matches the referenced doc's current content. + +## Label-prefix inconsistencies +Any cross-reference label that doesn't follow this repo's +section-prefix convention (tutorial-*, how-to-*, exp-*, ref-*) for the +Diátaxis category it's in. + +## Coverage gaps +Questions a user could reasonably ask that fall in the gap between two +docs — where each assumes the other covers it, and neither actually +does. + +For each finding, state which doc(s) should be corrected and a one-line +suggested fix. Do not rewrite the docs here — just report. + +DOCUMENTS: +{{DOCS}} +``` + +**Repo usage:** write output to +`.github/ai-friendly-audit/consistency-report.md`. Feed each "should be +corrected" item back through `rewrite_prompt.md` for the relevant doc, +then re-run this check on the updated batch until clean. If a fix +involves renaming/moving a page, add the redirect in `docs/conf.py` per +the existing convention. diff --git a/.github/skills/docs-ai-friendly-rewrite/references/rewrite_prompt.md b/.github/skills/docs-ai-friendly-rewrite/references/rewrite_prompt.md new file mode 100644 index 0000000..d6f1003 --- /dev/null +++ b/.github/skills/docs-ai-friendly-rewrite/references/rewrite_prompt.md @@ -0,0 +1,83 @@ +# Rewrite Prompt — Stage 3 + +Run per document, after Stage 1 (audit) and Stage 2 (glossary update) +are done. Fill in `{{DOCUMENT}}`, `{{CATEGORY}}`, `{{AUDIT_FINDINGS}}`, +and `{{GLOSSARY}}`. + +``` +You are rewriting a page from a Sphinx/MyST (or RST) documentation set +so an LLM can reason over it and correctly infer answers to questions +it doesn't state verbatim — while keeping it just as readable for +humans and fully compliant with this repo's existing conventions. + +This page's Diátaxis category is: {{CATEGORY}}. Preserve that category +— a how-to must stay task-oriented, reference stays specification-only, +explanation stays conceptual, tutorial stays procedural. Do not blur +categories while clarifying. + +Apply these rules: + +1. Replace every ambiguous or undefined term (see audit findings and + internal working glossary below) with the canonical term, and give a + one-line inline definition on first use in this doc. This working + glossary is separate from docs/reference/glossary.md (a different, + reader-facing file this workflow does not touch) — do not add or + suggest entries there. +2. Rewrite buried conditionals as explicit if/then/unless statements or + a small table, wherever the audit flagged one — mainly relevant for + how-to and reference pages. Don't convert every sentence into a + table. +3. State the scope of every claim that needs one (kernel version, + architecture, Ubuntu release, environment) inline in prose near the + relevant content. Do not add new front-matter fields for this. +4. Add one line of rationale ("why") next to instructions where the + audit flagged missing rationale — only where it helps someone + correctly handle a similar case not explicitly covered. +5. Where the audit flagged an unstated capability or implication, add + an explicit sentence spelling it out — phrased so it also answers + the plausible derived question the audit identified. Only state + capabilities the doc's own mechanism actually supports; do not + extrapolate beyond what's verifiably true. +6. Make the doc self-contained: resolve "as mentioned above" / "in this + case" into the actual referent, or a proper {ref}/{doc} cross- + reference (MyST) or :ref:/:doc: (RST) per this repo's convention, + with enough surrounding context that the sentence still makes sense + on its own. +7. Do NOT touch the existing meta description front matter beyond + verifying it's still accurate to the rewritten content — update the + description text only if the rewrite changed what the page is about. +8. If new cross-references were added, use section-prefixed labels + (tutorial-*, how-to-*, exp-*, ref-*) per the existing convention. If + this page should link to related pages that weren't linked before, + add or update its "Related topics" section rather than inventing a + new field. +9. Keep every fact from the original. Do not invent new facts, numbers, + or behavior. Do not remove information — only clarify, restructure, + and make implicit things explicit. +10. Keep existing headings, structure, and all formatting rules from + this repo's copilot-instructions.md (sentence case headings, active + voice, blank line after headings/before lists, no emoji, no skipped + heading levels). + +At the end, output two things separately: +- The rewritten document. +- A "Changes made" list, and a "New/updated terms" list (term + + definition) for this pipeline's internal working glossary only. + +AUDIT FINDINGS FOR THIS DOC: +{{AUDIT_FINDINGS}} + +CURRENT INTERNAL WORKING GLOSSARY: +{{GLOSSARY}} + +DOCUMENT TO REWRITE ({{CATEGORY}}): +{{DOCUMENT}} +``` + +**Repo usage:** `{{AUDIT_FINDINGS}}` = contents of the matching file in +`.github/ai-friendly-audit/`; `{{GLOSSARY}}` = contents of +`.github/ai-friendly-audit/glossary.md`. Rewrite the doc in place. +Merge the "New/updated terms" list into +`.github/ai-friendly-audit/glossary.md` (Stage 2), and run `make +spelling`, `make linkcheck`, `make lint-md`, `make vale`, `make html` +before considering the doc done. diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..9a097bb --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ + +# AI output +.github/ai-friendly-audit/ \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..5fc774e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,83 @@ +# AGENTS.md + +`AGENTS.md` is a shared convention read by most coding agents (GitHub +Copilot coding agent, Codex, Cursor, Claude Code, Amp, and others) — no +tool-specific setup needed for this file itself. See +`.github/copilot-instructions.md` for this repo's full documentation +conventions (Diátaxis, Sphinx/MyST formatting, cross-references, PR +requirements) — this file only covers the AI-inference-friendliness +pipeline on top of those. + +## Docs AI-friendliness pipeline + +Full stage instructions live in +`.github/skills/docs-ai-friendly-rewrite/SKILL.md` and its +`references/` prompts (also duplicated as plain files in `prompts/`). + +### Persistent state + +- `.github/ai-friendly-audit/glossary.md` — internal working glossary + for this pipeline's own term-consistency checks. Separate from, and + never merged into, `docs/reference/glossary.md`, which is the + reader-facing glossary and serves a different purpose. +- `docs/.custom_wordlist.txt` — existing spelling-exceptions file; new + valid technical terms go here too. +- `.github/ai-friendly-audit/.audit.md` — audit findings per + source doc, mirroring its path under `docs/`. Kept outside `docs/` so + the Sphinx build never picks it up. +- `.github/ai-friendly-audit/consistency-report.md` — latest cross-doc + consistency check. +- `.github/ai-friendly-audit/validation-questions.md` — held-out + inference-question test set (Stage 5). Append over time; don't delete + existing questions without flagging it in the PR. + +### Pipeline + +1. **Audit** every doc that doesn't have an up-to-date `.audit.md` (or + that changed since its last audit). Criteria: + `.github/skills/docs-ai-friendly-rewrite/references/audit_prompt.md`. +2. **Update the internal working glossary**: merge audit findings' + undefined-term lists into `.github/ai-friendly-audit/glossary.md` + (this pipeline's own file — never `docs/reference/glossary.md`); add + recurring valid technical terms to `docs/.custom_wordlist.txt`. +3. **Rewrite** each audited doc in place. Criteria: + `.github/skills/docs-ai-friendly-rewrite/references/rewrite_prompt.md`. + Preserve Diátaxis category, all existing formatting rules, and every + fact from the original. +4. **Consistency check** the rewritten batch. Criteria: + `references/consistency_check_prompt.md`. Fix flagged issues, re-run + until clean. +5. **Validate**: (a) `make spelling`, `make linkcheck`, `make lint-md`, + `make vale`, `make html` all pass; (b) rewritten docs correctly + answer the held-out inference questions in + `validation-questions.md`. + +### PR conventions (same as this repo's existing conventions) + +- Link PRs to issues with `Fixes #`. +- Use Conventional Comments for review feedback. +- One PR per batch of related docs, not the whole repo at once. +- PR description includes: docs audited/rewritten, any new glossary + terms added, and consistency-check results. +- If a rewrite renames/moves a page, add the redirect in `docs/conf.py` + per the existing convention. +- Preview build via the Read the Docs check on the PR, same as any + other doc change. + +## Using this with a specific tool + +The pipeline works by having any agent read +`.github/skills/docs-ai-friendly-rewrite/SKILL.md` and act on it, or by +pasting files from `prompts/` into a chat by hand. + +- **GitHub Copilot (Visual Studio / VS Code / JetBrains)**: the skill + under `.github/skills/` auto-discovers; `.github/copilot-instructions.md` + provides always-on baseline context. +- **Claude / Claude Code**: copy + `.github/skills/docs-ai-friendly-rewrite/` into `.claude/skills/` + (project) or `~/.claude/skills/` (personal) — same SKILL.md, no + changes needed. +- **Other agentic CLIs**: point the tool at this `AGENTS.md`, or pass a + file from `prompts/` directly as a prompt. +- **No agent tooling**: open a file from `prompts/`, paste into any + chat LLM, fill in the placeholders by hand. diff --git a/README.md b/README.md index 5095167..08b5f18 100644 --- a/README.md +++ b/README.md @@ -93,7 +93,9 @@ gh copilot suggest "Follow .github/copilot-instructions.md and suggest improveme ## Use Copilot Skills -For comprehensive documentation reviews, use the documentation-review skill: +### Documentation-review skill + +For comprehensive documentation reviews: - Skill file: [.github/skills/documentation-review/SKILL.md](.github/skills/documentation-review/SKILL.md) @@ -103,3 +105,39 @@ For comprehensive documentation reviews, use the documentation-review skill: Prompt example: > Follow .github/copilot-instructions.md. Run the documentation-review skill from SKILL.md on docs/how-to/contribute.md only. + +### AI-friendly-rewrite skill + +For making documentation AI-inference-friendly: + +- Skill file: + [.github/skills/docs-ai-friendly-rewrite/SKILL.md](.github/skills/docs-ai-friendly-rewrite/SKILL.md) +- Coverage: 5-stage pipeline (audit, glossary update, rewrite, consistency check, validate) to improve how LLMs can reason about and infer from documentation. + +Prompt example: + +> Follow .github/copilot-instructions.md and AGENTS.md. Run the docs-ai-friendly-rewrite skill from .github/skills/docs-ai-friendly-rewrite/SKILL.md on [docs to audit/rewrite]. + +## AI-friendly documentation pipeline + +This repository includes an advanced pipeline for making documentation AI-inference-friendly. +See [AGENTS.md](AGENTS.md) for detailed workflow instructions. + +### Pipeline overview + +The 5-stage pipeline helps documentation become more reasoning-friendly for LLMs: + +1. **Audit** - Analyze docs for inference-blocking patterns +2. **Update glossary** - Merge findings into internal working glossary +3. **Rewrite** - Improve documentation structure and clarity +4. **Consistency check** - Ensure cross-doc consistency +5. **Validate** - Run quality checks and inference-question tests + +### Key files and directories + +- **[AGENTS.md](AGENTS.md)** - Shared convention read by coding agents; documents the complete pipeline, persistent state, and PR conventions +- **[.github/ai-friendly-audit/](.github/ai-friendly-audit/)** - Working directory containing audit findings, internal glossary, consistency reports, and validation questions +- **[prompts/](prompts/)** - Standalone prompt files for each pipeline stage, usable in any chat LLM: + - `audit-docs.md` - Audit criteria and procedure + - `rewrite-docs.md` - Rewrite guidelines + - `consistency-check-docs.md` - Cross-doc consistency checks diff --git a/docs/how-to/source-code/obtain-kernel-source-git.md b/docs/how-to/source-code/obtain-kernel-source-git.md index e762f9d..15fa2a4 100644 --- a/docs/how-to/source-code/obtain-kernel-source-git.md +++ b/docs/how-to/source-code/obtain-kernel-source-git.md @@ -23,8 +23,10 @@ sudo apt-get install git ## Get local copy of kernel source for single release -You can use `git clone` with the selected protocol to obtain a local copy of -the kernel source for the release you are interested in. +Each Ubuntu release (e.g. "Noble", "Resolute") has its own codename-based kernel +series and corresponding Git repository. +Use `git clone` with the selected protocol to obtain a local copy of the +kernel source for the Ubuntu kernel series you are interested in. For example, to obtain a local copy of the Jammy kernel tree, run any of the following `git clone` commands: @@ -39,10 +41,11 @@ See {ref}`exp-ubuntu-kernel-source-protocols` for more information. ## Clone multiple releases using a shared reference repository -Cloning a single kernel tree downloads several hundred megabytes of data. If -you plan to work with more than one kernel release, you can save space and time -by first downloading the upstream kernel tree and using it as a reference for -subsequent clones: +Cloning a single kernel tree downloads several hundred megabytes of data. +If you plan to work with more than one Ubuntu kernel series, clone the upstream +Linux kernel repository first and use it as a reference repository for the +subsequent Ubuntu clones. +This saves time and disk usage by reducing duplicate downloads. ```{code-block} shell git clone https://kernel.ubuntu.com/ubuntu/linux.git @@ -50,19 +53,22 @@ git clone --reference linux https://git.launchpad.net/~ubuntu-kernel/ubuntu/+sou git clone --reference linux https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/noble ``` -Each `git clone` creates a new directory for a given release, containing the full source and history of the repository. +Each Ubuntu `git clone` creates a separate directory for a given series and has +the full source and history available locally. ```{caution} Once two trees are linked this way, you cannot delete or move the upstream -`linux` reference tree without manually updating -`.git/objects/info/alternates` in each Ubuntu kernel tree that references it. +`linux` reference tree without manually updating `.git/objects/info/alternates` +in each Ubuntu kernel tree that references it. ``` ## Add multiple series as remotes -If you are an advanced Git user, you can add each Ubuntu series as a remote to -have all kernel series in a single Git repository, and switch between them -using branches: +If you want to compare or switch between several Ubuntu kernel series in one +working repository, add each series as a remote and switch between them using +branches. +This keeps the series in one repository, but makes its remote and branch +structure more complex than using separate clones: ```{code-block} shell git remote add jammy https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy @@ -71,10 +77,18 @@ git checkout -b jammy --track jammy/master git checkout -b jammy-next --track jammy/master-next ``` +For example, if you want to check out the current Ubuntu kernel source for +Jammy, use the `jammy` branch, which tracks `master`. +If you instead want to check out the commits staged for the next Stable Release +Update (SRU), you can create the `jammy-next` branch, which tracks +`master-next`. + ## Work with multiple series in separate subdirectories -To have the source for each kernel series available in its own subdirectory -within a single Git repository, use `git subtree add`: +If you need the source for each Ubuntu kernel series in its own subdirectory +within one repository, use `git subtree add`. This layout is useful when the +series should be visible together in one working tree, but it puts the large +kernel histories into that repository and requires explicit update commands. ```{code-block} shell git subtree add --prefix=jammy https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy master @@ -94,9 +108,13 @@ git subtree pull --prefix=jammy https://git.launchpad.net/~ubuntu-kernel/ubuntu/ ## Work with a specific kernel version using tags -By default, cloning gives you the latest state of the `master` branch. To work -with a specific, previously released kernel version, use release tags. To list -all available tags for a release: +By default, cloning checks out the repository's default branch, which is +`master` for the Ubuntu kernel repositories described here. To work with a +specific, previously released kernel version, use an Ubuntu release tag. An +Ubuntu release tag starts with `Ubuntu-` and identifies a released kernel +version. + +To list matching tags in the cloned repository: ```{code-block} shell git tag -l Ubuntu-* @@ -113,18 +131,19 @@ Ubuntu-5.4.0-52.57 ... ``` -To check out a specific version, create a branch pointing to that tag: +To check out a specific version while keeping a branch for additional local +commits, create a branch pointing to that tag: ```{code-block} shell git checkout -b temp Ubuntu-5.4.0-52.57 ``` -You can then work with that version - for example, by adding new commits. +The new branch points to the tagged snapshot, so you can add commits without +leaving the repository in a detached `HEAD` state. ## Related topics - {doc}`/explanation/ubuntu-linux-kernel-sources` -- % LINKS diff --git a/prompts/audit-docs.md b/prompts/audit-docs.md new file mode 100644 index 0000000..740a191 --- /dev/null +++ b/prompts/audit-docs.md @@ -0,0 +1,60 @@ +# Audit Prompt — Stage 1 + +Plain, tool-agnostic prompt template. Paste one document at a time as +`{{DOCUMENT}}` into any chat LLM, or point an agentic CLI at this file +with the target doc as input. + +``` +You are auditing a piece of documentation to find everything that makes it +hard for an LLM to *infer* correct answers, as opposed to just retrieve +matching sentences. You are not rewriting anything yet — only diagnosing. + +Read the document below and return findings in exactly this structure: + +## Undefined or ambiguous terms +List every term, acronym, or concept used but not defined in this doc, +and every term that's defined loosely enough that two readers could +interpret it differently. For each: the term, where it appears, why it's +ambiguous. + +## Implicit context +List every place the doc assumes knowledge it doesn't state — references +to "the system," "this case," "as above," a process, or a decision without +explaining what it is. Quote the phrase and say what's missing. + +## Buried conditionals +List every rule that is actually conditional ("if X then Y, unless Z") +but is written as flat narrative prose, making the logic implicit rather +than explicit. Quote the passage and restate the logic as an explicit +if/then/unless statement. + +## Missing scope or boundaries +Does the doc say what it applies to (version, plan tier, environment, +region, user role, date range)? List any claim made without a stated +scope where scope plausibly matters. + +## Missing rationale +List statements that say what to do without saying why. Flag only cases +where the "why" would actually help someone (or a model) correctly +generalize to a similar-but-not-identical situation not covered by the +doc. + +## Retrieval-only smells +List anything written in a way that only makes sense as an isolated +search-result snippet (e.g. dangling pronouns, headers that assume the +reader just read the previous section, examples with no stated general +rule behind them). + +## Severity ranking +Rank the above findings by how much each one would actually cause an LLM +to give a *wrong* answer to a plausible user question (not just an +incomplete one). Top 5 only. + +Be concrete and cite the exact text for every finding. Do not invent +issues that aren't in the text. If a category has no findings, say "none." + +DOCUMENT: +{{DOCUMENT}} +``` + +**Repo usage:** write the output to `docs/_ai-friendly-audit/.audit.md`. diff --git a/prompts/consistency-check-docs.md b/prompts/consistency-check-docs.md new file mode 100644 index 0000000..8cf5170 --- /dev/null +++ b/prompts/consistency-check-docs.md @@ -0,0 +1,51 @@ +# Cross-Doc Consistency Check Prompt — Stage 4 + +Plain, tool-agnostic prompt template. Run against a batch of rewritten, +related docs together (not one at a time — this stage needs cross-doc +visibility). Fill in `{{DOCS}}` as a concatenation of the docs, each +clearly labeled with its title/path. + +``` +You are checking a set of documentation pages for cross-document +consistency, since inconsistency between docs is one of the main ways +an LLM gives a wrong answer even when each individual doc looks fine. + +For the documents below, find and report: + +## Contradictions +Any place where two docs state something incompatible about the same +topic (different numbers, different rules, different scope for the +same feature). Quote both passages and name both docs. + +## Terminology drift +Any place where the same concept is named differently across docs, or +where a term defined in one doc is used with a different meaning in +another. + +## Duplicated-but-diverged content +Sections that clearly cover the same ground but have drifted apart +(one was updated, the other wasn't). Flag which one looks more current +if there's a dated field, otherwise just flag the pair. + +## Broken or vague cross-references +Any "see [doc]" or similar reference that doesn't point to something +findable, or that used to make sense but no longer matches the +referenced doc's current content. + +## Coverage gaps +Questions a user could reasonably ask that fall in the gap between two +docs — where each doc assumes the other one covers it, and neither +actually does. + +For each finding, state which doc(s) should be corrected and a one-line +suggested fix. Do not rewrite the docs here — just report. + +DOCUMENTS: +{{DOCS}} +``` + +**Repo usage:** write output to +`docs/_ai-friendly-audit/consistency-report.md`. Feed each "should be +corrected" item back through `rewrite_prompt.md` for the relevant doc +(add the fix as an extra audit-findings bullet), then re-run this check +on the updated batch until clean. diff --git a/prompts/rewrite-docs.md b/prompts/rewrite-docs.md new file mode 100644 index 0000000..07b5f42 --- /dev/null +++ b/prompts/rewrite-docs.md @@ -0,0 +1,61 @@ +# Rewrite Prompt — Stage 3 + +Plain, tool-agnostic prompt template. Run per document, after Stage 1 +(audit) and Stage 2 (glossary) exist. Fill in `{{DOCUMENT}}`, +`{{AUDIT_FINDINGS}}`, and `{{GLOSSARY}}`. + +``` +You are rewriting a documentation page so an LLM can reason over it and +correctly infer answers to questions it doesn't state verbatim — while +keeping it just as readable for humans. + +Apply these rules: + +1. Replace every ambiguous or undefined term (see audit findings and + glossary below) with the canonical term and, on first use in this + doc, a one-line inline definition. +2. Rewrite buried conditionals as explicit if/then/unless statements + or a small decision table, wherever the audit flagged one. Don't + convert every sentence into a table — only genuinely conditional + logic. +3. State the scope of every claim that needs one: what version, plan, + environment, role, or condition it applies to. If scope is unknown, + say so explicitly rather than leaving it ambiguous — "applies to all + plans unless noted" is fine; silence is not. +4. Add one line of rationale ("why") next to instructions where the + audit flagged missing rationale — only where it would help someone + correctly handle a similar case not explicitly covered. +5. Make the doc self-contained: resolve "as mentioned above" / "in this + case" into the actual referent, or an explicit link with enough + context that the sentence still makes sense in isolation. +6. Add a short front-matter block at the top: + - title + - applies_to (version/plan/scope) + - last_reviewed + - related_docs (canonical titles/links, not just "see also") +7. Keep every fact from the original. Do not invent new facts, numbers, + or behavior. Do not remove information — only clarify, restructure, + and make implicit things explicit. +8. Keep existing headers/chunk boundaries where they already work for + search — this is about adding inference-readiness, not redoing + information architecture. +9. Use the canonical terminology from the glossary consistently + throughout, even if the original doc used different words. + +At the end, output a short "Changes made" list summarizing what you +clarified, so a human reviewer can spot-check. + +AUDIT FINDINGS FOR THIS DOC: +{{AUDIT_FINDINGS}} + +CANONICAL GLOSSARY (corpus-wide): +{{GLOSSARY}} + +DOCUMENT TO REWRITE: +{{DOCUMENT}} +``` + +**Repo usage:** `{{AUDIT_FINDINGS}}` = contents of +`docs/_ai-friendly-audit/.audit.md`; `{{GLOSSARY}}` = +contents of `docs/_glossary.md`. Rewrite the doc in place; note any new +or renamed glossary terms in the PR description.