Aligns directive's block_text content to rST's#1048
Conversation
|
Thanks for this @kdeldycke, and apologies it sat unreviewed for so long. #1164 now supersedes this: rather than passing the unparsed content only, it reconstructs the full directive text from the fence token — including the opening I'll close this once #1164 is merged. Generated by Claude Code |
Fixes three classes of wrong source-line attribution, plus two
rST-parity alignments.
## Directive option warnings point at the offending option
Previously all option warnings pointed at the directive's opening line
(`:opt:` style) or the opening `---` (YAML style), and unknown keys were
aggregated into one warning:
````markdown
```{note}
---
class: tip
name: mynote
align: whatever <- line 7
---
```
````
reported `<string>:4: 'note': Unknown option keys: ['align'] ...`. Now
each warning (unknown key, invalid value, bad YAML, `#` comments)
carries the offending line — `<string>:7: 'note': Unknown option key:
'align' ...` — for both option styles, resolving the long-standing `TODO
advance line number`:
- `parsers/options.py` gains a public `options_to_tokens` (per-key
`Position`s; `options_to_items` is now a thin wrapper), and its `State`
records comment lines.
- `_parse_directive_options` computes an `options_position` per block
style and per-key absolute lines; the `---` block split is now
**line-based**: text trailing a closing delimiter still becomes body
content (as previously), but is kept as its own line, so it — and every
body line after it — maps exactly (the old character-based split shifted
all subsequent body lines by one).
- Directives synthesized from non-source content (HTML
images/admonitions) attribute option warnings to the element's own line.
## First-line body content was off by one
For no-argument directives with content on the opening line (``
```{note} text ``), every body warning reported one line too low.
`body_offset` is now `-1` for the merged first line (documented in
`DirectiveParsingResult`; all internal consumers verified, and myst-nb's
`code-cell` cannot hit this branch since it takes an argument).
## Stale `document.current_line` (closes #546)
`run_directive` set `document.current_line` and never reset it, so
docutils' `Node.setup_child` stamped the most recent directive's line
(or 0) onto every subsequently-created line-less node — every `Text`
node in the document. This is exactly the "errors attributed to the most
recent directive" / "line 0" behaviour reported for
sphinxcontrib-spelling, and visible in gettext output (table cells
extracted at `index.md:0`). The renderer now keeps `current_line` in
sync as nodes are created; the regenerated `.pot` fixtures show cells at
their real lines.
## rST-parity alignments
- **`block_text`** (supersedes #1048, with thanks to @kdeldycke):
directives now receive the full directive text — opening fence, options
block, unparsed body, closing fence — matching rST, including the
header/closing lines that #1048 noted it could not reconstruct.
Third-party directives that compute absolute lines from `block_text`
(e.g. click-extra) now get rST-consistent input. Note: erroring
directives that embed `self.block_text` in their error output
(`list-table`, `figure`, etc.) now show the full fence rather than only
the body.
- **`include` with `:literal:`**: the literal block's line is now
`start-line + 1` (exact docutils behaviour), not a hardcoded 1.
## Behaviour changes to be aware of
- Unknown-key warnings changed shape: one per key (`Unknown option key:
'x' (allowed: [...])`) instead of one aggregate — the
`myst.directive_option` subtype is unchanged, so suppression configs
keep working.
- Old-vs-new probes were run against master for: the full docutils
rendering suite, a sphinx + sphinx-design project (HTML and
`searchindex.js` byte-identical), myst-nb notebook reading
(byte-identical, including malformed metadata blocks), gettext,
includes, and a ~30k-input fuzz of the option-parsing rewrite — no
rendering differences beyond the intended warning lines/messages and the
two rST-parity items above.
## Known limitations (documented, out of scope)
- Warnings for inline syntax report the enclosing paragraph's first line
(markdown-it inline tokens carry no per-line maps).
- Warnings inside multi-line substitution output can map past the
substitution's location.
- Line attribution inside `{include}`d markdown is off by one
(pre-existing, both before and after this PR) — to be tracked
separately.
All changed fixture numbers were hand-verified against their source
snippets; new conformance cases in `reporter_warnings.md` lock in the
corrected lines (markdown in → warning+line out, reusable as a
language-agnostic corpus).
---
_Generated by [Claude
Code](https://claude.ai/code/session_01GByoptRecR2GAwj5qrLz6C)_
|
#1164 is merged, so closing this as superseded — Generated by Claude Code |
In rST, a directive's
block_textcontains the entire directive, including options and full, unparsed content.This aligns MyST parser to rST.
Context
I stumbled upon this issue while implementing my own
click:exampleandclick:rundirectives within my Click Extra project: https://kdeldycke.github.io/click-extra/sphinx.htmlMy implementation is trying to report, within these directives, of the absolute line in the document where a problem occured. The line number and offset calculation works perfectly in rST, but requires some hacks to report the right one in MyST.
But even with these hacks, there is no way to retro-actively compute the absolute line number, because the directive object do not carry its unparsed content.
Example
Compare a given rST directive defined as:
Its properties are:
And for its MyST equivalent:
Limitations
Note that it is not perfect, as it is supposed to also contain:
```{name} {arguments}first line), and```line).But I doubt we can find a way to do it unless we introduce heavier refactor.