diff --git a/README.md b/README.md index 2309ea5..e6791b5 100644 --- a/README.md +++ b/README.md @@ -255,7 +255,7 @@ Paragraph with **bold**, *italic*, `code`, [links](https://example.com), and foo ) ``` -Supported Markdown includes headings, paragraphs, bold, italic, inline code, strikethrough, links, images, linked images, image captions, code blocks, blockquotes, ordered lists, unordered lists, horizontal rules, and footnotes. +Supported Markdown includes headings, paragraphs, bold, italic, inline code, strikethrough, superscript, subscript, links, images, linked images, image captions, code blocks, blockquotes, ordered lists, unordered lists, horizontal rules, footnotes, LaTeX math, pull quotes, and callouts. See [docs/markdown.md](docs/markdown.md) for the full reference with examples. When an `Api` instance is passed to `from_markdown`, local image paths are uploaded before the draft is created: diff --git a/docs/markdown.md b/docs/markdown.md new file mode 100644 index 0000000..0adc784 --- /dev/null +++ b/docs/markdown.md @@ -0,0 +1,160 @@ +# Markdown support + +`Post.from_markdown(markdown_content, api=None)` converts a Markdown document +into Substack's document format. Parsing is handled by +[markdown-it-py](https://github.com/executablebooks/markdown-it-py) (CommonMark) +plus a few plugins, so standard CommonMark works as you'd expect. This page +documents everything that maps to a Substack node. + +```python +from substack.post import Post + +post = Post(title="My Post", subtitle="", user_id=api.get_user_id()) +post.from_markdown(open("post.md").read(), api=api) +draft = api.post_draft(post.get_draft()) +``` + +Pass `api=` when your Markdown references local images so they can be uploaded +(see [Images](#images)); it is optional otherwise. + +## Text formatting + +| Markdown | Result | +|-------------------------------|-------------------| +| `**bold**` | **bold** | +| `*italic*` | *italic* | +| `***bold italic***` | ***bold italic*** | +| `` `inline code` `` | `inline code` | +| `~~strikethrough~~` | ~~strikethrough~~ | +| `^superscript^` | superscript | +| `~subscript~` | subscript | +| `[text](https://example.com)` | link | +| `` | autolinked URL | + +Note that subscript uses a single tilde (`~x~`) and strikethrough uses a double +tilde (`~~x~~`); both work in the same document. + +## Headings + +Levels 1–6, using `#` through `######`. Headings may contain inline formatting +and links. + +```markdown +# Heading level 1 +## Heading level 2 with **bold** and a [link](https://example.com) +``` + +## Paragraphs and line breaks + +Blank lines separate paragraphs. A single newline within a paragraph is treated +as a space (soft break), matching CommonMark. + +## Lists + +Bullet lists (`-`, `*`, or `+`) and ordered lists (`1.`), including nesting: + +```markdown +- Bullet one +- Bullet two + - Nested bullet + 1. Nested number + +1. Ordered one +2. Ordered two +``` + +## Blockquotes + +```markdown +> A blockquote. +> +> With multiple paragraphs. +``` + +## Code blocks + +Fenced code blocks, with an optional language for syntax highlighting, and +indented code blocks: + +````markdown +```python +print("hello") +``` +```` + +## Horizontal rule + +```markdown +--- +``` + +## Images + +A paragraph containing only an image becomes a captioned image. + +```markdown +![Alt text](https://example.com/image.png) +``` + +- **Caption:** use the CommonMark title slot — `![alt](url "caption text")`. +- **Link:** wrap the image in a link — `[![alt](url)](https://target.com)`. +- **Local upload:** if `api=` is passed and the `src` is a local path (not an + `http(s)` URL), the file is uploaded to Substack and the returned URL is used. + Local paths resolve relative to the current working directory. + +```markdown +![A chart](chart.png "Figure 1: quarterly results") +``` + +## Footnotes + +References become inline anchors; definitions become footnote blocks at the end, +numbered by order of first appearance. Labels may be numeric or named, and a +definition may contain block content such as lists or multiple paragraphs. + +```markdown +A claim that needs support.[^1] Another, with a named label.[^source] + +[^1]: The supporting detail, with a [link](https://example.com). +[^source]: Author, *Title* (2025). +``` + +A reference used more than once is emitted as a separate numbered anchor each +time, mirroring the Substack editor. Definitions that are never referenced are +dropped. + +## Math (LaTeX) + +Inline math with single dollars, block math with double dollars: + +```markdown +Einstein showed $E=mc^2$ inline. + +$$ +\int_0^\infty e^{-x} \, dx = 1 +$$ +``` + +## Pull quotes and callouts + +These use fenced-container syntax (`:::`), since they have no native Markdown +equivalent: + +```markdown +:::pullquote +A highlighted pull quote. **Formatting** works inside. +::: + +:::callout +A callout block, e.g. an aside or note. +::: +``` + +## Not supported + +- **Tables** — Substack has no table renderer or editor UI for them, so GFM + table syntax is not converted. To include tabular data, embed a chart (for + example via [Datawrapper](https://support.substack.com/hc/en-us/articles/15722290158100)) + and add it in the Substack editor. +- Widgets authored only in the Substack editor (buttons, polls, embeds, etc.) + have no Markdown equivalent. diff --git a/poetry.lock b/poetry.lock index 79a12f5..587f517 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,4 +1,4 @@ -# This file is automatically @generated by Poetry 2.3.2 and should not be changed by hand. +# This file is automatically @generated by Poetry 2.4.1 and should not be changed by hand. [[package]] name = "aiofile" @@ -1040,23 +1040,23 @@ ws = ["websockets (>=15.0.1)"] [[package]] name = "mdit-py-plugins" -version = "0.4.2" +version = "0.6.1" description = "Collection of plugins for markdown-it-py" optional = false -python-versions = ">=3.8" +python-versions = ">=3.10" groups = ["main"] files = [ - {file = "mdit_py_plugins-0.4.2-py3-none-any.whl", hash = "sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636"}, - {file = "mdit_py_plugins-0.4.2.tar.gz", hash = "sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5"}, + {file = "mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d"}, + {file = "mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0"}, ] [package.dependencies] -markdown-it-py = ">=1.0.0,<4.0.0" +markdown-it-py = ">=2.0.0,<5.0.0" [package.extras] code-style = ["pre-commit"] rtd = ["myst-parser", "sphinx-book-theme"] -testing = ["coverage", "pytest", "pytest-cov", "pytest-regressions"] +testing = ["coverage", "pytest", "pytest-cov", "pytest-regressions", "pytest-timeout"] [[package]] name = "mdurl" @@ -2229,4 +2229,4 @@ mcp = ["fastmcp"] [metadata] lock-version = "2.1" python-versions = "<4.0,>=3.10" -content-hash = "012ebc118846bd27cfffa8519e480a238d65ed7a38f77a92d13e721c05552b02" +content-hash = "11fb43d2644aa0a1c9a300d0aa7a642aea5f2017642cd9cabe344d6497c67f5f" diff --git a/pyproject.toml b/pyproject.toml index 86f1fad..18ce7df 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -39,7 +39,7 @@ requests = "^2.32.0" python-dotenv = "^1.2.1" PyYAML = "^6.0" markdown-it-py = "^3.0" -mdit-py-plugins = "^0.4" +mdit-py-plugins = ">=0.5,<0.7" fastmcp = { version = "^3.1.1", optional = true } [tool.poetry.extras] diff --git a/substack/mdrender.py b/substack/mdrender.py index e04b97e..2f091b5 100644 --- a/substack/mdrender.py +++ b/substack/mdrender.py @@ -22,7 +22,11 @@ from markdown_it import MarkdownIt from markdown_it.tree import SyntaxTreeNode +from mdit_py_plugins.container import container_plugin +from mdit_py_plugins.dollarmath import dollarmath_plugin from mdit_py_plugins.footnote import footnote_plugin +from mdit_py_plugins.subscript import sub_plugin +from mdit_py_plugins.superscript import superscript_plugin from substack import nodes from substack.nodes import MarkType, NodeType @@ -31,11 +35,22 @@ "strong": {"type": MarkType.STRONG}, "em": {"type": MarkType.EM}, "s": {"type": MarkType.STRIKETHROUGH}, + "sup": {"type": MarkType.SUPERSCRIPT}, + "sub": {"type": MarkType.SUBSCRIPT}, } def _make_parser() -> MarkdownIt: - return MarkdownIt("commonmark").use(footnote_plugin).enable("strikethrough") + return ( + MarkdownIt("commonmark") + .use(footnote_plugin) + .use(dollarmath_plugin) + .use(sub_plugin) + .use(superscript_plugin) + .use(container_plugin, name="pullquote") + .use(container_plugin, name="callout") + .enable("strikethrough") + ) def _coalesce(out_nodes: List[Dict]) -> List[Dict]: @@ -64,6 +79,8 @@ def _render_inline(node: SyntaxTreeNode, marks: List[Dict], ctx: Dict) -> List[D out.append(nodes.text(child.content, marks)) elif t == "code_inline": out.append(nodes.text(child.content, marks + [nodes.code_mark()])) + elif t == "math_inline": + out.append(nodes.latex_inline(child.content.strip())) elif t in _MARK_FOR: out.extend(_render_inline(child, marks + [_MARK_FOR[t]], ctx)) elif t == "link": @@ -159,10 +176,26 @@ def _render_block(node: SyntaxTreeNode, api, ctx: Dict) -> List[Dict]: if t == "ordered_list": return [nodes.ordered_list(_render_list_items(node, api, ctx))] + if t == "math_block": + return [nodes.latex_block(node.content.strip())] + + if t == "container_pullquote": + return [nodes.pullquote(_render_container_body(node, api, ctx))] + + if t == "container_callout": + return [nodes.callout_block(_render_container_body(node, api, ctx))] + # footnote_block is handled separately in markdown_to_doc; ignore it here. return [] +def _render_container_body(node: SyntaxTreeNode, api, ctx: Dict) -> List[Dict]: + body: List[Dict] = [] + for child in node.children: + body.extend(_render_block(child, api, ctx)) + return body + + def _render_list_items(list_node: SyntaxTreeNode, api, ctx: Dict) -> List[Dict]: items = [] for li in list_node.children: diff --git a/substack/nodes.py b/substack/nodes.py index c5571ba..1d361e3 100644 --- a/substack/nodes.py +++ b/substack/nodes.py @@ -33,6 +33,10 @@ class NodeType: FOOTNOTE_ANCHOR = "footnoteAnchor" CAPTIONED_IMAGE = "captionedImage" CAPTION = "caption" + LATEX_BLOCK = "latex_block" + LATEX_INLINE = "latex" + PULLQUOTE = "pullquote" + CALLOUT_BLOCK = "calloutBlock" class MarkType: @@ -40,6 +44,8 @@ class MarkType: EM = "em" CODE = "code" STRIKETHROUGH = "strikethrough" + SUPERSCRIPT = "superscript" + SUBSCRIPT = "subscript" LINK = "link" @@ -146,3 +152,32 @@ def footnote(number: int, paragraphs: List[Dict]) -> Dict: "attrs": {"number": number}, "content": paragraphs or [paragraph()], } + + +def latex_block(expression: str) -> Dict: + return { + "type": NodeType.LATEX_BLOCK, + "attrs": {"persistentExpression": expression, "dirty": True}, + } + + +def latex_inline(expression: str) -> Dict: + return { + "type": NodeType.LATEX_INLINE, + "attrs": {"expression": expression, "persistentExpression": expression}, + } + + +def pullquote(paragraphs: List[Dict]) -> Dict: + return { + "type": NodeType.PULLQUOTE, + "attrs": {"align": None, "color": None}, + "content": paragraphs or [paragraph()], + } + + +def callout_block(paragraphs: List[Dict]) -> Dict: + return { + "type": NodeType.CALLOUT_BLOCK, + "content": paragraphs or [paragraph()], + } diff --git a/tests/substack/fixtures/full_features.expected.json b/tests/substack/fixtures/full_features.expected.json index ad7e75c..ecd9259 100644 --- a/tests/substack/fixtures/full_features.expected.json +++ b/tests/substack/fixtures/full_features.expected.json @@ -548,6 +548,120 @@ { "type": "horizontal_rule" }, + { + "type": "paragraph", + "content": [ + { + "type": "text", + "text": "A paragraph with inline math " + }, + { + "type": "latex", + "attrs": { + "expression": "E=mc^2", + "persistentExpression": "E=mc^2" + } + }, + { + "type": "text", + "text": ", H" + }, + { + "type": "text", + "text": "2", + "marks": [ + { + "type": "subscript" + } + ] + }, + { + "type": "text", + "text": "O subscript, and x" + }, + { + "type": "text", + "text": "2", + "marks": [ + { + "type": "superscript" + } + ] + }, + { + "type": "text", + "text": " superscript." + } + ] + }, + { + "type": "latex_block", + "attrs": { + "persistentExpression": "E=mc^2", + "dirty": true + } + }, + { + "type": "pullquote", + "attrs": { + "align": null, + "color": null + }, + "content": [ + { + "type": "paragraph", + "content": [ + { + "type": "text", + "text": "A pull quote with " + }, + { + "type": "text", + "text": "bold", + "marks": [ + { + "type": "strong" + } + ] + }, + { + "type": "text", + "text": " text." + } + ] + } + ] + }, + { + "type": "calloutBlock", + "content": [ + { + "type": "paragraph", + "content": [ + { + "type": "text", + "text": "A callout block with a " + }, + { + "type": "text", + "text": "link", + "marks": [ + { + "type": "link", + "attrs": { + "href": "https://example.com" + } + } + ] + }, + { + "type": "text", + "text": "." + } + ] + } + ] + }, { "type": "captionedImage", "content": [ diff --git a/tests/substack/fixtures/full_features.md b/tests/substack/fixtures/full_features.md index f2eca43..3cd028c 100644 --- a/tests/substack/fixtures/full_features.md +++ b/tests/substack/fixtures/full_features.md @@ -55,6 +55,20 @@ plain code block without a language --- +A paragraph with inline math $E=mc^2$, H~2~O subscript, and x^2^ superscript. + +$$ +E=mc^2 +$$ + +:::pullquote +A pull quote with **bold** text. +::: + +:::callout +A callout block with a [link](https://example.com). +::: + ![A captioned image](https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png) [![Linked image alt](https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png)](https://example.com/target) diff --git a/tests/substack/test_from_markdown_features.py b/tests/substack/test_from_markdown_features.py index 5a56323..5222e51 100644 --- a/tests/substack/test_from_markdown_features.py +++ b/tests/substack/test_from_markdown_features.py @@ -174,3 +174,69 @@ def test_linked_image(self): attrs = block["content"][0]["attrs"] assert attrs["src"] == "https://i/x.png" assert attrs["href"] == "https://link" + + def test_latex_block(self): + post = make_post() + post.from_markdown("$$\nE=mc^2\n$$\n") + block = body(post)[0] + assert block["type"] == "latex_block" + assert block["attrs"]["persistentExpression"] == "E=mc^2" + assert block["attrs"]["dirty"] is True + + def test_latex_inline(self): + post = make_post() + post.from_markdown("A paragraph with $E=mc^2$ inline.") + para = body(post)[0] + assert para["type"] == "paragraph" + latex = [n for n in para["content"] if n["type"] == "latex"] + assert len(latex) == 1 + assert latex[0]["attrs"]["expression"] == "E=mc^2" + assert latex[0]["attrs"]["persistentExpression"] == "E=mc^2" + + def test_superscript(self): + post = make_post() + post.from_markdown("E=mc^2^ here.") + para = body(post)[0] + sup = [n for n in para["content"] if n.get("marks")] + assert sup[0]["text"] == "2" + assert sup[0]["marks"] == [{"type": "superscript"}] + + def test_subscript(self): + post = make_post() + post.from_markdown("H~2~O here.") + para = body(post)[0] + sub = [n for n in para["content"] if n.get("marks")] + assert sub[0]["text"] == "2" + assert sub[0]["marks"] == [{"type": "subscript"}] + + def test_subscript_does_not_clash_with_strikethrough(self): + post = make_post() + post.from_markdown("~~struck~~ and H~2~O") + para = body(post)[0] + marks = { + m["type"] + for n in para["content"] + for m in n.get("marks", []) + } + assert marks == {"strikethrough", "subscript"} + + def test_pullquote(self): + post = make_post() + post.from_markdown(":::pullquote\nA **bold** quote.\n:::\n") + block = body(post)[0] + assert block["type"] == "pullquote" + assert block["attrs"] == {"align": None, "color": None} + para = block["content"][0] + assert para["type"] == "paragraph" + assert para["content"][1]["text"] == "bold" + assert para["content"][1]["marks"] == [{"type": "strong"}] + + def test_callout(self): + post = make_post() + post.from_markdown(":::callout\nA callout with a [link](https://example.com).\n:::\n") + block = body(post)[0] + assert block["type"] == "calloutBlock" + para = block["content"][0] + assert para["type"] == "paragraph" + link = [n for n in para["content"] if n.get("marks")][0] + assert link["marks"][0]["attrs"]["href"] == "https://example.com"