sphinx-gp-mermaid: build-time Mermaid diagrams as dual inline SVG by tony · Pull Request #61 · git-pull/gp-sphinx · GitHub
Skip to content

sphinx-gp-mermaid: build-time Mermaid diagrams as dual inline SVG - #61

Merged
tony merged 20 commits into
mainfrom
mermaid
Jul 4, 2026
Merged

tony merged 20 commits into
mainfrom
mermaid

Conversation

@tony

@tony tony commented Jul 2, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Add sphinx-gp-mermaid, a workspace package rendering MyST mermaid fences to inline SVG at build time via mmdc (@mermaid-js/mermaid-cli) — no client-side mermaid runtime, no async pop-in, no layout shift, and diagrams ride SPA navigation as finished DOM.
  • Render twice, toggle with CSS: each diagram produces a light and a dark SVG (palettes mapped to gp-furo tokens), both inlined and switched on body[data-theme]. Mermaid bakes id-scoped !important colors into its SVGs, so a single render cannot follow the theme toggle.
  • Cache aggressively: SVGs are content-addressed under <confdir>/_mermaid_cache, keyed on render version + theme + full mermaid config JSON + source — styling changes bust the cache, and the cache survives rm -rf docs/_build. Writes are atomic (pid-suffixed temp + rename), so parallel writers racing on one digest never read a torn SVG. The directory is excluded from source discovery automatically.
  • Degrade, don't fail: a missing or failing mmdc warns once per builder and falls back to the escaped diagram source, keeping the diagram's :name: anchor so cross-references still land; text/latex/man/texinfo builders emit a [diagram: <alt>] stand-in instead of crashing on the node.
  • Route plain fences at conf level: merge_sphinx_config() sets myst_fence_as_directive = ["mermaid"] whenever the extension lands in the final extension list; explicit overrides still win.
  • Document and demo: the package ships its full per-package Diátaxis tree (how-to, reference, dependents, examples) plus a committed, regenerable example corpus across diverse diagram types — rendered live on the docs Examples page as the extension's real dual-theme output, with no mermaid-cli/Chrome in the docs CI.
  • Provenance: extracted from tmuxp's docs/_ext/mermaid_inline.py (Docs: build-time mermaid + tmux-layout diagram rendering tmux-python/tmuxp#1071), where it is proven in production docs. tmuxp adopts this package in the companion PR.

Changes by area

New package: packages/sphinx-gp-mermaid

  • sphinx_gp_mermaid/__init__.py: mermaid directive (:caption:, :alt:, :name:) → marker node → HTML write-phase visitor → mmdc subprocess → dual normalized SVGs in a <figure>. Renderer resolution walks mermaid_cmd config → <confdir>/node_modules/.bin/mmdc → PATH; puppeteer config is generated with --no-sandbox and a Chrome discovered in puppeteer's cache. SVG normalization rewrites mermaid's hardcoded my-svg token everywhere it acts as an id — attribute values (including my-svg_… marker ids), #-refs in the scoped <style> and url(#…) marker references, and chart-desc-/chart-title- a11y ids — to a per-diagram-per-theme id while leaving visible label text intact. Repeats of the same diagram on one page get per-occurrence disambiguated ids (counted on the translator, which Sphinx creates fresh per document). The root width="100%" is replaced with explicit width/height from the viewBox, and the inline max-width is stripped.
  • _static/css/sphinx_gp_mermaid.css: ships inside the package and registers at builder-inited. Classes use the package's Tier B namespace: gp-sphinx-mermaid, gp-sphinx-mermaid__variant--theme-{light,dark}, gp-sphinx-mermaid__fallback.

Coordinator: gp_sphinx/config.py

  • Fence routing auto-set when sphinx_gp_mermaid is active; documented with a doctest.

Documentation

  • Per-package Diátaxis tree: docs/packages/sphinx-gp-mermaid/{index,how-to,reference,dependents,examples}.md — how-to (authoring, renderer/CI install, theming & caching behavior), reference (autoconfigvalues + autodirectives + setup), dependents, and a live Examples gallery; plus a curated packages-index bullet.
  • Example corpus: packages/sphinx-gp-mermaid/examples/ — diverse .mmd sources and their committed dual light/dark SVGs, rendered through the extension's own pipeline (furo palette + normalization), a typed generate.py regenerator, and a README gallery. A test pairs each source with its committed outputs.
  • docs/_ext/mermaid_examples.py: a mermaid-examples directive that inlines the committed SVGs in the extension's <figure class="gp-sphinx-mermaid"> markup, so the docs Examples page shows the real dual-theme output. docs/conf.py loads sphinx_gp_mermaid (which ships the toggle CSS).

Workspace wiring

  • Root pyproject.toml (uv source, dev group, isort first-party, doctest testpath), wheel smoke runner in scripts/ci/package_tools.py, ux cluster assignment in docs/_ext/package_reference.py, and a legacy extensions/ redirect.

Tests: tests/ext/mermaid/

  • Unit: SVG normalization (including label-text preservation and a11y id rewriting), digest stability, palettes, visitor markup, duplicate-diagram id disambiguation, renderer-missing fallback with anchor preservation, cache idempotency with no temp-file leftovers, setup registration.
  • Integration: a full HTML build over a stub fake_mmdc.py that honors the -i/-o/-c contract and bakes themeVariables.primaryColor into the SVG — proving the light and dark configs each cross the subprocess boundary; plus a text-builder build asserting the alt-text stand-in.
  • Examples: a hermetic build of the docs Examples page asserting one dual-theme figure per committed source with no renderer invoked, and a source↔SVG pairing guard over the committed corpus.
  • Palette tripwire: scrapes gp-furo-tokens light.ts/dark.ts and asserts every non-font themeVariable equals its token value, so a token retune fails in CI instead of drifting silently.

Design decisions

  • Dual render instead of client-side theming: mermaid's theming engine only accepts literal hex, and its output styles are id-scoped with !important — one SVG cannot be re-themed by page CSS. Two renders share identical geometry, so the CSS display toggle never shifts layout.
  • Conf-level fence routing: myst-parser reads myst_* config once at builder-inited; setting the value in the returned conf dict guarantees it exists before any Sphinx event fires and keeps routing in the one place consumers already configure. Explicit overrides win, proven by test.
  • Opt-in, not DEFAULT_EXTENSIONS: rendering requires a node toolchain (mermaid-cli + headless Chrome). Auto-loading would push that onto every consumer for a capability few docs use today.
  • Embed committed SVGs on the docs page, don't live-render: a live :::{mermaid} fence would need mmdc + headless Chrome in the docs CI, which runs under -W and gates the S3/CloudFront publish. Loading the extension ships its CSS and touches mmdc only on a live node, so the Examples page embeds the already-committed SVGs — keeping the docs build Node-free while still showing the extension's true dual-theme output.
  • Warn-once memo on the builder rather than a module global, keeping the extension's parallel_write_safe declaration honest — which the atomic temp-then-rename cache write backs up on the filesystem side.

Test plan

  • uv run ruff check . and uv run ruff format . — clean
  • just mypy — strict, clean
  • uv run py.test --reruns 0 — full suite including module doctests
  • just build-docs — the full Diátaxis tree and the Examples gallery render warning-clean; no mmdc/Chrome touched
  • tests/ext/mermaid/test_integration.py — both fence spellings render, dual variants carry per-theme fills, normalization holds, options pass through, packaged CSS ships, cache is excluded and populated, figure markup snapshot is stable
  • tests/ext/mermaid/test_mermaid.py — label text containing my-svg survives normalization, a11y ids rewritten, test_visitor_disambiguates_duplicate_diagram_ids proves repeats never share an id, test_fallback_preserves_name_anchor covers named and anonymous diagrams, cache leaves no .tmp behind
  • tests/ext/mermaid/test_examples_page.py — the docs Examples page builds and renders one dual-theme figure per committed source, no renderer invoked
  • tests/ext/mermaid/test_palette_sync.py — palettes match gp-furo tokens in both themes
  • Consumed end-to-end from tmuxp (companion PR): all existing production diagrams render from cache with zero re-renders

Companion PR

@codecov-commenter

codecov-commenter commented Jul 2, 2026 •

Copy link
Copy Markdown

@tony
tony force-pushed the mermaid branch 2 times, most recently from 4b0b91d to e12a0ca Compare July 2, 2026 00:43
tony added 16 commits July 3, 2026 17:46
why: Build-time mermaid rendering is proven in tmuxp's docs but lives in
its docs/_ext, unavailable to other gp-sphinx consumers. Scaffold the
workspace package so the renderer can move in and be reused.

what:
- packages/sphinx-gp-mermaid: pyproject (sphinx>=8.1, hatchling),
  README, py.typed, skeleton setup() returning parallel-safe metadata
- Root pyproject: uv source, dev group, isort first-party, pytest
  testpath entries
- package_tools: smoke_sphinx_gp_mermaid wheel smoke runner
- package_reference: assign the ux cluster
- docs: landing stub + legacy extensions/ redirect
- tests: add to the publishable-package set
why: The renderer is proven in tmuxp production docs; moving it into the
workspace package makes it installable by every gp-sphinx consumer.

what:
- Full pipeline: mermaid directive -> mermaid_inline node -> HTML
  write-phase visitor -> mmdc subprocess -> dual light/dark inline SVG
- Content-hash SVG cache under <confdir>/_mermaid_cache, keyed on
  render version, theme, full mermaid config JSON, and source; cache
  key inputs unchanged from the tmuxp version so existing caches hit
- mmdc resolution (config -> confdir node_modules -> PATH), puppeteer
  config generation with cached-Chrome discovery, graceful degradation
  to escaped source when the renderer is missing
- Config values: mermaid_cmd, mermaid_puppeteer_config
- CSS classes join the workspace namespace: gp-sphinx-diagram,
  gp-sphinx-diagram__variant--theme-{light,dark},
  gp-sphinx-diagram__fallback; stylesheet ships in the package's
  _static and registers at builder-inited
- Unit tests: SVG normalization, digest, palettes, visitor, fallback,
  cache idempotency, setup registration, static-path idempotence
why: The HTML-only visitor left text/latex/man/texinfo builds to crash
on an unhandled node, and the module-global warn-once flag contradicted
the extension's parallel_write_safe declaration.

what:
- text/man/latex/texinfo visitors emit an "[diagram: <alt>]" stand-in
  (alt falling back to caption), following sphinx.ext.graphviz
- Warn-once memo moves to a builder attribute (imgmath's pattern), so
  each writer process warns at most once and fresh builds start clean
- setup() excludes _mermaid_cache from source discovery at config-inited
- _render treats OSError (e.g. a resolved but non-executable command)
  as MermaidRendererMissing instead of crashing the build
- Tests: fallback-text cases, all four visitors, per-builder memo,
  PermissionError degradation, and a text-builder integration scenario
why: The unit suite never exercises the fence -> directive -> subprocess
-> dual-figure HTML pipeline, and the palette hexes are hand copies of
gp-furo-tokens values that could drift silently.

what:
- Full HTML build over a stub mmdc (a python script honouring the
  -i/-o/-c contract) that bakes themeVariables.primaryColor into the
  SVG, proving both theme configs cross the subprocess boundary
- Asserts both fence spellings render, SVG normalization, option
  passthrough (caption/alt/name), packaged CSS shipping, cache
  exclusion and population, and a stable figure-markup snapshot
- Palette tripwire: scrapes light.ts/dark.ts custom properties and
  checks every non-font themeVariable against its token (black/white
  keywords normalized), plus a coverage guard for new palette keys
why: myst-parser snapshots myst_* config at its own config-inited, so
sphinx_gp_mermaid cannot reliably register its fence routing from
setup(); conf level is the deterministic place, and consumers should
not need boilerplate for it.

what:
- merge_sphinx_config() sets myst_fence_as_directive = ["mermaid"]
  whenever sphinx_gp_mermaid lands in the final extension list
- Explicit myst_fence_as_directive overrides still win (overrides
  apply last)
- Tests: routing on with the extension, absent without it, override
  wins; docstring doctest documents the behavior
why: The extension declares parallel_write_safe, but a plain
write_text lets a parallel writer read a torn half-written SVG
from the shared confdir cache.

what:
- Write to a pid-suffixed .tmp sibling, then Path.replace onto
  the final path (POSIX-atomic; Sphinx's dump_search_index
  pattern)
- Assert cached content and no .tmp leftovers in the cache
  idempotency test
why: A :name:-ed diagram loses its anchor when mmdc is missing,
so cross-references to it 404 in degraded builds.

what:
- Hoist ids/fig_id computation above the render try block
- Emit fig_id on the fallback <pre>, matching the <figure> path
- Parametrized FallbackAnchorCase test for named and anonymous
  diagrams
why: Two identical diagrams on one page emitted identical
mermaid-<digest>-<theme> ids -- invalid duplicate DOM ids that
break the id-scoped styles and marker url() refs.

what:
- Count digest occurrences per translator (fresh per document,
  per worker process) and fold "-<n>" into repeat ids
- Occurrence 0 keeps today's id, so snapshots stay stable
- _svg_element_id gains occurrence kwarg + doctest
- Parametrized DuplicateDiagramCase visitor test
why: str.replace rewrote every "my-svg" substring, corrupting
visible label text such as "Deploy my-svg-viewer".

what:
- Compiled regex rewrites only id-token contexts: attribute
  values, #-refs/CSS selectors, and chart-desc-/chart-title-
  a11y ids
- Docstring contract + doctest for preserved label text
- NormalizeCase entries: label text preserved, a11y ids rewritten
why: The comment claimed myst-parser snapshots myst_* config at
config-inited; it actually snapshots at builder-inited
(create_myst_config). The code is right, the reason given wasn't.

what:
- Name the real event and tighten the comment to two lines
why: The package was missing from the categorized package
listing on the packages index.

what:
- Add sphinx-gp-mermaid bullet under Common libraries alongside
  its ux-cluster siblings; widen the section intro to cover
  shared page primitives
why: CSS standards reserve package-owned classes for
gp-sphinx-<pkg>__<thing>; gp-sphinx-diagram read as a Tier A
shared concept it isn't.

what:
- Rename gp-sphinx-diagram* -> gp-sphinx-mermaid* across the
  visitor, stylesheet, and tests (block/element/modifier shape
  unchanged, matching the gp-sphinx-fastmcp precedent)
- Re-record test_integration.ambr snapshot
why: The package shipped only a landing stub while its README
promised full documentation and every sibling carries the
how-to/reference/dependents tree.

what:
- how-to.md: authoring, renderer/CI install, theming/caching
  behavior, graceful degradation
- reference.md: autoconfigvalues + autodirectives + setup
- dependents.md + pyproject showcase opt-in
why: The package shipped no example diagrams, so readers had no way
to see what its dual light/dark output looks like across the range
of mermaid diagram types.

what:
- 11 diverse src/*.mmd sources (flowchart, sequence, class, state,
  er, gantt, pie, gitgraph, mindmap, timeline, block)
- Committed dual light/dark SVGs rendered through the extension's
  own pipeline (furo palette + id/size normalization)
- generate.py regenerator + pinned mermaid-cli toolchain manifest
- README gallery and a test pairing each source with its outputs
why: The docs never rendered a mermaid diagram — the extension
wasn't loaded and the committed example gallery was orphaned from
the site.

what:
- docs/_ext/mermaid_examples.py: a directive that inlines each
  committed example's source + its dual-theme SVG in the
  extension's gp-sphinx-mermaid figure markup (no live render)
- docs/conf.py: load sphinx_gp_mermaid (ships the toggle CSS) and
  the mermaid_examples directive
- Hermetic integration test asserting one dual-theme figure per
  committed source, no renderer invoked
why: mermaid was the only package without an examples.md; its
committed diagram gallery never appeared on the docs site.

what:
- examples.md: a {mermaid-examples} block, auto-discovered by the
  package landing as the Examples subpage
- Widen the docs-package-pages content guard to cover sphinx-gp-*
  and recognise the {mermaid-examples} live-demo marker
@tony

tony commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Code review

Found 1 issue:

  1. The examples gallery's regeneration instruction cites examples/generate.py, a path that only resolves from inside packages/sphinx-gp-mermaid/. A docs reader's context is the repo root, where that command fails; the script's own docstring and examples/README.md both give the runnable form, uv run python packages/sphinx-gp-mermaid/examples/generate.py. (docs/CLAUDE.md § "Keeping examples honest" — examples should be accurate and runnable.)

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

tony added 4 commits July 3, 2026 19:23
why: The examples page told readers to run `examples/generate.py`,
a path that only resolves inside the package dir, and a reader-
facing gallery shouldn't carry a maintainer regen note at all.

what:
- Drop the regen sentence from examples.md (siblings are reader-
  only; regen lives in the package README)
- Make the mermaid_examples module-docstring path repo-relative
why: setup()'s doctest only checked callable(); its docs/_ext
siblings assert the directive registers. _figure_markup claims to
mirror the visitor but never tested the role/aria markup.

what:
- setup() doctest records add_directive and asserts the directive
  name + class registered
- _figure_markup doctest asserts role/aria-hidden/aria-label
why: The helper's glob now also admits sphinx-gp-* packages (to
cover the mermaid examples page), but its docstring still said
"autodoc / ux" only.

what:
- Widen the docstring to name the gp family
why: Record the new build-time Mermaid diagram package under the
forthcoming unreleased entry.
@tony
tony merged commit f1592fe into main Jul 4, 2026
44 checks passed
tony added a commit to tmux-python/tmuxp that referenced this pull request Jul 4, 2026
The build-time mermaid diagram renderer, previously vendored in the
docs, now comes from the reusable upstream sphinx-gp-mermaid package.
Adoption keeps the cache-key inputs identical, so rendered diagrams are
unchanged and nothing re-renders on upgrade. The dependency lives only
in the dev and docs groups, so installed tmuxp is unaffected.

- **Adopt the package**: docs/conf.py loads sphinx_gp_mermaid, and
  sphinx-gp-mermaid==0.0.1a32 is pinned in the dev and docs groups
  alongside the gp-sphinx family (all moved to 0.0.1a32).

- **Drop the vendored copy**: the docs/_ext extension, its stylesheet,
  and its test suite are removed — all now maintained upstream in the
  gp-sphinx workspace.

- **Identical output**: same dual light/dark inline SVG rendered at
  build time; existing docs/_mermaid_cache entries stay valid, so
  adoption re-renders nothing. Rendered markup moves to the package's
  gp-sphinx-diagram* CSS namespace.

- **Provisioning unchanged**: the docs workflow's mmdc/Chrome toolchain
  is retained — the package resolves mmdc but does not vendor it — and
  code comments plus .gitignore are retargeted at the package.

Companion PR: git-pull/gp-sphinx#61 (the extracted sphinx-gp-mermaid
package)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants