{{ message }}
Repository navigation
fix(muya): pair emphasis delimiters the way CommonMark does (#2086) - #5555
Merged
Merged
Conversation
The live editor's inline lexer matched `*` / `_` runs with a regexp, so every opener took the first plausible closer to its right. `*a *b* c*` came out as `<em>a *b</em> c*`, and `**a **b** c**` likewise — while the static/export path (marked) rendered both as nested emphasis. Replace the `strong` / `em` patterns with the spec's delimiter-run algorithm: scan the inline string's runs, decide for each whether it can open and/or close from its flanking characters, then let every closer bind to the nearest still-unmatched opener, honouring the rule of three and spending two characters at a time for strong. Runs inside a construct that binds more tightly — a code span, autolink, raw HTML tag, math span, link or image — are skipped, so `*a `*`*` is one em around a code span. The CJK flanking widening from #4307 is carried over unchanged, and `lowerPriority` still guards §6.4 rule 17. Against the CommonMark 0.31 "Emphasis and strong emphasis" section the inline lexer goes from 82/132 to 131/132. The one remaining example is #354, which needs `PUNCTUATION_REG` extended to CommonMark 0.30's Unicode symbol categories — a separate gap, locked in the new suite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ring
Follow-up to the emphasis rewrite, from code review.
A run that closes one pair and then opens the next spends characters from
opposite ends, so a single "how many are left" counter is not enough: it
handed the same character to both pairings. `*foo***bar*` emitted the
second span one character early, and `**foo***bar**` emitted two
overlapping spans, which cost the tokenizer the whole `<em>bar</em>`.
Track the run's unspent window as `[left, right)` instead — the closer
advances `left`, the opener retreats `right`.
Three more problems, all of them the scan not agreeing with the
tokenizer:
- `backlash` matches `\(`, so trying it before the math rules made the
`case '\\'` branch unreachable and left the `*` inside `\(x*y\)` a
delimiter. Order the scan like `INLINE_HANDLERS` does.
- Each nesting level re-scanned its own content, which judged the
content's edge runs against the string boundary rather than the
markers now around them: `*_b.__*` grew an inner `<em>` the enclosing
scan had deliberately refused. Key the spans by absolute position and
hand the same map down to the children instead.
- `lowerPriority` ran per level over the whole span, turning a
keystroke on deeply nested emphasis into seconds of work
(`'*'*2000 + a + '*'*2000`: 34 ms before the rewrite, 4.6 s after).
It is redundant now that the scan steps over what binds more tightly,
so drop it and complete that set instead: reference links and images
whose label resolves, extended autolinks, and the tighter-binding
check `tryLink` applies. The same input now tokenizes in 4.3 ms.
`~~` is deliberately not in that set: GFM strikethrough is a delimiter
run of its own, so `*a ~~b*c~~ d*` is `<em>a ~~b</em>c~~ d*`, as
cmark-gfm has it.
Over every line of the CommonMark corpus this changes three results
against the previous commit, all of them towards the spec — CM 0.31 #523
`*foo [bar* baz]` now yields `<em>foo [bar</em> baz]`.
Adds a seeded property test for the invariant the spec examples do not
pin down: no two spans may partially overlap, and a span's markers must
match.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The delimiter scan has to know where a link, image, reference, autolink or
math span starts and ends, so it can step over the `*` and `_` inside one.
It answered that with its own copy of each handler's acceptance gate, and
two of the copies were already wrong:
- a reference link was taken as inert without `tryReferenceLink`'s
`lowerPriority` check, so a code span overrunning the brackets hid
delimiters the tokenizer would go on to treat as real;
- an extended autolink was taken as inert everywhere, though
`tryAutoLinkExtension` only ever makes one at the top level — inside a
`~~…~~` the scan skipped a bare URL the tokenizer renders as plain
text, losing the emphasis in `~~www.x.com/a*b*c~~`.
Lift the three gates into `utils.ts` — `matchBracketed`, `matchReference`,
`matchExtendedAutoLink`, over a shared sticky `matchAt` so they can answer
at an index as well as at the head of a string — and call them from both
sides. `tryImage`, `tryLink`, `tryReferenceLink`, `tryReferenceImage` and
`tryAutoLinkExtension` shrink to one call each. `BACKSLASH_MATH_RULES`
moves to `rules.ts` for the same reason.
`collectRuns` walks the whole inline string, and three of its per-character
tests were regexps where a comparison does. On a 2000-character prose
paragraph the scan spent most of its time deciding that ordinary letters
start nothing:
- the alphanumeric test before an extended autolink ran `.test()` on a
freshly materialized one-character string, once per character;
- an extended autolink was then attempted at every word start, 351 times,
though its pattern needs `www.`, `http` or `@` somewhere;
- the boundary test before it was a second regexp on another one-char
string.
Character codes for the first and third, and one `includes` pass per scan
for the second. The scan drops from 0.047 ms to 0.005 ms, ~6% off a
keystroke's tokenize — this runs on the focused paragraph on every key.
Also names the two things the pairing loop kept spelling out: `unspent(run)`
for the `[left, right)` window the `IDelimiterRun` comment describes, and
`canPair` for the opener search, whose `break` used to mean "found" five
lines from a `break` meaning "not found".
…ated
Three places still describe the world before `emphasis.ts`:
- `INLINE_HANDLERS` claims its array order "IS the rule-precedence
contract". True for every handler but `tryStrongEm`, which sits third
and defers to constructs handled further down the list.
- `validateRules` was the "binds tighter than emphasis" set; emphasis no
longer consults it and its only remaining caller is the emoji guard.
Renamed `emojiValidateRules`, with a note that its membership is
inherited from that era and is wider than an emoji guard needs.
- `tokenizerFac`'s `emphasisSpans` parameter is defaulted, so the four
recursions that must NOT inherit the parent's map say so only by
omission — and they are correct for two opposite reasons. Written down
at the parameter, because getting it wrong yields a mis-ranged token
rather than a type error.
…ches The suite's token-tree renderer carried a local `escapeHtml` that the public `escapeHTML` already covers (the only difference, `'` → `'`, appears in none of the 132 emphasis examples), plus cases for `image`, `html_escape` and `hard_line_break` that no input in the file can reach.
Removes the 154 comment lines this branch added across the emphasis scanner, the lexer, the rules and the two new specs, keeping only the functional directives (eslint-disable, @vitest-environment, @ts-expect-error) and the pre-existing comments that merely moved file or had to be reworded because the symbol they named is gone.
csells
added a commit
to csells/MarkText
that referenced
this pull request
Oct 13, 2026
…arktext#5555) CommonMark's rule of three reads the lengths of the delimiter runs that hold the opener and the closer, and its openers-bottom classes read the closer run's length. cmark and upstream MarkText (marktext#5555) use each run's original length. Core used what earlier pairings had left of the run, so a run that closed one pair and then opened the next was judged by its remainder: - `*foo***bar*` read as `<em>foo</em>**bar*`, and now reads as `<em>foo</em>*<em>bar</em>`, as upstream does. - `**foo***bar**` read as `<strong>foo</strong>*bar**`, and now reads as `<strong>foo</strong><em>bar</em>*`. - `..**a* a*b` read as `..<em><em>a</em> a</em>b`, and now reads as `..*<em>a</em> a*b`. Each inline delimiter now keeps its run's original length. The pairing check and the openers-bottom and closer-index keys use it, while the run's remaining length still decides how many characters a pairing spends. Upstream's lexer also turned `a*\*b` into an empty emphasis. That is not copied: CommonMark, the previous upstream and micromark keep the escaped run literal, and a test pins it. emphasis-original-run-length.spec.ts holds upstream's own cases from emphasisDelimiterRuns.spec.ts, the corpus case above, the escaped run, and every CommonMark 0.31.2 emphasis example. The first three cases fail without this change. E2E: emphasis-delimiter-runs.spec.ts types the closing asterisk of `*foo***bar` with the real keyboard in the built app, sees two emphases and saves `*foo***bar*`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Closes #2086.
The bug
The live editor renders
*a *b* c*as<em>a *b</em> c*and**a **b** c**as<strong>a **b</strong> c**. CommonMark (and GitHub,and pandoc) read both as nested emphasis:
The export/static path is fine — it goes through marked, which passes the
whole emphasis section. It is muya's own inline lexer, which the editor
uses, that is wrong, so the two paths disagree on the same document.
Root cause
inlineRenderer/rules.tsmatched emphasis with two regexps:A lazy quantifier stops at the first syntactically plausible closer. But
CommonMark §6.2 says a closer binds to the nearest still-unmatched
opener. In
*a *b* c*the middle*is preceded by a space, so it canonly open — it claims the third
*, and the outer pair is left to spanthe whole line. No single pattern can express that, because the answer
depends on the runs in between.
The fix
New
packages/muya/src/inlineRenderer/emphasis.tsimplements the specalgorithm for one inline string:
*/_delimiter runs, recording eachrun's length and whether it can open and/or close, from the code
points on either side (left/right-flanking, plus the intraword
_rule).
autolinks, raw HTML tags, math spans, links, images, and backslash
escapes — so their contents hold no delimiters. That is what makes
*a `*`*one em around a code span rather than two stray asterisks.three and spending two characters at a time when both sides have them.
Openers spend their characters right-to-left and closers left-to-right,
so nested pairs nest properly and leftover characters stay literal
(
**foo*→*<em>foo</em>).tryStrongEmnow looks the current offset up in that map instead ofrunning a regexp. Token shape is unchanged (
raw,marker,range,children,backlash), so the renderer,format()and the markerhighlighting need no changes.
The CJK flanking widening from #4307 is carried over verbatim — it stays
additive — and
lowerPrioritystill enforces §6.4 rule 17 on theaccepted span.
Result
The inline lexer against the CommonMark 0.31 "Emphasis and strong
emphasis" section: 82/132 → 131/132.
Newly correct includes all of the issue's cases plus, e.g.:
*a *b* c*<em>a *b</em> c*<em>a <em>b</em> c</em>_foo _bar_ baz_<em>foo _bar</em> baz_<em>foo <em>bar</em> baz</em>foo***bar***bazfoo<em><strong>bar</strong></em>baz**foo*<em>*foo</em>*<em>foo</em>*foo****<em>foo</em>****a `*`**a <code>*</code>*<em>a <code>*</code></em>foo *****foo <strong>*</strong>foo *****(literal)The one example still failing is #354 (
*£*bravo.): CommonMark 0.30widened "Unicode punctuation" to the S (symbol) categories and
PUNCTUATION_REGstill carries the 0.29 list. That is a separate gapfrom delimiter pairing — the ASCII third of that same example,
*$*alpha., passes — and it is locked in the new suite so it can only beremoved by fixing it.
Tests
src/inlineRenderer/__tests__/emphasisDelimiterRuns.spec.ts— replaysthe whole emphasis section through the inline lexer with the same
expected-failures contract
test/spec/uses: a listed example thatstarts passing fails the suite, so compliance can only go up. Plus
direct assertions for the issue's two lines and the outer token's
raw/range/children.e2e/tests/inline/nested-emphasis-2086.spec.ts— real Chromium: bothloading and typing
*a *b* c*produce a nested<em>and round-tripback to the same markdown.
Verified locally: muya unit suite and
test:specunchanged apart fromthe new tests; the muya Playwright suite shows the same failure set
before and after the change.
Follow-up commit (code review)
A review of the first commit found four things worth fixing, all now in
be4d6f4:A run spent from both ends handed out the same character twice. A run
that closes one pair and then opens the next spends characters from
opposite ends, so a single "how many are left" counter is not enough.
*foo***bar*placed the second span one character early, and**foo***bar**produced two overlapping spans — which cost thetokenizer the whole
<em>bar</em>. The run now tracks its unspent windowas
[left, right): the closer advancesleft, the opener retreatsright.backlashmatches\(, so the backslash-math branch was dead. Thescan tried the escape rule before the math rules — the reverse of
INLINE_HANDLERS— so withtexMathSingleBackslashon, the*inside\(x*y\)stayed a delimiter and*a \(x*y\) b*lost its emphasis.Reordered.
Each nesting level re-scanned its own content. That judged the
content's edge runs against the string boundary rather than the markers
now wrapped around them, so
*_b.__*grew an inner<em>the enclosingscan had deliberately refused. Spans are now keyed by absolute position
and the same map is handed down to the children.
lowerPriorityper level made deep nesting quadratic. On'*'*2000 + a + '*'*2000— 1000 nested spans — a single tokenize took4.6 s (34 ms before this PR). It is redundant now that the scan steps over
what binds more tightly, so it is gone from
tryStrongEmand that set iscompleted instead: reference links/images whose label resolves, GFM
extended autolinks (extent-trimmed, as
tryAutoLinkExtensiontrims it),and the tighter-binding check
tryLinkapplies. Same input now tokenizesin 4.3 ms. Patterns are matched with sticky clones rather than
substring(i), so the scan allocates nothing per position.~~is deliberately not in that set: GFM strikethrough is a delimiterrun in its own right, so
*a ~~b*c~~ d*is<em>a ~~b</em>c~~ d*, ascmark-gfm has it.
Diffing every line of the CommonMark corpus against the first commit
gives three changes, all towards the spec — CM 0.31 #523
*foo [bar* baz]now yields<em>foo [bar</em> baz].Added a seeded property test for the invariant the spec examples do not
pin down and the first commit violated: no two spans may partially
overlap, and a span's two markers must match.
Note on the one red e2e run above: it was a flake. Running the full muya
Playwright suite locally with CI's settings gives the identical failure
set on both commits, and the re-run of that job on the same commit is
green.
🤖 Generated with Claude Code