{{ message }}
Repository navigation
fix(tokenizer): reject non-ASCII entity names - #1093
Merged
Merged
Conversation
irimias57
added
contributor
compiler
irimias57
approved these changes
Oct 20, 2026
irimias57
left a comment
irimias57
left a comment
Member
There was a problem hiding this comment.
Thank you for the thorough fix, @jinchuntan! The shared ASCII check rejects accented and supplementary characters before WebAssembly emission while preserving Unicode string literals and the existing ASCII naming rules, and the tokenizer/compiler tests, type checks, and lint all pass against current main. If you would like another tokenizer issue, #1047 is currently unassigned and labeled help wanted.
andorthehood
approved these changes
Oct 20, 2026
andorthehood
left a comment
andorthehood
left a comment
Member
There was a problem hiding this comment.
Thanks again for the contribution!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1046.
Non-ASCII names now fail during tokenization with
INVALID_IDENTIFIERand the source line before reaching Wasm emission. Valid ASCII punctuation, constant capitalization rules and quoted strings keep their existing behavior.Tests cover accented names and emoji in declarations, references and compile time operands plus an ASCII program that validates and runs as WebAssembly. Also documents the naming restriction.
Validation on Node 24.16.0: 508 tokenizer tests and 333 compiler tests passed, along with both type checks, Biome and the pre-commit checks.