Fix Unicode width, precision and escaping for wide strings by tianrking · Pull Request #4970 · fmtlib/fmt · GitHub
Skip to content

Fix Unicode width, precision and escaping for wide strings - #4970

Open
tianrking wants to merge 2 commits into
fmtlib:mainfrom
tianrking:codex/unicode-wide-string-formatting
Open

tianrking wants to merge 2 commits into
fmtlib:mainfrom
tianrking:codex/unicode-wide-string-formatting

Conversation

@tianrking

@tianrking tianrking commented Oct 4, 2026 •

Copy link
Copy Markdown

Fix Unicode display width, precision and debug escaping for wide strings. UTF-16 surrogate pairs are decoded together; unpaired surrogates keep their individual escapes.

The wide-character codepoint traversal and string-formatting implementation now live in xchar.h. Narrow-string formatting remains in format.h, with the existing escape scanner available to headers that do not include xchar.h.

Tests focus on Unicode width and precision, surrogate pairs, and debug escaping. The extra char8_t, range, output-iterator and dynamic-width cases have been removed.

Validation with MSVC 19.44, x64, C++20 Debug: full CMake build passes and all 45 xchar tests pass. CTest passes 20 of 22 test programs; chrono_test.locale and unicode_test.legacy_locale fail with the Windows locale errors already recorded in the original validation. Changed code passes clang-format 20 and git diff --check.

@tianrking
tianrking requested a review from vitaut as a code owner October 4, 2026 18:16

@vitaut vitaut left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Could the wide-character-specific code be moved to xchar.h? In particular, the new for_each_codepoint overload and wide-string formatting logic.

@vitaut

vitaut commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

@vitaut

vitaut commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

@vitaut vitaut left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there are CI failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants