ICU-23078 Use CharString for file names in genrb parseTransliterator and parseDependency by ManoharPaturi · Pull Request #4258 · unicode-org/icu · GitHub
Skip to content

ICU-23078 Use CharString for file names in genrb parseTransliterator and parseDependency - #4258

Open
ManoharPaturi wants to merge 1 commit into
unicode-org:mainfrom
ManoharPaturi:icu-23078-genrb-filenames
Open

ManoharPaturi wants to merge 1 commit into
unicode-org:mainfrom
ManoharPaturi:icu-23078-genrb-filenames

Conversation

@ManoharPaturi

Copy link
Copy Markdown

Description

ICU-23078 reports stack buffer overflows in genrb when building data from a deep enough directory. Following the report, I reproduced and fixed the two remaining fixed size file name buffers in parse.cpp.

parseTransliterator and parseDependency both declared:

char filename[256] = { '\0' };
char cs[128]       = { '\0' };

and filled them with unchecked uprv_strcat calls: the input or output directory first, then u_UCharsToChars(tokenValue->fChars, cs, tokenValue->fLength) which copies the resource string token of any length into the 128 byte buffer.

A bundle source with a 200 character file name in a process(transliterator) resource, run against a genrb built with address sanitizer:

ERROR: AddressSanitizer: stack-buffer-overflow
WRITE of size 1
    #0 u_UCharsToChars
    #1 parseTransliterator(...)
    #2 realParseTable(...)
    #3 parse
    #4 processFile
    #5 main
  This frame has 4 object(s):
    [32, 288) 'filename' (line 466)
    [352, 480) 'cs' (line 467) <== Memory access at offset 480 overflows this variable

The same report appears for parseDependency with a process(dependency) resource. Beyond the long token, the directory concatenation overflows filename[256] when the configured input or output directory is long, which matches the random build system crashes from the ticket.

The fix builds the file names with CharString, the same approach parseUCARules in the same file already uses, so the buffer grows as needed and non invariant characters in the token are reported as an error rather than copied. With this change the reproducer bundles produce a clean file open error for the transliterator case and a clean warning for the dependency case.

parseUCARules was already converted to CharString, so this closes the remaining two functions named in the ticket.

Testing

Added genrb_test.py, wired into check-local of the genrb Makefile following the pattern of the makeconv tool test. The test writes a valid transliterator bundle and bundles with 300 character file names in both process(transliterator) and process(dependency) resources, and checks that genrb accepts the first and last with a written .res file and reports a clean error for the unopenable transliterator file. Run against the pre-fix genrb the test fails with the sanitizer abort; against the fixed genrb it passes.

The full ICU data, including translit/root.txt which uses process(transliterator), rebuilds with the changed genrb, and intltest translit passes.

ICU-23078

Checklist

  • Required: Issue filed: ICU-23078
  • Required: The PR title must be prefixed with a JIRA Issue number. Example: "ICU-NNNNN Fix xyz"
  • Required: Each commit message must be prefixed with a JIRA Issue number. Example: "ICU-NNNNN Fix xyz"
  • Issue accepted (done by Technical Committee after discussion)
  • Tests included, if applicable
  • API docs and/or User Guide docs changed or added, if applicable
  • Approver: Feel free to merge on my behalf

…and parseDependency

parseTransliterator and parseDependency assembled file names in fixed
size stack buffers with unchecked concatenation: the resource string
token was copied into a 128 byte buffer and the input or output
directory and the token were appended to a 256 byte buffer. A bundle
with a long file name token, or a deep enough directory, overran the
buffers, which shows up as random genrb crashes in build systems with
long paths.

Build the file names with CharString, as parseUCARules already does,
so the buffers grow as needed and non invariant characters are
reported instead of being copied. Add a genrb regression test that
runs the tool on bundles with overlong transliterator and dependency
file names.
Copilot AI balanced review requested due to automatic review settings October 4, 2026 03:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@markusicu markusicu self-assigned this Oct 8, 2026
@ManoharPaturi

Copy link
Copy Markdown
Author

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants