Data URI fragments are decoded as payload, appending text or corrupting bytes · Issue #2690 · microsoft/markitdown · GitHub
Skip to content

Data URI fragments are decoded as payload, appending text or corrupting bytes #2690

Description

A raw #fragment becomes part of the converted content. With base64, it can silently add bytes.

from markitdown import MarkItDown

md = MarkItDown(enable_plugins=False)
print(repr(md.convert_uri('data:text/plain,Hello#section').text_content))
# Actual: 'Hello#section'; expected: 'Hello'

print(repr(md.convert_uri('data:text/plain;base64,SGVsbG8h#AAAA').text_content))
# Actual: 'Hello!\x00\x00\x00'; expected: 'Hello!'

Reproduced on main 4cc9fa17653d695d64fb9eee5b33d4de55ff84e8, macOS, Python 3.12.15. It also occurs on #2682, which fixes a different decoding problem.

parse_data_uri decodes everything after the comma, including the fragment. Strip the raw fragment before percent-decoding: %23 must still produce a literal #, and ? must remain in the payload. The WHATWG Fetch data URL processor excludes fragments; Node's fetch returns the expected text for both examples.

Parser and public API regression tests reproduced both failures. Escaped-hash and query controls passed. I searched existing issues and PRs and didn't find this failure covered.

Investigated with Hermes AI assistance.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions