{{ message }}
Repository navigation
Fix jsonable_encoder raising an unclear UnicodeDecodeError for non-UTF-8 bytes - #16350
Open
usamabhatti1998 wants to merge 1 commit into
Open
usamabhatti1998 wants to merge 1 commit into
usamabhatti1998 wants to merge 1 commit into
Conversation
Shriprasad-P
left a comment
There was a problem hiding this comment.
Thanks for the PR. Looks focused from a first pass — please confirm tests cover the new path.
jasraj1111
reviewed
Oct 5, 2026
jasraj1111
left a comment
There was a problem hiding this comment.
Pulled this branch locally and verified with pytest tests/test_jsonable_encoder.py and ruff check. The three new tests pass cleanly on Python 3.11 (Windows) and the ValueError message gives clear, actionable guidance on using custom_encoder.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
jsonable_encoder(some_bytes)currently callssome_bytes.decode()with no error handling. For anybytesvalue that isn't valid UTF-8 (raw binary data — hashes, encrypted blobs, protobuf, DB blob columns), this raises an unhandledUnicodeDecodeErrorstraight from deep inside FastAPI's internals, with no indication of what went wrong or how to fix it — even though a working escape hatch (custom_encoder={bytes: ...}) already exists and just isn't discoverable from the error.Fix
Wrap the decode in a named
bytes_encoder()function that re-raises as aValueError(consistent with theValueErrorjsonable_encoderalready raises for other unencodable types) with a message that explains the problem and points atcustom_encoderas the fix. The existing pattern of naming non-trivial encoders as top-level functions (e.g.decimal_encoder,isoformat) is followed rather than keeping an inline lambda.Testing
Added three tests to
tests/test_jsonable_encoder.py:test_encode_bytes_valid_utf8: valid UTF-8 bytes still encode correctly (unchanged behavior)test_encode_bytes_invalid_utf8_raises_clear_error: invalid UTF-8 now raisesValueErrorwith an actionable messagetest_encode_bytes_invalid_utf8_with_custom_encoder: the existingcustom_encoderescape hatch still works and now has a regression testNot a breaking change for the success path; the only change is the exception type/message for previously-crashing input.