Repository navigation
[Bugfix] tolerate malformed EXIF metadata during hashing (#56527) - #56576
Conversation
…m-project#56527) When an image contains malformed or corrupt EXIF segments (e.g. an invalid TIFF header in the APP1 segment), calls to `Image.getexif()` raise exceptions such as `SyntaxError: not a TIFF file` or Pillow internal parsing errors. While `normalize_image()` in `vllm/multimodal/image.py` already suppresses exceptions from EXIF processing, `MultiModalHasher.serialize_item()` called `getexif()` without error handling in both the `Image.Image` and `MediaWithBytes` branches, causing HTTP 500 server errors on requests containing such images. Extract `_get_image_id_bytes()` with safe exception handling: - Wrap `image.getexif()` in try-except block. - Fall back to standard pixel hashing / raw bytes when EXIF reading fails. - Add unit test `test_hash_image_malformed_exif` in `tests/multimodal/test_hasher.py`. Fixes vllm-project#56527 Signed-off-by: amasen02 <amasen02@users.noreply.github.com>
Address review feedback from @DarkLight1337 on vllm-project#56576: Move ImageOps to module-level imports. Signed-off-by: amasen02 <amasen02@users.noreply.github.com>
|
/ci run |
|
✅ Triggered Buildkite CI #88535 for commit |
|
The fix looks right to me, and I want to record why the new test is valid, because it is easy to glance at and think it is vacuous — I did at first. On a freshly opened image this payload does not raise at all: So the raise needs the earlier (suppressed) One call site still unguardedWhile checking whether this was the only place,
img = Image.open(BytesIO(data))
with contextlib.suppress(Exception): # normalize_image
img = ImageOps.exif_transpose(img)
img.load()
ImageOps.exif_transpose(img) # molmo2.py:1308
# SyntaxError: not a TIFF file (header b'XXXX\x00\x00\x00\x08' not valid)Same root cause and same uncaught- Happy either way on scope — it is a one-line For disclosure: I have an open PR (#54283) that reworks the framing inside |
|
/ci retry |
|
✅ Triggered Buildkite CI #89282 for commit |
Resolve vllm/multimodal/hasher.py: keep the framing from this branch and upstream's _get_image_id_bytes EXIF tolerance from vllm-project#56576. Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
…m-project#56527) (vllm-project#56576) Signed-off-by: amasen02 <amasen02@users.noreply.github.com> Co-authored-by: amasen02 <amasen02@users.noreply.github.com>

Purpose
Fixes #56527
When handling images with malformed or corrupt EXIF segments (for example, an APP1 segment containing an invalid TIFF header), calls to
Image.getexif()raise exceptions such asSyntaxError: not a TIFF file (header b'...' not valid)or Pillow internal parser errors.While
normalize_imageinvllm/multimodal/image.pywrapsImageOps.exif_transposeincontextlib.suppress(Exception)to gracefully tolerate malformed metadata,MultiModalHasher.serialize_itempreviously calledobj.getexif()directly without error handling in both theImage.ImageandMediaWithBytesbranches (vllm/multimodal/hasher.py). BecauseSyntaxErrordoes not inherit fromValueError, this uncaught exception causes the API server request to abort with HTTP 500.This PR introduces a dedicated helper
_get_image_id_bytes()with safe exception handling:Image.ExifTags.Base.ImageIDUUID tag inside atry ... except Exception:block.Image.Image) ororiginal_bytes(forMediaWithBytes), matching the error tolerance ofnormalize_image.Test Plan
Added unit test
test_hash_image_malformed_exifintests/multimodal/test_hasher.py:MultiModalHasher.hash_kwargs("blake3", image=image)produces a valid hash without raisingSyntaxError.MediaWithBytes(image, data)wrapping the malformed image also serializes and hashes successfully.test_hash_image_exif_idcontinues to pass and correctly extract valid UUID ImageIDs.Run tests:
pytest tests/multimodal/test_hasher.py -k "test_hash_image_malformed_exif or test_hash_image_exif_id"Test Result
Linting & formatting:
Output: