{{ message }}
Repository navigation
Conversation
…eplace on binary files (browser-use#6115)
|
|
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Reply to a comment to ask cubic a question or push back. It learns from your replies.
Click Fix with cubic on a comment, or reply @cubic-dev-ai fix this, and cubic writes the fix.
View guided diff | Re-trigger cubic
Author
|
Addressed in commit 0d2c4e3: extended MIME detection in AgentMessagePrompt for .gif (image/gif) and .webp (image/webp) with unit tests. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Enables structured image reading for internal
Base64BinaryFilefiles inFileSystem.read_file_structured()so vision models receive image payloads when reading agent-created images, adds a clear error guard inFileSystem.replace_file_str()to prevent invalid text replacements on binary files, and extends MIME detection inread_state_imagesprompt generation for GIF and WebP image files.Problem
PngFile,GifFile,JpgFile,JpegFile,WebpFile) are written to the agent's file system, callingread_file_structured()treated them as text. It extractedfile_obj.read()(the summary string[binary <ext> file, <N> bytes]) and returnedimages=None. Consequently, invoking theread_fileaction on internal images gave the vision model no image data inActionResult.images.replace_file_str()on binary files attempted string replacement on the stub description and passed it back tofile_obj.write(), resulting in a base64 decode validation failure instead of a clear error.AgentMessagePrompt,read_state_imagesonly explicitly identified.pngand defaulted all other image files toimage/jpeg, mislabeling.gifand.webppayloads.Solution
FileSystem.read_file_structured(): checkisinstance(file_obj, Base64BinaryFile). If true, populateresult['images'] = [{'name': file_obj.full_name, 'data': file_obj.content}]and setresult['message'] = f'{sanitize_note}Read image file {resolved}.', matching external image read behavior.FileSystem.replace_file_str(): guard againstisinstance(file_obj, Base64BinaryFile)and returnError: Cannot replace text in binary file '{full_filename}'. Overwrite it instead., matchingappend_file_content.browser_use/agent/prompts.py: extend MIME detection inAgentMessagePromptfor.gif(image/gif) and.webp(image/webp) so the model receives the correct media type.Testing
test_read_internal_binary_image_returns_images_payloadverifyingread_file_structuredandread_filereturn the image data payload for internal binary files.test_replace_file_str_on_binary_file_returns_clear_errorverifyingreplace_file_strreturns a descriptive error and does not corrupt existing binary files.test_agent_message_prompt_images_includedverifying GIF and WebP MIME types and data URLs inread_state_images.tests/ci/test_file_system_images.py,tests/ci/test_file_system_llm_integration.py, andtests/ci/infrastructure/test_filesystem.py): all 124 tests passed.ruff check,codespell, andpyright: 0 errors.Related Issue
Closes #6115