Bug: FileSystem.read_file_structured returns empty images for internal image files and replace_file_str fails on binary files · Issue #6115 · browser-use/browser-use · GitHub
Skip to content

Bug: FileSystem.read_file_structured returns empty images for internal image files and replace_file_str fails on binary files #6115

Open

Description

@codeXsidd

Problem

When binary/image files (PngFile, GifFile, JpgFile, JpegFile, WebpFile via Base64BinaryFile) are created in the agent's file system (e.g. via write_file tool):

  1. FileSystem.read_file_structured() does not return the image data in result['images']. Instead, it treats the binary file as text, extracting file_obj.read() (the summary stub [binary <ext> file, <N> bytes]) and returning {'message': ..., 'images': None}. Consequently, when the agent invokes read_file on an internal image, the vision model receives no image payload in ActionResult.images.
  2. FileSystem.replace_file_str() does not guard against Base64BinaryFile. It attempts string replacement on the stub description and passes it to file_obj.write(), which fails with a base64 validation error rather than cleanly explaining that binary files cannot have text replaced.

Current Behavior

  • Calling await fs.read_file_structured('test.png') on an internal PNG file returns {'message': 'Read from file test.png.\n<content>\n[binary png file, 68 bytes]\n</content>', 'images': None}.
  • Calling await fs.replace_file_str('test.png', 'old', 'new') on an internal binary file attempts to validate the description string as base64 and raises/returns: Error: content for 'test.png' is not valid base64....

Expected Behavior

  • Calling await fs.read_file_structured('test.png') on an internal image file should populate result['images'] = [{'name': file_obj.full_name, 'data': file_obj.content}] and set result['message'] = 'Read image file test.png.', matching external image reads so read_file supplies images to the vision model.
  • Calling await fs.replace_file_str('test.png', 'old', 'new') should cleanly return Error: Cannot replace text in binary file 'test.png'. Overwrite it instead., matching Base64BinaryFile.append_file_content.

Reproduction

import asyncio
from browser_use.filesystem.file_system import FileSystem

async def main():
    fs = FileSystem('test_workspace')
    png_1x1 = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR4nGNgAAIAAAUAAXpeqz8AAAAASUVORK5CYII='
    await fs.write_file('test.png', png_1x1)

    # 1. read_file_structured returns images=None
    res = await fs.read_file_structured('test.png')
    assert res['images'] is not None, f"Expected images list, got {res['images']}"

    # 2. replace_file_str crashes with base64 decode error
    replace_res = await fs.replace_file_str('test.png', 'binary', 'text')
    print(replace_res)

asyncio.run(main())

Technical Context

  • Base64BinaryFile was introduced in PR filesystem: write_file can create tiny binary images (png/gif/jpg/webp) for upload flows #5280 to allow write_file to create real image files for upload workflows.
  • browser_use/filesystem/file_system.py:
    • In read_file_structured(), lines 844-868 currently state # For internal files, only non-image types are supported and call file_obj.read(). For Base64BinaryFile, content is base64 string, so read_file_structured() should set result['images'] = [{'name': file_obj.full_name, 'data': file_obj.content}].
    • In replace_file_str(), lines 949-960 perform string replacement and rewrite the file without checking isinstance(file_obj, Base64BinaryFile).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions