Poor performance of `_io.BufferedReader.tell` · Issue #107158 · python/cpython · GitHub
Skip to content

Poor performance of _io.BufferedReader.tell #107158

Description

@Mysteryem

Bug report

I've noticed unexpectedly poor performance of calling _io.BufferedReader.tell() in a Python script I've been working on, when compared to calling _io.BytesIO.tell(). From looking at the source in https://github.com/python/cpython/blob/main/Modules/_io/bufferedio.c it looks to me like _io.BufferedReader.tell() may be slow because it ignores the cached stream position, instead re-getting the position from the raw stream on every call:

cpython/Modules/_io/bufferedio.c/#_io__Buffered_tell_impl appears to always get the position by calling _buffered_raw_tell(). The implementation notes specify that the absolute position of the raw stream is cached if possible, so I'm suspecting that _io__Buffered_tell_impl should be using the RAW_TELL macro instead of _buffered_raw_tell so that it uses the cached value when available.

It may be that the cached value cannot be relied upon in _io__Buffered_tell_impl, and that calling _buffered_raw_tell() is necessary here, but I couldn't tell if that was the case from a quick look at the source and git blame, so I have made this bug report.

The script below does some simple timing of the _io.BufferedReader.tell() and _io.BytesIO.tell() functions.
Example output from the script:

BufferedReader.tell() time:
        0.5720981000049505s
BytesIO.tell() time:
        0.050327699980698526s

I would have expected both functions to be within the same order of magnitude.

import timeit
from io import BytesIO
import _io
import tempfile
import os

timing_count = 1_000_000

print()
print()

temp_file_path = None
try:
    handle, temp_file_path = tempfile.mkstemp()
    # Use `open` so we get an _io.BufferedReader
    with open(handle, 'rb') as f:
        assert isinstance(f, _io.BufferedReader)
        tell = f.tell
        t = timeit.timeit("tell()", globals=globals(), number=timing_count)
        # About 0.6s on my system.
        print(f"BufferedReader.tell() time:\n\t{t}s")
finally:
    # Remove the temporary file
    if temp_file_path is not None:
        os.remove(temp_file_path)

# I would expect _io_BufferedReader.tell() to have similar performance to BytesIO.tell(),
# but it takes about 10 times the duration compared to BytesIO.tell().
with BytesIO(b"") as f:
    assert isinstance(f, _io.BytesIO)
    tell = f.tell
    t = timeit.timeit("tell()", globals=globals(), number=timing_count)
    # About 0.05s on my system.
    print(f"BytesIO.tell() time:\n\t{t}s")

Your environment

I am working on an addon script for Blender 4.0 alpha with its bundled Python version, which reports the Python version as 3.10.11 (main, May 9 2023, 12:29:22) [MSC v.1928 64 bit (AMD64)]

  • CPython versions tested on: 3.10.11
  • Operating system and architecture: Windows-10-10.0.19045-SP0

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance or resource usage

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions