Bug report
I've noticed unexpectedly poor performance of calling _io.BufferedReader.tell() in a Python script I've been working on, when compared to calling _io.BytesIO.tell(). From looking at the source in https://github.com/python/cpython/blob/main/Modules/_io/bufferedio.c it looks to me like _io.BufferedReader.tell() may be slow because it ignores the cached stream position, instead re-getting the position from the raw stream on every call:
cpython/Modules/_io/bufferedio.c/#_io__Buffered_tell_impl appears to always get the position by calling _buffered_raw_tell(). The implementation notes specify that the absolute position of the raw stream is cached if possible, so I'm suspecting that _io__Buffered_tell_impl should be using the RAW_TELL macro instead of _buffered_raw_tell so that it uses the cached value when available.
It may be that the cached value cannot be relied upon in _io__Buffered_tell_impl, and that calling _buffered_raw_tell() is necessary here, but I couldn't tell if that was the case from a quick look at the source and git blame, so I have made this bug report.
The script below does some simple timing of the _io.BufferedReader.tell() and _io.BytesIO.tell() functions.
Example output from the script:
BufferedReader.tell() time:
0.5720981000049505s
BytesIO.tell() time:
0.050327699980698526s
I would have expected both functions to be within the same order of magnitude.
import timeit
from io import BytesIO
import _io
import tempfile
import os
timing_count = 1_000_000
print()
print()
temp_file_path = None
try:
handle, temp_file_path = tempfile.mkstemp()
# Use `open` so we get an _io.BufferedReader
with open(handle, 'rb') as f:
assert isinstance(f, _io.BufferedReader)
tell = f.tell
t = timeit.timeit("tell()", globals=globals(), number=timing_count)
# About 0.6s on my system.
print(f"BufferedReader.tell() time:\n\t{t}s")
finally:
# Remove the temporary file
if temp_file_path is not None:
os.remove(temp_file_path)
# I would expect _io_BufferedReader.tell() to have similar performance to BytesIO.tell(),
# but it takes about 10 times the duration compared to BytesIO.tell().
with BytesIO(b"") as f:
assert isinstance(f, _io.BytesIO)
tell = f.tell
t = timeit.timeit("tell()", globals=globals(), number=timing_count)
# About 0.05s on my system.
print(f"BytesIO.tell() time:\n\t{t}s")
Your environment
I am working on an addon script for Blender 4.0 alpha with its bundled Python version, which reports the Python version as 3.10.11 (main, May 9 2023, 12:29:22) [MSC v.1928 64 bit (AMD64)]
- CPython versions tested on: 3.10.11
- Operating system and architecture: Windows-10-10.0.19045-SP0
Bug report
I've noticed unexpectedly poor performance of calling
_io.BufferedReader.tell()in a Python script I've been working on, when compared to calling_io.BytesIO.tell(). From looking at the source in https://github.com/python/cpython/blob/main/Modules/_io/bufferedio.c it looks to me like_io.BufferedReader.tell()may be slow because it ignores the cached stream position, instead re-getting the position from the raw stream on every call:cpython/Modules/_io/bufferedio.c/#_io__Buffered_tell_implappears to always get the position by calling_buffered_raw_tell(). The implementation notes specify that the absolute position of the raw stream is cached if possible, so I'm suspecting that_io__Buffered_tell_implshould be using theRAW_TELLmacro instead of_buffered_raw_tellso that it uses the cached value when available.It may be that the cached value cannot be relied upon in
_io__Buffered_tell_impl, and that calling_buffered_raw_tell()is necessary here, but I couldn't tell if that was the case from a quick look at the source and git blame, so I have made this bug report.The script below does some simple timing of the
_io.BufferedReader.tell()and_io.BytesIO.tell()functions.Example output from the script:
I would have expected both functions to be within the same order of magnitude.
Your environment
I am working on an addon script for Blender 4.0 alpha with its bundled Python version, which reports the Python version as
3.10.11 (main, May 9 2023, 12:29:22) [MSC v.1928 64 bit (AMD64)]