Issue 26927: test_mmap does not handle ValueError when no large file support - Python tracker

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
process
Status: closed Resolution: fixed
Dependencies: Superseder: test_io large file test failure on 32 bits Android platforms
View: 26926
Assigned To: Nosy List: Alex.Willmer, pitrou, python-dev, rosslagerwall, serhiy.storchaka, twouters, xdegaye
Priority: normal Keywords: patch

Created on 2016-05-03 14:00 by xdegaye, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
value-error.patch xdegaye, 2016-05-22 10:14 review
Messages (8)
msg264726 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-05-03 14:00
test_mmap fails on an android emulator running an x86 system image at API level 21.

======================================================================
ERROR: test_large_filesize (test.test_mmap.LargeMmapTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 755, in test_large_filesize
    with self._make_test_file(0x17FFFFFFF, b" ") as f:
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 738, in _make_test_file
    f.seek(num_zeroes)
ValueError: cannot fit 'int' into an offset-sized integer

======================================================================
ERROR: test_large_offset (test.test_mmap.LargeMmapTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 750, in test_large_offset
    with self._make_test_file(0x14FFFFFFF, b" ") as f:
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_mmap.py", line 738, in _make_test_file
    f.seek(num_zeroes)
ValueError: cannot fit 'int' into an offset-sized integer

----------------------------------------------------------------------
Ran 35 tests in 0.134s

FAILED (errors=2, skipped=6)
test test_mmap failed
1 test failed:
    test_mmap
Total duration: 0:00:01
msg264771 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-03 21:35
This is other manifestation of issue26926.
msg266065 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-05-22 10:14
f.seek(number) raises ValueError (ValueError: cannot fit 'int' into an offset-sized integer) when 'number' is greater than the size allowed by off_t. ValueError is not handled in test_mmap to detect that large file is not supported.

With this patch, test_large_filesize and test_large_offset are skipped as expected on an Android emulator.
msg266067 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-22 11:27
I'm wondering in what cases other exceptions (OSError, OverflowError) can be raised? Initial test was added in issue4681.
msg266072 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-05-22 12:22
On linux with large file support, OSError is raised when the file system limit is exceeded, here with ext4:

>>> f.seek(2**44 - 2**11)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
>>> f.seek(2**44 - 2**12)
17592186040320
>>> f.write(b'A' * 2**11)
2048
>>> f.write(b'A' * 2**11)
2048
>>> f.tell()
17592186044416
>>> f.write(b'A' * 2**11)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 27] File too large


On the Android emulator (no large file support):

>>> f.seek(2**31)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: cannot fit 'int' into an offset-sized integer
>>> f.seek(2**31 - 1)
2147483647
>>> f.write(b'A')
1
>>> f.tell()
-2147483648
>>> f.flush()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 75] Value too large for defined data type


Surprisingly f.write(b'A') does not raise an exception.
msg266088 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-22 16:28
When OverflowError is raised? Shouldn't it be replaced with ValueError?
msg266122 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-05-23 05:42
New changeset bca81ea8f898 by Serhiy Storchaka in branch '3.5':
Issue #26927: Fixed test_mmap on platforms with 32-bit off_t (like Android).
https://hg.python.org/cpython/rev/bca81ea8f898

New changeset 6147a2c99db0 by Serhiy Storchaka in branch 'default':
Issue #26927: Fixed test_mmap on platforms with 32-bit off_t (like Android).
https://hg.python.org/cpython/rev/6147a2c99db0
msg266139 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-23 12:42
> When OverflowError is raised? Shouldn't it be replaced with ValueError?

At least Python implementation of io.FileIO can raise OverflowError (from os.lseek()). Thus OverflowError should be kept.