Issue 40458: test_bad_getattr crashes on APPX test - Python tracker

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: lukasz.langa, miss-islington, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2020-04-30 23:10 by steve.dower, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19845 merged steve.dower, 2020-05-01 20:00
PR 19941 merged miss-islington, 2020-05-05 17:46
Messages (8)
msg367803 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-04-30 23:10
The Windows CI machines on Azure Pipelines run additional tests to check an "installed" layout and with the UWP entry point that's used for the Windows Store package.

These tests have been failing intermittently (though regularly) with a stack overflow crash in the PyPickler tests.

Example: https://dev.azure.com/Python/cpython/_build/results?buildId=62055&view=results

test_attribute_name_interning (test.test_pickle.PyPicklerTests) ... ok
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
  File "D:\a\1\b\layout-appx-amd64\lib\test\pickletester.py", line 3085 in __getattr__
...

I assume this is due to having more code on the start at the start, and so the recursion limit isn't low enough. But it might also be worth checking this particular case to see whether there is unnecessary data being kept on the stack (e.g. in local C variables).

The crash occurs in both 3.8 and master, but not 3.7.
msg367880 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-01 20:01
Turns out the stack reservation was different, which is why it was crashing.

As part of diagnosing it, I added the recursion count to faulthandler's output on Windows, but couldn't see if/where to do it for other platforms - Victor, any suggestions?
msg367881 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-01 20:02
Also, it was really test_bad_getattr that was crashing.
msg367888 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-01 21:26
> Also, it was really test_bad_getattr that was crashing.

Ah, an old friend. It seems like an issue with the maximum stack size. Here are my notes:
https://pythondev.readthedocs.io/unstable_tests.html#unlimited-recursion

Either reduce Python maximum stack depth, or increase the C stack size.
msg368060 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-04 15:50
Yeah, I already got that part. If you check the PR, I added some better diagnostics to faulthandler for this case, but I don't see where I can add it for other platforms?
msg368185 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-05 17:45
New changeset ac4bf424119d1300f57929120968e216a85d3a25 by Steve Dower in branch 'master':
bpo-40458: Increase reserved stack space to prevent overflow crash on Windows (GH-19845)
https://github.com/python/cpython/commit/ac4bf424119d1300f57929120968e216a85d3a25
msg368186 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-05-05 17:48
Merged without the improved diagnostics. I might add that later if I get time.

Łukasz - this is a fairly trivial crash fix that'd be nice (and safe) to pull into 3.8.3, but not critical. Your call.
msg368189 - (view) Author: miss-islington (miss-islington) Date: 2020-05-05 18:03
New changeset a6a116c1b964b3d1fdff0f533861ed2a2227de1f by Miss Islington (bot) in branch '3.8':
bpo-40458: Increase reserved stack space to prevent overflow crash on Windows (GH-19845)
https://github.com/python/cpython/commit/a6a116c1b964b3d1fdff0f533861ed2a2227de1f