Fix stack overflow segfault on debug builds for Python threads by JamesClarke7283 · Pull Request #7986 · RustPython/RustPython · GitHub
Skip to content

Fix stack overflow segfault on debug builds for Python threads#7986

Open
JamesClarke7283 wants to merge 5 commits into
RustPython:mainfrom
JamesClarke7283:fix-7941-thread-stack-overflow
Open

Fix stack overflow segfault on debug builds for Python threads#7986
JamesClarke7283 wants to merge 5 commits into
RustPython:mainfrom
JamesClarke7283:fix-7941-thread-stack-overflow

Conversation

@JamesClarke7283

@JamesClarke7283 JamesClarke7283 commented May 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Spawned Python threads were getting Rust's std::thread::Builder default stack of 2 MB, which is too small for the call chains the Python stdlib runs on helper threads in debug builds (where Rust stack frames are substantially larger). CPython on glibc Linux relies on pthread's ~8 MB default instead — a 4× difference that explains why this hits us but not CPython.
  • Apply 8 MB in apply_thread_stack_size whenever the user hasn't explicitly set a value via threading.stack_size(N). This matches CPython's effective default while keeping the Python API contract exact: threading.stack_size() still returns 0 ("platform default"), and explicit overrides still take precedence.
  • Single-file change in crates/vm/src/stdlib/_thread.rs.

Closes #7941

Test plan

  • cargo build succeeds.
  • Original reproducer passes on a debug build: cargo run -- -m test.test_ssl ThreadedTests.test_socketserverRan 1 test in 21.59s, OK (was segfaulting on main).
  • Python API contract preserved: threading.stack_size() returns 0 by default; setting to 1 MiB then reading back returns 1048576; previous value reported correctly.
  • Basic threaded workload (5 threads doing work + join) still completes normally.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Made thread stack sizing more predictable in debug builds: when no stack size is configured, debug builds now use a consistent explicit fallback to avoid platform-dependent variance. Configured stack sizes are applied to newly spawned threads. Release builds continue to use the platform default when no stack size is set.

Rust's std::thread::Builder defaults to a 2 MB stack when no size is
set, which is too small for the call chains the Python stdlib runs on
helper threads in debug builds (e.g. test.test_ssl's threaded server).
CPython on glibc Linux relies on pthread's ~8 MB default instead.

Apply 8 MB as the default in apply_thread_stack_size when the user has
not explicitly called threading.stack_size(N). This matches CPython's
effective default across builds while preserving the existing API:
threading.stack_size() still returns 0 by default ("platform default"),
and explicit user values still take precedence.

Fixes RustPython#7941

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

JamesClarke7283 and others added 3 commits May 27, 2026 09:47
Satisfy docstring coverage check on the PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit applied an 8 MB default unconditionally, which
slowed down multiprocessing tests on release: many forked children
each spawning many threads with oversized virtual stack mappings
caused the flaky MP CI step to time out (60 min, vs ~9 min on main).

Issue RustPython#7941 only manifested in debug builds — release builds were
already fine on Rust's 2 MB std default. Restrict the 8 MB override
to `#[cfg(debug_assertions)]` so release behavior is unchanged from
before the fix, while the original SSL segfault on debug stays fixed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@youknowone

Copy link
Copy Markdown
Member

please also add failing test before patch

@JamesClarke7283 JamesClarke7283 marked this pull request as draft June 3, 2026 13:59
@JamesClarke7283 JamesClarke7283 marked this pull request as ready for review June 3, 2026 17:02
@JamesClarke7283

This comment was marked as outdated.

@JamesClarke7283

Copy link
Copy Markdown
Contributor Author

Brings the branch up to date with main so CI job names match the
required status checks (the windows snippets/cpython job was renamed
by RustPython#8004 when its matrix skips were cleared). Also picks up RustPython#8018's
_thread.rs stack-margin updates, which are orthogonal to the
apply_thread_stack_size fix in this PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Some python tests are crashing with segmentation fault when running on debug build

2 participants