gh-112536: Set up TSAN CI for free-threading by corona10 · Pull Request #116555 · python/cpython · GitHub
Skip to content

gh-112536: Set up TSAN CI for free-threading - #116555

Closed
corona10 wants to merge 19 commits into
python:mainfrom
corona10:gh-112536-tsan
Closed

gh-112536: Set up TSAN CI for free-threading#116555
corona10 wants to merge 19 commits into
python:mainfrom
corona10:gh-112536-tsan

Conversation

@corona10

@corona10 corona10 commented Mar 10, 2024

Copy link
Copy Markdown
Member

@corona10

This comment was marked as resolved.

@corona10
corona10 marked this pull request as draft March 10, 2024 05:36
@corona10

corona10 commented Mar 10, 2024

Copy link
Copy Markdown
Member Author

@corona10

Copy link
Copy Markdown
Member Author
checking for builtin __atomic_load_n and __atomic_store_n functions... no
configure: error: --disable-gil requires mimalloc memory allocator (--with-mimalloc).

Weired..

@corona10

corona10 commented Mar 10, 2024

Copy link
Copy Markdown
Member Author

I need to check on my Linux machine later.

./Include/cpython/pyatomic_gcc.h:531:3: warning: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Wtsan]
  531 | { __atomic_thread_fence(__ATOMIC_SEQ_CST); }

Comment on lines +485 to +488
- name: Set up GCC-10 for ASAN
uses: egor-tensin/setup-gcc@v1
with:
version: 11

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version says "11" but the step name says "GCC-10"

- name: Display build info
run: make pythoninfo
- name: Tests
run: ./python -m test --pgo -j4 # Reduce test scope

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll want a configuration specifically for tsan. Maybe a --tsan option? We'll only want to run tests that actually use threading.

Brett had a list of tests that fail with PYTHON_GIL=0 -- those are probably a good starting point:

swtaarrs@1fe9165#diff-590c02490d066378045dd48d62e86dc85d3fb4ad934b6a2b2b0b1112e59eaefb

Comment thread Objects/obmalloc.c
/***************************/

static int
_Py_NO_SANITIZE_THREAD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a better experience using suppression lists rather than marking functions as _Py_NO_SANITIZE_THREAD:

  1. Add a file supressions.txt somwhere (maybe Tools/tsan/supressions.txt?)
  2. Set the environment variable TSAN_OPTIONS="suppressions=<path_to_supressions.txt>"

See https://github.com/google/sanitizers/wiki/ThreadSanitizerSuppressions

with:
version: 11
- name: Configure CPython
run: ./configure --disable-gil --with-thread-sanitizer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want --with-pydebug?

@corona10

Copy link
Copy Markdown
Member Author

@corona10 corona10 closed this Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants