bpo-43244: test_peg_generator defines _Py_TEST_PEGEN macro by vstinner · Pull Request #24912 · python/cpython · GitHub
Skip to content

bpo-43244: test_peg_generator defines _Py_TEST_PEGEN macro#24912

Merged
vstinner merged 1 commit intopython:masterfrom
vstinner:test_pegen
Mar 18, 2021
Merged

bpo-43244: test_peg_generator defines _Py_TEST_PEGEN macro#24912
vstinner merged 1 commit intopython:masterfrom
vstinner:test_pegen

Conversation

@vstinner
Copy link
Copy Markdown
Member

@vstinner vstinner commented Mar 18, 2021

test_peg_generator now defines the _Py_TEST_PEGEN macro to specialize
Python-ast.c, rather than relying on Py_BUILD_CORE.

Python-ast.c can now also includes pycore_ast_state.h when built by
test_peg_generator. The test now also builds it with the
Py_BUILD_CORE macro defined.

https://bugs.python.org/issue43244

@pablogsal
Copy link
Copy Markdown
Member

test_peg_generator now defines _Py_TEST_PEGEN macro when building C
code to not call PyAST_Validate() in Parser/pegen.c. Moreover, it
defines Py_BUILD_CORE_MODULE macro to get access to the internal
C API.

Remove "global_ast_state" from Python-ast.c when it's built by
test_peg_generator: get the AST state from the current interpreter.
@vstinner
Copy link
Copy Markdown
Member Author

I am not super happy with this approach. I don't think we should introduce test-dependent macros in the actual codebase. In any case, if there isn't a simpler solution...then we can go ahead I suppose (I don't want to block only on this).

The first version of PR didn't introduce code specific to test_peg_generator, I already added in commit 5cf4782. It only renamed the #ifdef.

My problem was SystemError: impossible module node error. In fact, the error comes from Parser/pegen.c which only calls PyAST_Validate() if Py_BUILD_CORE macro is defined.

I rewrote my PR to restrict the #ifdef to a single line in Parser/pegen.c. It sounds less surprising that test_peg_generator is related to Parser/pegen.c rather than Python/Python-ast.c.

My first approach avoided Py_BUILD_CORE by any mean to work around the SystemError: impossible module node error.

@vstinner vstinner merged commit e0bf70d into python:master Mar 18, 2021
@vstinner vstinner deleted the test_pegen branch March 18, 2021 01:46
@bedevere-bot
Copy link
Copy Markdown

jab added a commit to jab/cpython that referenced this pull request Mar 20, 2021
* master: (129 commits)
  bpo-43452: Micro-optimizations to PyType_Lookup (pythonGH-24804)
  bpo-43517: Fix false positive in detection of circular imports (python#24895)
  bpo-43494: Make some minor changes to lnotab notes (pythonGH-24861)
  Mention that code.co_lnotab is deprecated in what's new for 3.10. (python#24902)
  bpo-43244: Remove symtable.h header file (pythonGH-24910)
  bpo-43466: Add --with-openssl-rpath configure option (pythonGH-24820)
  Fix a typo in c-analyzer (pythonGH-24468)
  bpo-41561: Add workaround for Ubuntu's custom security level (pythonGH-24915)
  bpo-43521: Allow ast.unparse with empty sets and NaN (pythonGH-24897)
  bpo-43244: Remove the PyAST_Validate() function (pythonGH-24911)
  bpo-43541: Fix PyEval_EvalCodeEx() regression (pythonGH-24918)
  bpo-43244: Fix test_peg_generators on Windows (pythonGH-24913)
  bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (pythonGH-18011)
  bpo-43244: Fix test_peg_generator for PyAST_Validate() (pythonGH-24912)
  bpo-42128: Add 'missing :' syntax error message to match statements (pythonGH-24733)
  bpo-43244: Add pycore_ast.h header file (pythonGH-24908)
  bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (pythonGH-24907)
  Remove unnecessary imports in the grammar parser (pythonGH-24904)
  bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pythonGH-24843)
  Add PEP 626 to what's new in 3.10. (python#24892)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants