gh-153568: Stop re-walking memo lists when growing left-recursive rul… · python/cpython@ac86e12 · GitHub
Skip to content

Commit ac86e12

Browse files
authored
gh-153568: Stop re-walking memo lists when growing left-recursive rules (#153574)
The left-recursion driver now keeps a direct reference to its memo entry and updates it in place on every iteration.
1 parent e66bec0 commit ac86e12

5 files changed

Lines changed: 91 additions & 53 deletions

File tree

Lines changed: 2 additions & 0 deletions

Parser/parser.c

Lines changed: 70 additions & 50 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Parser/pegen.c

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,17 @@ _PyPegen_insert_memo(Parser *p, int mark, int type, void *node)
102102
return 0;
103103
}
104104

105+
// Like _PyPegen_insert_memo(), but returns the inserted Memo so callers
106+
// can update it in place without re-walking the token's memo list.
107+
Memo *
108+
_PyPegen_insert_memo_direct(Parser *p, int mark, int type)
109+
{
110+
if (_PyPegen_insert_memo(p, mark, type, NULL) < 0) {
111+
return NULL;
112+
}
113+
return p->tokens[mark]->memo;
114+
}
115+
105116
// Like _PyPegen_insert_memo(), but updates an existing node if found.
106117
int
107118
_PyPegen_update_memo(Parser *p, int mark, int type, void *node)

Parser/pegen.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,7 @@ PyObject *_PyPegen_get_memo_statistics(void);
148148

149149
int _PyPegen_insert_memo(Parser *p, int mark, int type, void *node);
150150
int _PyPegen_update_memo(Parser *p, int mark, int type, void *node);
151+
Memo *_PyPegen_insert_memo_direct(Parser *p, int mark, int type);
151152
int _PyPegen_is_memoized(Parser *p, int type, void *pres);
152153

153154
int _PyPegen_lookahead(int, void *(func)(Parser *), Parser *);

Tools/peg_generator/pegen/c_generator.py

Lines changed: 7 additions & 3 deletions

0 commit comments

Comments
 (0)