The test are failing to download openssl for some reason... :S

Merging upstream master would probably solve the problem.

Merging upstream master would probably solve the problem.

Hummm, merging upstream master in this PR creates a mess. Do you mean merging upstream/master in the pegen branch?

Merging upstream master would probably solve the problem.

Hummm, merging upstream master in this PR creates a mess. Do you mean merging upstream/master in the pegen branch?

Yes. And then merging pegen into this branch. That's how I do it.

Sorry for being a bit abrupt amd not explaining things carefully. I'm on my phone.

Sorry for being a bit abrupt amd not explaining things carefully. I'm on my phone.

No problem, I know that feeling very well :)

Seems that we need to update pegen to account for python#19283

What about del_targets? Does something like del <'a.a'*20> parse normally?

What about del_targets? Does something like del <'a.a'*20> parse normally?

I tried with:

del a.a.a.a.a......a.a (more than 4000) and seems to parse fine. Could you double check?

What about del_targets? Does something like del <'a.a'*20> parse normally?

I tried and something like f"del {'(' * nesting_depth}x,{'),' * nesting_depth}" also has exponential time, so we need to memorize del_targets as well.

Edit: PR updated

Edit: I forgot something. Before we merge this, have you measured what the memory cost is for these changes?

It increased 0.62% parsing xxl.txt and 0.83% parsing ((((((((((((((((((((1)))))))))))))))))))).

Checked also with the largest python file in the standard library (Lib/test/test_socket.py) and it increased the memory 1.58%.

Are you ok with these results for landing this? :)

How does it compare to the current parser now? Still a little bit ahead?

How does it compare to the current parser now? Still a little bit ahead?

The speed seems untouched (maybe a bit faster due to the extra memoization) if you parse something that relies heavily on target, del_target or star_targer . It would be good if you could confirm these findings on your side when possible

How does it compare to the current parser now? Still a little bit ahead?

The speed seems untouched (maybe a bit faster due to the extra memoization) if you parse something that relies heavily on target, del_target or star_targer . It would be good if you could confirm these findings on your side when possible

Of course, I'm working on a bug-fix for #38 and I'll verify everything right afterwards. Shouldn't be long.

Oops! Sorry, accidentally pushed the wrong button.

Verified everything. My measurements are very similar to those by @pablogsal. Everything that was too difficult to parse before, can be parsed smoothly now. So let's land this.

I confirmed the quadratic behavior as well, and the fix works. Land it.

Any chance of also committing the script you used to plot the times? (In a separate PR if you need to clean it up first.)

I will make another PR once I clean it up a bit :)

Thanks for the reviews and the confirmation of the problem!