Updated token + weakref libraries + weakref test - v3.13.11 by terryluan12 · Pull Request #6633 · RustPython/RustPython · GitHub
Skip to content

Updated token + weakref libraries + weakref test - v3.13.11 - #6633

Merged
youknowone merged 6 commits into
RustPython:mainfrom
terryluan12:update_token_weakref
Jan 4, 2026
Merged

youknowone merged 6 commits into
RustPython:mainfrom
terryluan12:update_token_weakref

Conversation

@terryluan12

Copy link
Copy Markdown
Contributor

No description provided.

@coderabbitai

coderabbitai Bot commented Jan 3, 2026

Copy link
Copy Markdown
Contributor

@terryluan12 terryluan12 changed the title Updated token + weakref libraries Updated token + weakref libraries + weakref test Jan 3, 2026
@terryluan12

Copy link
Copy Markdown
Contributor Author

I could not find a token test file

@youknowone

Copy link
Copy Markdown
Member

by searching import token, test_tokenize seems to test it

@terryluan12

terryluan12 commented Jan 3, 2026

Copy link
Copy Markdown
Contributor Author

by searching import token, test_tokenize seems to test it

I saw that, & assumed that it would test out the tokenize library, but as I'm writing it out I'm realizing I should probably jointly update the two libraries. Will add it, thanks!

@terryluan12 terryluan12 changed the title Updated token + weakref libraries + weakref test Updated token + weakref libraries + weakref test - v3.13.11 Jan 3, 2026
@terryluan12

Copy link
Copy Markdown
Contributor Author

So it seems a rewrite of tokenize has been done. the _tokenize function was moved to a precompiled library, and logic has been changed, so it's not as simple as keeping the older _tokenize version. Is it okay to merge, without making any tokenization migrations?

@terryluan12

Copy link
Copy Markdown
Contributor Author

@youknowone
youknowone force-pushed the update_token_weakref branch from cd1bf05 to c961277 Compare January 4, 2026 00:27
Comment thread Lib/test/test_ftplib.py Outdated

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@youknowone
youknowone merged commit 68d65a7 into RustPython:main Jan 4, 2026
12 of 13 checks passed
terryluan12 added a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026
…on#6633)

* Updated the token library

* Updated the weakref library

* Annotated the failing weakref test
terryluan12 added a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026
…on#6633)

* Updated the token library

* Updated the weakref library

* Annotated the failing weakref test
terryluan12 added a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026
…on#6633)

* Updated the token library

* Updated the weakref library

* Annotated the failing weakref test
@terryluan12
terryluan12 deleted the update_token_weakref branch January 5, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants