FilesExpand file tree

 docs

/

tokenize_test.py

Copy path

Blame

More file actions

Blame

More file actions

Latest commit

 

History

History

History

7 lines (6 loc) · 189 Bytes

 docs

/

tokenize_test.py

Copy path

Top

File metadata and controls

• Code
• Blame

7 lines (6 loc) · 189 Bytes

Raw

Copy raw file

Download raw file

Open symbols panel

Edit and raw actions

1

2

3

4

5

6

7

# stdlib

import tokenize

from io import StringIO

from pprint import pprint

tokens = tokenize.generate_tokens(StringIO('foo = "abcdefg" # doc: a docstring').readline)

pprint(list(tokens))