- Python Home
- About
- News
- Documentation
- Downloads
- Community
- Foundation
- Developer's Guide
- Issue Tracker
- Issues
- Summaries
- User
- Administration
- Help
Issue44850
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2021-08-06 10:42 by Antony.Lee, last changed 2022-04-11 14:59 by admin.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 27782 | open | Antony.Lee, 2021-08-16 15:59 | |
| Messages (4) | |||
|---|---|---|---|
| msg399066 - (view) | Author: Antony Lee (Antony.Lee) * | Date: 2021-08-06 10:42 | |
Currently, methodcaller is not faster than a plain lambda:
```
In [1]: class T:
...: a = 1
...: def f(self): pass
...:
In [2]: from operator import *
In [3]: %%timeit t = T(); mc = methodcaller("f")
...: mc(t)
...:
...:
83.1 ns ± 0.862 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [4]: %%timeit t = T(); mc = lambda x: x.f()
...: mc(t)
...:
...:
81.4 ns ± 0.0508 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
```
(on some machines, I find that it is even slower).
Compare with attrgetter, which *is* faster:
```
In [5]: %%timeit t = T(); ag = attrgetter("a")
...: ag(t)
...:
...:
33.7 ns ± 0.0407 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [6]: %%timeit t = T(); ag = lambda x: x.a
...: ag(t)
...:
...:
50.1 ns ± 0.057 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
```
Given that the operator module explicitly advertises itself as being "efficient"/"fast", it seems reasonable to try to optimize methodcaller. Looking at its C implementation, methodcaller currently uses PyObject_GetAttr followed by PyObject_Call; I wonder whether this can be optimized using a LOAD_METHOD-style approach to avoid the construction of the bound method (when applicable)?
|
|||
| msg399178 - (view) | Author: Dennis Sweeney (Dennis Sweeney) * ![]() |
Date: 2021-08-07 12:50 | |
Using _PyObject_GetMethod similarly to the way that LOAD_METHOD/CALL_METHOD does seems like a reasonable idea to me -- do you want to make a pull request? It would also be nice to see some microbenchmarks for the change once it's ready. |
|||
| msg399184 - (view) | Author: Dennis Sweeney (Dennis Sweeney) * ![]() |
Date: 2021-08-07 14:16 | |
For what it's worth, in my benchmarks on 3.11, methodcaller was already a bit faster than lambda:
#################### Builtin calls ####################
PS > .\python.bat -m pyperf timeit -s "from operator import methodcaller as mc" -s "reverse_it = mc('reverse')" -s "arr = []" "reverse_it(arr)"
Running Release|x64 interpreter...
.....................
Mean +- std dev: 84.3 ns +- 1.9 ns
PS >.\python.bat -m pyperf timeit -s "reverse_it = lambda x: x.reverse()" -s "arr = []" "reverse_it(arr)"
Running Release|x64 interpreter...
.....................
Mean +- std dev: 95.5 ns +- 2.9 ns
#################### Python calls ####################
PS > .\python.bat -m pyperf timeit -s "from operator import methodcaller as mc" -s "reverse_it = mc('reverse')" -s "class A: reverse = lambda self: None" -s "arr=A()" "reverse_it(arr)"
Running Release|x64 interpreter...
.....................
Mean +- std dev: 140 ns +- 4 ns
PS > .\python.bat -m pyperf timeit -s "reverse_it = lambda x: x.reverse()" -s "class A: reverse = lambda self: None" -s "arr=A()" "reverse_it(arr)"
Running Release|x64 interpreter...
.....................
Mean +- std dev: 159 ns +- 4 ns
|
|||
| msg399903 - (view) | Author: Mark Shannon (Mark.Shannon) * ![]() |
Date: 2021-08-19 10:07 | |
If the problem is that methodcaller is not faster than a lambda, then why not use a lambda? Lambdas are just Python functions and calling them will get faster. We aren't going to spend time optimizing calls to methodcaller, so you might as well use a lambda. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:48 | admin | set | github: 89013 |
| 2021-08-19 10:07:47 | Mark.Shannon | set | nosy:
+ Mark.Shannon messages: + msg399903 |
| 2021-08-16 15:59:26 | Antony.Lee | set | keywords:
+ patch stage: patch review pull_requests: + pull_request26252 |
| 2021-08-07 14:16:03 | Dennis Sweeney | set | messages: + msg399184 |
| 2021-08-07 12:50:43 | Dennis Sweeney | set | versions:
+ Python 3.11 nosy: + Dennis Sweeney messages: + msg399178 type: performance |
| 2021-08-06 10:42:14 | Antony.Lee | create | |


