Issue 10212: struct.unpack and cStringIO.StringIO don't support new buffer - Python tracker

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, benjamin.peterson, eric.araujo, kristjan.jonsson, lemburg, pitrou, python-dev, skrah, terry.reedy, tseaver
Priority: normal Keywords: needs review, patch

Created on 2010-10-27 10:36 by kristjan.jonsson, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
newbuffer.patch kristjan.jonsson, 2010-10-27 10:36 review
Messages (11)
msg119687 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2010-10-27 10:36
When doing socket IO, it is beneficial to use a bytearra() and then using sock.recv_into() to avoid moving data about.
However, many useful functions still don't accept new style buffers, such as the bytearray and memoryview.  In particular, the struct module cannot unpack from them, and the StringIO doesn't accept them as input.

The attached patch adds new-buffer support to the struct module and cStringIO.
msg119689 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-10-27 11:13
You can't add buffer protocol support to cStringIO in a bugfix release, since it would be a new feature.
msg119750 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2010-10-28 01:46
I disagree.  It's not a new feature.  We're merely completing an old feature (adding new-style buffers from 3.x to 2.7) that wasn't fully implemented. by the core.  The new buffer isn't accepted in a lot of places where you'd expect it to be.

The good alternative, of course, would be to add this to "trunk" but for some reason that is frowned upon.

See also issue 10211
msg119781 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-28 13:52
MAL’s viewpoint from msg119721:

“The memoryview object was added to simplify porting applications to Python3. If that backport is incomplete in the sense that the memoryview object is not compatible with the standard Python2 object for such memory views, then I'd consider that a bug.

Without compatibility to the buffer objects, there's no way to make other Python2 buffer interface compatible object compatible to memoryviews.

Alternatively, the Python2 memoryview object implementation could also accept objects with the old buffer interface, much like "s*" does.

Note that the patch needs to check the buffer flags - writing to such buffers is not allowed.”
msg119939 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-10-29 22:03
In the absence of doc references in this and #10211 that would *clearly* settle the bug vs. feature issue, it strikes me as a bit murky. So I am inclined to agree with MAL that failure to achieve the stated, documented purpose is a bug. I strongly suspect that if these issues had been filed during the 2.7 beta phase, a fix would have been deemed permissible without much controversy.
msg156389 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2012-03-20 09:14
The patch is still there.  Any new consensus?
msg156426 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-03-20 16:08
Stefan, since you have been working on Memoryview, any opinions?
msg184524 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2013-03-18 21:19
We don't seem to be getting any traction on this.  2.7.4 is to be cut next weekend so it is becoming more urgent :)
msg184549 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-18 23:09
For the backport policy question, since there seems to be disagreement, consider a post on pydev asking for discussion and if no quick consensus, a ruling from Guido or designee.
msg184562 - (view) Author: Tres Seaver (tseaver) * Date: 2013-03-19 00:07
Patch looks good to me.  I would call it a bugfix, not a feature.
msg184717 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-03-20 00:40
New changeset be4bec689de3 by Kristján Valur Jónsson in branch '2.7':
Issue #10212: Support new buffer interface for struct.unpack and
http://hg.python.org/cpython/rev/be4bec689de3