Stefan Krah <stefan-use...@bytereef.org> added the comment:
I thought it might be productive to switch to documentation/test driven
development for PEP-3118 in general. So I updated the documentation,
trying to spell out the responsibilities of both exporter and consumer
as clearly as possible.
In order to have a PEP-3118 reference implementation, I wrote
Modules/_testbuffer.c and Lib/test/test_buffer.py. The test module
contains an ndarray object (independent from NumPy's ndarray) with
these features:
o Full base object capabilities, including responding to flag
specific requests.
o Full re-exporter capability: The object obtains a buffer from
another exporter and poses as a base object.
o Optional capability to change layout while buffers are exported.
o Full support for arbitrary format strings using the struct
module.
o Fortran style arrays.
o Arbitrary multidimensional structures, including offsets and
negative strides.
o Support for converting arrays to suboffset representations.
o Support for multidimensional indexing, slicing and tolist().
o Optional support for testing against NumPy.
In memoryobject.c I only fixed the buffer release issues that came up.
Before proceeding with allocating private arrays for each memoryview,
it would be great to have agreement on the revised documentation and a
couple of other issues.
Documentation
-------------
I'll highlight some changes here:
1) view->obj: Must be a new reference to the provider if the buffer
is obtained via getbuffer(), otherwise NULL. In case of failure
the field MUST be set to NULL (was: undefined).
So, logically this should be seen the same way as returning
a new reference from a standard Python function.
2) view->buf: This can (and _does_ for Numpy arrays) point to
any location in the underlying memory block. If a consumer
doesn't request one of the simple or contiguous buffers,
all bets are off.
3) view->len: Make it clear that the PEP defines it as
product(shape) * itemsize.
4) Ownership for shape, strides, internal: read-only for the
consumer.
5) Flag explanations: The new section puts emphasis on which fields
a consumer can expect in response to a buffer request, starting
with the fields that will always be set.
6) Rule for writable/read-only requests. Only raise if the
buffer is read-only and the request is 'writable'. This
seems to be the most practical solution.
7) Add NumPy-style and PIL-style sections to make clear what
a consumer must be able to handle if complicated structures
are requested (thanks Pauli Virtanen for clarifying what
valid NumPy arrays may look like).
What I would like to spell out clearly:
8) A PEP-3118 compliant provider MUST always be able to respond
to a PyBUF_FULL_RO request (i.e. fill in shape AND strides).
ctypes doesn't currently do that, but could be fixed easily.
This is easy to implement for the exporter (see how
PyBuffer_FillInfo() does it in a few lines).
Memoryview
----------
8) Add PyMemoryView_FromBytes(). This could potentially replace
PyMemoryView_FromBuffer().
9) I've come to think that memoryviews should only be created
from PyBUF_FULL_RO requests (this also automatically succeeds
if an object is writable, see above).
One reason is that we don't constantly have to check for
the presence of shape and strides and format (unless ndim = 0).
Currently it is possible to create any kind of view via
PyMemoryView_FromBuffer(). It would be possible to deprecate
it or make it a rule that the buffer argument must have full
information.
10) NumPy has a limit of ndim = 64. It would be possible to
use that limit as well and make shape, strides and
suboffsets static arrays of size 64. Perhaps this is a bit
wasteful, it depends on how many views are typically
created.
11) test_buffer.py contains an example (see: test_memoryview_release())
that Antoine's test case will not work if a re-exporter is
involved. Should we leave that or fix it, too?
My apologies for the long list, but it'll be easier to proceed if
some rules are written in stone. :)
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue10181>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
