Wes McKinney created ARROW-1783: ----------------------------------- Summary: [Python] Convert SerializedPyObject to/from sequence of component buffers with minimal memory allocation / copying Key: ARROW-1783 URL: https://issues.apache.org/jira/browse/ARROW-1783 Project: Apache Arrow Issue Type: New Feature Components: Python Reporter: Wes McKinney Fix For: 0.8.0
See discussion on Dask org: https://github.com/dask/distributed/pull/931 It would be valuable for downstream users to compute the serialized payload as a sequence of memoryview-compatible objects without having to allocate new memory on write. This means that the component tensor messages must have their metadata and bodies in separate buffers. This will require a bit of work internally reassemble the object from a collection of {{pyarrow.Buffer}} objects see also ARROW-1509 -- This message was sent by Atlassian JIRA (v6.4.14#64029)