Re: pyarrow schema in protobuf

2019-02-14 Thread Wes McKinney
hi Ryan, see http://arrow.apache.org/docs/python/generated/pyarrow.Buffer.html#pyarrow.Buffer you can call to_pybytes() on the result of serialize() HTH Wes On Thu, Feb 14, 2019 at 4:24 PM Ryan White wrote: > > I've found the deserialize in pa.ipc.read_schema(). From schema.serialize, > I get

Re: pyarrow schema in protobuf

2019-02-14 Thread Ryan White
I've found the deserialize in pa.ipc.read_schema(). From schema.serialize, I get a pyarrow Buffer. Do I need to write this to a BufferOutputStream to get raw bytes (e.g. sink.getvalue())? In Flight, I see this is casted to std:string https://github.com/apache/arrow/blob/69d595ae4c61902b3f2778e536fc

pyarrow schema in protobuf

2019-02-14 Thread Ryan White
Hi, I'm using protocol buffers to retain metadata, and I would like to store the Arrow Schema in the protobuf as Arrow is doing in Flight. Looking at the Flight perf.proto, I can do the same and define a bytes field in my proto. From pyarrow, can I serialize/deserialize pyarrow.schema? I've only f