Hi Antoine, On Tue, Nov 17, 2020 at 2:34 AM Antoine Pitrou <anto...@python.org> wrote: > > Le 17/11/2020 à 03:34, Rares Vernica a écrit : > > > > I'm using an arrow::io::BufferReader and > > arrow::ipc::RecordBatchStreamReader to read an arrow::RecordBatch from a > > file. There is only one batc in the file so I do a single > > RecordBatchStreamReader::ReadNext call. I store the populated RecordBatch > > in memory for reuse (cache). The memory buffer wrapped by the BufferReader > > is reallocated. > > What do you mean with "reallocated"? As long as you keep a strong > reference to a RecordBatch (through shared_ptr), the buffers are kept > intact. This is an intended consequence of the Buffer design and the > pervasive use of shared_ptr.
I have something like this: std::unique_ptr<char[]> data; data = std::make_unique<char[]>(...); // populate data std::shared_ptr<arrow::io::BufferReader> bufferReader; std::shared_ptr<arrow::RecordBatchReader> batchReader; std::shared_ptr<arrow::RecordBatch> batch; bufferReader = std::make_shared<arrow::io::BufferReader>(data.get()); arrow::ipc::RecordBatchStreamReader::Open(bufferReader, &batchReader); // Arrow < 0.17.0 batchReader->ReadNext(&batch); data = std::make_unique<char[]>(...); // populate "data" Is "batch" still a valid RecordBatch after the "data" buffer has been relocated and repopulated? Thanks! Rares
