I would support removing Tensor and SparseTensors from IPC because FixedShapeTensorArray and VariableShapeTensorArray seem like a better way forward and can be made to provide the same functionality. Also securing IPC messages is quite difficult given complexity of tensor data structures especially once one factors in that every IPC implementation needs to think about this.
It would of course also be good to do some research downstream to see if we have downsream users who depend on Tensor via IPC. Rok On Wed, Jul 1, 2026 at 5:59 PM Antoine Pitrou <[email protected]> wrote: > > Hello, > > While the Arrow IPC format is primarily meant for RecordBatch transport, > it also has dedicated messages for transporting N-dimensional Tensor and > SparseTensor data. > > The Tensor message was added in 2017 > (https://github.com/apache/arrow/pull/435), while the SparseTensor > message was added in 2019 (https://github.com/apache/arrow/pull/3651). > > Both have not had any significant additions since 2020. Furthermore: > > * They are still marked experimental. > * They are not exercised by the the cross-implementation integration > test suite. > * Convenience IPC reading APIs are generally architected around > RecordBatch iteration, requiring more manual plumbing to read Tensor and > SparseTensor messages from an IPC stream. > * They are generally not reachable from Flight RPC APIs, or other > high-level APIs such the C++ Dataset API. > * A code search on GitHub reveals no apparent third-party usage of these > message types. > > Nowadays, Arrow has canonical extension types for fixed-shape and > variable-shaped tensors which allow passing (dense) N-dimensional > tensors as regular RecordBatch columns. We could also add canonical > extension types for the different kinds of sparse tensors, if there is > demand for it. > > Despite not seeing much usage, these message types carry their > maintenance baggage. Recently, we've had a number of security reports > around the lack of validation of untrusted Tensor and SparseTensor > messages in Arrow C++, requiring core developer intervention to analyze > and fix those issues. > > I would like to suggest that we start deprecating these messages types > in the IPC format, and that we recommend deprecating the corresponding > APIs in all major Arrow implementations. What do you think? > > Regards > > Antoine. > >
