Robert Kruszewski created SPARK-57929:
-----------------------------------------
Summary: Support Apache Arrow View types in ArrowColumnVector
Key: SPARK-57929
URL: https://issues.apache.org/jira/browse/SPARK-57929
Project: Spark
Issue Type: Improvement
Components: Spark Core, SQL
Affects Versions: 4.1.2
Reporter: Robert Kruszewski
Apache Arrow have added View types in 1.4 of the spec. The arrow-java version
in use in spark already has support for them. Modern data systems might choose
to produce those arrow types instead of usual List/Utf8/Binary types since they
support random reads and writes without having to reallocate whole vector.
Spark should support reading those vectors in its columnar execution
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
