[
https://issues.apache.org/jira/browse/HIVE-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273792#comment-14273792
]
Jimmy Xiang commented on HIVE-8853:
-----------------------------------
Looked into it and found vectorization uses Map/Reduce work cache a lot. I
re-opened HIVE-9135.
> Make vectorization work with Spark [Spark Branch]
> -------------------------------------------------
>
> Key: HIVE-8853
> URL: https://issues.apache.org/jira/browse/HIVE-8853
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Xuefu Zhang
> Assignee: Jimmy Xiang
>
> In Hive to make vectorization work, the reader needs to be also vectorized,
> which means that the reader can read a chunk of rows (or a list of column
> chunks) instead of one row at a time. However, we use Spark RDD for reading,
> which again utilized the underlying inputformat to read. Subsequent
> processing also needs to hapen in batches. We need to make sure that
> vectorizatoin is working as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
