[
https://issues.apache.org/jira/browse/HIVE-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064191#comment-16064191
]
Colin Ma commented on HIVE-16969:
---------------------------------
[~csun], [~Ferd], can you help to review the patch? Thanks.
> Improvement performance of MapOperator for Parquet
> --------------------------------------------------
>
> Key: HIVE-16969
> URL: https://issues.apache.org/jira/browse/HIVE-16969
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 3.0.0
> Reporter: Colin Ma
> Assignee: Colin Ma
> Fix For: 3.0.0
>
> Attachments: HIVE-16969.001.patch
>
>
> For a table with many partition files,
> MapOperator.cloneConfsForNestedColPruning() will update the
> hive.io.file.readNestedColumn.paths many times. The larger value of
> hive.io.file.readNestedColumn.paths will cause the poor performance for
> ParquetHiveSerDe.processRawPrunedPaths().
> So, the unnecessary paths should be appended to
> hive.io.file.readNestedColumn.paths.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
