[
https://issues.apache.org/jira/browse/HIVE-11675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169715#comment-15169715
]
Sergey Shelukhin commented on HIVE-11675:
-----------------------------------------
[~prasanth_j] can you review? The new files are mostly moved/renamed code, with
some small changes. Every time this patch sits for a while it gets a lot of
conflicts.
> make use of file footer PPD API in ETL strategy or separate strategy
> --------------------------------------------------------------------
>
> Key: HIVE-11675
> URL: https://issues.apache.org/jira/browse/HIVE-11675
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-11675.01.patch, HIVE-11675.02.patch,
> HIVE-11675.03.patch, HIVE-11675.04.patch, HIVE-11675.05.patch,
> HIVE-11675.06.patch, HIVE-11675.07.patch, HIVE-11675.patch
>
>
> Need to take a look at the best flow. It won't be much different if we do
> filtering metastore call for each partition. So perhaps we'd need the custom
> sync point/batching after all.
> Or we can make it opportunistic and not fetch any footers unless it can be
> pushed down to metastore or fetched from local cache, that way the only slow
> threaded op is directory listings
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
