[
https://issues.apache.org/jira/browse/HIVE-27804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikram Ahuja updated HIVE-27804:
--------------------------------
Description: Hive.getPartitions() methods returns partition list along with
auth info in one HMS call. These calls when made on wide tables(> 2000 columns)
with very large number of partitions(100,000+) can cause memory related issues
when the data is being transferred from HMS to HS2 using Thrift calls. These
APIs can be optimised by using PartitionIterable implementation where the
partition list if fetched in batched of a smaller size rather than one huge
call. (was: Hive.getPartitions() methods returns partition list along with
auth info in one . These calls when made on table with )
> Implement batching in getPartition calls which returns auth info
> ----------------------------------------------------------------
>
> Key: HIVE-27804
> URL: https://issues.apache.org/jira/browse/HIVE-27804
> Project: Hive
> Issue Type: Bug
> Reporter: Vikram Ahuja
> Assignee: Vikram Ahuja
> Priority: Major
>
> Hive.getPartitions() methods returns partition list along with auth info in
> one HMS call. These calls when made on wide tables(> 2000 columns) with very
> large number of partitions(100,000+) can cause memory related issues when the
> data is being transferred from HMS to HS2 using Thrift calls. These APIs can
> be optimised by using PartitionIterable implementation where the partition
> list if fetched in batched of a smaller size rather than one huge call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
