[
https://issues.apache.org/jira/browse/HIVE-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030131#comment-14030131
]
Mithun Radhakrishnan commented on HIVE-7195:
--------------------------------------------
[~sershe]: I'm sorry, I've not found the time to port my patch to 13 and raise
a JIRA. My work was primarily in the PartitionPruner code. It was to ensure
that {{listPartitions(db, table, -1)}} isn't called (during plan optimization),
if the call is a metadata-only query. I can post the 12-patch in a JIRA,
whatever that's worth.
Incidentally, I've raised HIVE-7223 to discuss the idea of using
{{PartitionSpecs}}. [~alangates] suggested that we explore if a PartitionSpec
abstract could also represent lighter Partition-groups that share commonality
(StorageDescs, etc.). Still thinking that through. (If only Thrift supported
polymorphism. :])
> Improve Metastore performance
> -----------------------------
>
> Key: HIVE-7195
> URL: https://issues.apache.org/jira/browse/HIVE-7195
> Project: Hive
> Issue Type: Improvement
> Reporter: Brock Noland
> Priority: Critical
>
> Even with direct SQL, which significantly improves MS performance, some
> operations take a considerable amount of time, when there are many partitions
> on table. Specifically I believe the issue:
> * When a client gets all partitions we do not send them an iterator, we
> create a collection of all data and then pass the object over the network in
> total
> * Operations which require looking up data on the NN can still be slow since
> there is no cache of information and it's done in a serial fashion
> * Perhaps a tangent, but our client timeout is quite dumb. The client will
> timeout and the server has no idea the client is gone. We should use
> deadlines, i.e. pass the timeout to the server so it can calculate that the
> client has expired.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
