Mustafa İman created HIVE-24415:
-----------------------------------
Summary: HiveSplitGenerator blocks Tez dispatcher
Key: HIVE-24415
URL: https://issues.apache.org/jira/browse/HIVE-24415
Project: Hive
Issue Type: Improvement
Reporter: Mustafa İman
Assignee: Mustafa İman
HiveSplitGenerator does a lot of heavyweight operations in its constructor.
These operations block AsyncDispatcher in Tez
[https://github.com/apache/tez/blob/989d286d09cac7c4e4c5a0e06dd75ea5a6f15478/tez-dag/src/main/java/org/apache/tez/dag/app/dag/RootInputInitializerManager.java#L141]
. We should move bulk of initialization out of constructor.
The only reason of setting up everything in constructor is
DynamicPartitionPruner. We can buffer incoming events in HiveSplitGenerator
until dynamic partition pruner is initialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
