Gopal V created HIVE-12999:
------------------------------
Summary: Tez: Vertex creation is slowed down when NN throttles IPCs
Key: HIVE-12999
URL: https://issues.apache.org/jira/browse/HIVE-12999
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 1.2.0, 1.3.0, 2.0.0, 2.1.0
Reporter: Gopal V
Assignee: Gopal V
Tez vertex building has a decidedly slow path in the code, which is not related
to the DAG plan at all.
The total number of RPC calls is not related to the total number of operators,
due to a bug in the DagUtils inner loops.
{code}
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1877)
at
org.apache.hadoop.hive.ql.exec.Utilities.createTmpDirs(Utilities.java:3207)
at
org.apache.hadoop.hive.ql.exec.Utilities.createTmpDirs(Utilities.java:3170)
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:548)
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1151)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:388)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:175)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
