On 11/21/14, 10:11 AM, peterm_second wrote:
Caused by: java.io.IOException: Previous writer likely failed to write
hdfs://hadoop-nn.mo-data.com:9000/tmp/hive/root/_tez_session_dir/a0087fb2-1430-43fa-b3e1-06644ab4961d/*.
Failing because I am unlikely to write too.
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:974)
at
I am a noob when it comes to tez and hive, has anyone seen this problem
? Any ideas what might be causing it.
I'll take a shot at the problem.
Somewhere in your configuration, there's the equivalent of a "*" in the
AUX jars or in an "ADD FILE" or in the tez.lib.uris, which we're
interpreting as-is instead of as a glob.
As far as I understand tez doesn't use the distributed cashe so it tries
to localize all the resources for each session. I have no idea what
process is holding those files and causing it to fail with this error.
I am using Hive 0.14.0 and tez 0.6
Actually, Tez tries to use distributed cache heavily (what's failing is
the load into dist-cache location) - we now have multiple levels of the
same concept, with re-localization in play during a session, if you do
an "ADD JAR".
This indirectly shows up when someone queries an Accumulo table or HBase
table immediately after a regular native ORC table query.
But unlike MR, it doesn't have a job.jar equivalent uploaded during job
submission. So if there is something like an invalid HDFS path (the
ending of "/*") it will fail when the session is spinning up.
Cheers,
Gopal
