[
https://issues.apache.org/jira/browse/HIVE-7591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao updated HIVE-7591:
-----------------------
Attachment: HIVE-7591-spark.patch
A quick fix for this issue. I think a better solution might be throwing
exception
when seeing an unexpected work type, but at the moment it involves propagating
the exception across multiple calls, which I'm not sure if it's the right thing
to do.
> GenMapRedUtils::addStatsTask only assumes either MapredWork or TezWork
> ----------------------------------------------------------------------
>
> Key: HIVE-7591
> URL: https://issues.apache.org/jira/browse/HIVE-7591
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Chao
> Attachments: HIVE-7591-spark.patch
>
>
> When running queries, I got exception like this:
> FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.SparkWork cannot be
> cast to org.apache.hadoop.hive.ql.plan.TezWork
> 14/07/31 15:08:53 ERROR ql.Driver: FAILED: ClassCastException
> org.apache.hadoop.hive.ql.plan.SparkWork cannot be cast to
> org.apache.hadoop.hive.ql.plan.TezWork
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.SparkWork cannot
> be cast to org.apache.hadoop.hive.ql.plan.TezWork
> at
> org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.addStatsTask(GenMapRedUtils.java:1419)
> at
> org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.isMergeRequired(GenMapRedUtils.java:1645)
> at
> org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.processFileSink(GenSparkUtils.java:313)
> at
> org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.generateTaskTree(SparkCompiler.java:180)
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:199)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9514)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:207)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:207)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:413)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:984)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1049)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:265)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:217)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:427)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:800)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Apparently, GenMapRedUtils::addStatsTask only assumes either MapredWork or
> TezWork, and since we are introducing SparkWork, this need to be fixed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
