In Hadoop 1.x TaskAttemptContext is a class (for example,
https://hadoop.apache.org/docs/r1.2.1/api/org/apache/hadoop/mapred/TaskAttemptContext.html)
In Hadoop 2.x TaskAttemptContext is an interface
(https://hadoop.apache.org/docs/r2.4.0/api/org/apache/hadoop/mapreduce/TaskAttemptContext.html)
From
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_vd_cdh_package_tarball.html
it looks like
CDH 5.3.2 uses Hadoop 2.5.
Are you using any third party libraries that come in hadoop1 (default) vs.
hadoop2 versions like avro-mapred (see
https://issues.apache.org/jira/browse/SPARK-3039)?
If so make sure you include:
<dependency>
...
<classifier>hadoop2</classifier>
</dependency>
What version of Spark are you using? Are you using Avro? If so
SPARK-3039 is fixed in Spark 1.3.
Markus
On 03/05/2015 01:31 PM, ey-chih chow wrote:
Hi,
I am using CDH5.3.2 now for a Spark project. I got the following exception:
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
I used all the CDH5.3.2 jar files in my pom file to generate the application
jar file. What else I should do to fix the problem?
Ey-Chih Chow
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/IncompatibleClassChangeError-tp21934.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
