"Found class org.apache.hadoop.mapreduce.TaskAttemptContext, but
interface was expected" is the classic error meaning "you compiled
against Hadoop 1, but are running against Hadoop 2"

I think you need to override the hadoop-client artifact that Spark
depends on to be a Hadoop 2.x version.

On Tue, Jun 3, 2014 at 5:23 PM, toivoa <toivo....@gmail.com> wrote:
> Hi
>
> Set up project under Eclipse using Maven:
>
>         <dependency>
>                 <groupId>org.apache.spark</groupId>
>                 <artifactId>spark-core_2.10</artifactId>
>                 <version>1.0.0</version>
>         </dependency>
>
> Simple example fails:
>
>   def main(args: Array[String]): Unit = {
>
>     val conf = new SparkConf()
>              .setMaster("local")
>              .setAppName("CountingSheep")
>              .set("spark.executor.memory", "1g")
>
>     val sc = new SparkContext(conf)
>
>     val indir = "src/main/resources/testdata"
>     val files = sc.wholeTextFiles(indir, 10)
>     for( pair <- files)
>       println(pair._1 + " = " + pair._2)
>
>
> 14/06/03 19:20:34 ERROR executor.Executor: Exception in task ID 0
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
>         at
> org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.initNextRecordReader(CombineFileRecordReader.java:164)
>         at
> org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.<init>(CombineFileRecordReader.java:126)
>         at
> org.apache.spark.input.WholeTextFileInputFormat.createRecordReader(WholeTextFileInputFormat.scala:44)
>         at 
> org.apache.spark.rdd.NewHadoopRDD$$anon$1.<init>(NewHadoopRDD.scala:111)
>         at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:99)
>         at org.apache.spark.rdd.NewHadoopRDD.compute(NewHadoopRDD.scala:61)
>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
>         at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
>         at org.apache.spark.scheduler.Task.run(Task.scala:51)
>         at 
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:187)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>         at
> org.apache.hadoop.mapreduce.lib.input.CombineFileRecordReader.initNextRecordReader(CombineFileRecordReader.java:155)
>         ... 13 more
> Caused by: java.lang.IncompatibleClassChangeError: Found class
> org.apache.hadoop.mapreduce.TaskAttemptContext, but interface was expected
>         at
> org.apache.spark.input.WholeTextFileRecordReader.<init>(WholeTextFileRecordReader.scala:40)
>         ... 18 more
>
> Any idea?
>
> thanks
> toivo
>
>
>
>
> --
> View this message in context: 
> http://apache-spark-user-list.1001560.n3.nabble.com/wholeTextFiles-java-lang-IncompatibleClassChangeError-Found-class-org-apache-hadoop-mapreduce-TaskAtd-tp6818.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.

Reply via email to