Re: Spark error NoClassDefFoundError: org/apache/hadoop/mapred/InputSplit

2015-03-23 Thread , Roy
now job started but it stuck at some different level 15/03/23 14:09:00 INFO BlockManagerInfo: Added rdd_10_0 on disk on ip-10-0-3-171.ec2.internal:58704 (size: 138.8 MB) so i checked yarn logs on ip-10-0-3-171.ec2.internal but I dint see any errors ? anyone know whats going on here ? *Thanks.

Re: Spark error NoClassDefFoundError: org/apache/hadoop/mapred/InputSplit

2015-03-23 Thread Marcelo Vanzin
This feels very CDH-specific (or even CM-specific), I'd suggest following up on cdh-u...@cloudera.org instead. On Mon, Mar 23, 2015 at 8:10 AM, , Roy wrote: > 15/03/23 11:06:49 WARN TaskSetManager: Lost task 9.0 in stage 0.0 (TID 7, > hdp003.dev.xyz.com): java.lang.NoClassDefFoundError: > org/apa

Re: Spark error NoClassDefFoundError: org/apache/hadoop/mapred/InputSplit

2015-03-23 Thread Ted Yu
InputSplit is in hadoop-mapreduce-client-core jar Please check that the jar is in your classpath. Cheers On Mon, Mar 23, 2015 at 8:10 AM, , Roy wrote: > Hi, > > > I am using CDH 5.3.2 packages installation through Cloudera Manager 5.3.2 > > I am trying to run one spark job with following com

Spark error NoClassDefFoundError: org/apache/hadoop/mapred/InputSplit

2015-03-23 Thread , Roy
Hi, I am using CDH 5.3.2 packages installation through Cloudera Manager 5.3.2 I am trying to run one spark job with following command PYTHONPATH=~/code/utils/ spark-submit --master yarn --executor-memory 3G --num-executors 30 --driver-memory 2G --executor-cores 2 --name=analytics /home/abc/co