Hi All,
Sorry to disturb you.
I have built a spark cluster based on mesos.
I run some tests on spark shell.
It works. However I can find some exceptions in webpage.
scala> val textFile = sc.textFile("hdfs://10.1.2.12:9000/README.md")
scala> textFile.count()
14/10/03 15:20:54 INFO mapred.FileInputFormat: Total input paths to process
: 1
14/10/03 15:20:54 INFO spark.SparkContext: Starting job: count at
<console>:15
14/10/03 15:20:54 INFO scheduler.DAGScheduler: Got job 0 (count at
<console>:15) with 2 output partitions (allowLocal=false)
14/10/03 15:20:54 INFO scheduler.DAGScheduler: Final stage: Stage 0(count
at <console>:15)
14/10/03 15:20:54 INFO scheduler.DAGScheduler: Parents of final stage:
List()
14/10/03 15:20:54 INFO scheduler.DAGScheduler: Missing parents: List()
14/10/03 15:20:54 INFO scheduler.DAGScheduler: Submitting Stage 0 (hdfs://
10.1.2.12:9000/README.md MappedRDD[1] at textFile at <console>:12), which
has no missing parents
......
res0: Long = 141
What's the problem? How can I solve it?
Thanks,
Tim
The error information:
count at <console>:15 +details
org.apache.spark.rdd.RDD.count(RDD.scala:904)
$line9.$read$$iwC$$iwC$$iwC$$iwC.<init>(<console>:15)
$line9.$read$$iwC$$iwC$$iwC.<init>(<console>:20)
$line9.$read$$iwC$$iwC.<init>(<console>:22)
$line9.$read$$iwC.<init>(<console>:24)
$line9.$read.<init>(<console>:26)
$line9.$read$.<init>(<console>:30)
$line9.$read$.<clinit>(<console>)
$line9.$eval$.<init>(<console>:7)
$line9.$eval$.<clinit>(<console>)
$line9.$eval.$print(<console>)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:789)
org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1062)
org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:615)
org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:646)
