Here is the worker track. 15/03/26 16:05:47 INFO Worker: Asked to kill executor app-20150326160534-0005/1 15/03/26 16:05:47 INFO ExecutorRunner: Runner thread for executor app-20150326160534-0005/1 interrupted 15/03/26 16:05:47 INFO ExecutorRunner: Killing process! 15/03/26 16:05:47 ERROR FileAppender: Error writing stream to file /opt/cloud/spark/work/app-20150326160534-0005/1/stderr java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162) at java.io.BufferedInputStream.read1(BufferedInputStream.java:272) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) at java.io.FilterInputStream.read(FilterInputStream.java:107) at org.apache.spark.util.logging.FileAppender.appendStreamToFile(FileAppender.scala:70) at org.apache.spark.util.logging.FileAppender$$anon$1$$anonfun$run$1.apply$mcV$sp(FileAppender.scala:39) at org.apache.spark.util.logging.FileAppender$$anon$1$$anonfun$run$1.apply(FileAppender.scala:39) at org.apache.spark.util.logging.FileAppender$$anon$1$$anonfun$run$1.apply(FileAppender.scala:39) at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1460) at org.apache.spark.util.logging.FileAppender$$anon$1.run(FileAppender.scala:38) 15/03/26 16:05:48 INFO Worker: Executor app-20150326160534-0005/1 finished with state KILLED exitStatus 1 15/03/26 16:05:48 WARN ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkExecutor@cloud1:37365] has failed, ad dress is now gated for [5000] ms. Reason is: [Disassociated].
2015-03-26 23:01 GMT+08:00 Michael Armbrust <mich...@databricks.com>: > I would suggest looking for errors in the logs of your executors. > > On Thu, Mar 26, 2015 at 3:20 AM, 李铖 <lidali...@gmail.com> wrote: > >> Again,when I do larger file Spark-sql query, error occured.Anyone have >> got fix it .Please help me. >> Here is the track. >> >> org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output >> location for shuffle 0 >> at >> org.apache.spark.MapOutputTracker$$anonfun$org$apache$spark$MapOutputTracker$$convertMapStatuses$1.apply(MapOutputTracker.scala:386) >> at >> org.apache.spark.MapOutputTracker$$anonfun$org$apache$spark$MapOutputTracker$$convertMapStatuses$1.apply(MapOutputTracker.scala:383) >> at >> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) >> at >> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) >> at >> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) >> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108) >> at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) >> at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108) >> at >> org.apache.spark.MapOutputTracker$.org$apache$spark$MapOutputTracker$$convertMapStatuses(MapOutputTracker.scala:382) >> at >> org.apache.spark.MapOutputTracker.getServerStatuses(MapOutputTracker.scala:178) >> at >> org.apache.spark.shuffle.hash.BlockStoreShuffleFetcher$.fetch(BlockStoreShuffleFetcher.scala:42) >> at >> org.apache.spark.shuffle.hash.HashShuffleReader.read(HashShuffleReader.scala:40) >> at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:92) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >> at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >> at >> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) >> at org.apache.spark.sql.SchemaRDD.compute(SchemaRDD.scala:120) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >> at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >> at >> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) >> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263) >> at org.apache.spark.rdd.RDD.iterator(RDD.scala:230) >> at >> org.apache.spark.api.python.PythonRDD$WriterThread$$anonfun$run$1.apply$mcV$sp(PythonRDD.scala:242) >> at >> org.apache.spark.api.python.PythonRDD$WriterThread$$anonfun$run$1.apply(PythonRDD.scala:204) >> at >> org.apache.spark.api.python.PythonRDD$WriterThread$$anonfun$run$1.apply(PythonRDD.scala:204) >> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1460) >> at >> org.apache.spark.api.python.PythonRDD$WriterThread.run(PythonRDD.scala:203) >> >> ) >> >> >
