Hi Mahendra Did you tried mapping the X case class members further to a String object and then saving the RDD[String] ?
Thanks Deepak On Oct 7, 2016 23:04, "Mahendra Kutare" <mahendra.kut...@gmail.com> wrote: > Hi, > > I am facing issue with writing RDD[X] to HDFS file path. X is a simple > case class with variable time as primitive long. > > When I run the driver program with - master as > > spark://<name_node>:7077 > > I get this - > > Caused by: java.io.EOFException > at java.io.ObjectInputStream$BlockDataInputStream. > readFully(ObjectInputStream.java:2744) > at java.io.ObjectInputStream.readFully(ObjectInputStream.java:1032) > at org.apache.hadoop.io.Text.readString(Text.java:473) > at org.apache.hadoop.io.Text.readString(Text.java:464) > at org.apache.hadoop.io.WritableUtils.readEnum(WritableUtils.java:415) > at org.apache.hadoop.mapreduce.TaskID.readFields(TaskID.java:223) > at org.apache.hadoop.mapreduce.TaskAttemptID.readFields( > TaskAttemptID.java:139) > at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285) > at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:77) > at org.apache.spark.SerializableWritable$$anonfun$ > readObject$1.apply$mcV$sp(SerializableWritable.scala:45) > at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1205) > at org.apache.spark.SerializableWritable.readObject( > SerializableWritable.scala:41) > > Full stack trace - https://gist.github.com/imaxxs/ > 650efd1cb367783897e3d37c1103512b > > The same code works fine with I run the driver program in local mode with > master as - local[4] > > Can anyone please provide some pointers to debug this issue? I have tried > many options - writing to the local file, writing to the HDFS file, > writing in different formats e.t.c. > > Thanks for the help in advance, > > Mahendra > about.me/mahendrakutare > > <https://about.me/mahendrakutare?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Only those who will risk going too far can possibly find out how far one > can go. >
