Re: Spark Streaming writing to HDFS

2014-10-05 Thread Sean Owen
On Sat, Oct 4, 2014 at 5:28 PM, Abraham Jacob wrote: > import org.apache.hadoop.conf.Configuration; > import org.apache.hadoop.io.IntWritable; > import org.apache.hadoop.io.Text; > import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; Good. There is also a org.apache.hadoop.mapred.TextO

Re: Spark Streaming writing to HDFS

2014-10-04 Thread Abraham Jacob
Hi Sean/All, I am importing among various other things the newer mapreduce version - import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; import org.apache.spark.Spa

Re: Spark Streaming writing to HDFS

2014-10-04 Thread Sean Owen
Are you importing the '.mapred.' version of TextOutputFormat instead of the new API '.mapreduce.' version? On Sat, Oct 4, 2014 at 1:08 AM, Abraham Jacob wrote: > Hi All, > > > Would really appreciate if someone in the community can help me with this. I > have a simple Java spark streaming applica