text file stream to HDFS

Hello, How should I write a text file stream DStream to HDFS. I tried the the following val lines = ssc.textFileStream("hdfs:/user/hadoop/spark/streaming/input/") lines.saveAsTextFile("hdfs:/user/hadoop/output1") val lines = ssc.textFileStream("hdfs:/user/hadoop/spark/streaming/input/")

spark streaming HDFS file issue

I am running a spark streaming example from learning spark book with one change. The change I made was for streaming a file from HDFS. val lines = ssc.textFileStream("hdfs:/user/hadoop/spark/streaming/input") I ran the application number of times and every time dropped a new file in the input dir

Fwd: Storing an action result in HDFS

Hello All, I am new to Spark. I have a very basic question.How do I write the output of an action on a RDD to HDFS? Thanks in advance for the help. Cheers, Ravi