nath Prathap (WT01 - BAS)
Cc: u...@spark.incubator.apache.org
Subject: Re: Persist streams to text files
Here's a quick version to store (append) in your local machine
val tweets = TwitterUtils.createStream(ssc, None)
val hashTags = tweets.flatMap(status => status.getText.split(
gt; The company accepts no liability for any damage caused by any virus
> transmitted by this email.
>
> www.wipro.com
>
>
> ------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-l
e it to a file in local machine*
>
>
>
> })
>
> })
>
>
>
> Thanks & Regards
>
> Jishnu Menath Prathap
>
> *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com]
> *Sent:* Friday, November 21, 2014 11:48 AM
> *To:* Jishnu Menath Prathap
2014 11:48 AM
To: Jishnu Menath Prathap (WT01 - BAS)
Cc: u...@spark.incubator.apache.org
Subject: Re: Persist streams to text files
To have a single text file output for each batch you can repartition it to 1
and then call the saveAsTextFiles
stream.repartition(1).saveAsTextFiles(locat
To have a single text file output for each batch you can repartition it to
1 and then call the saveAsTextFiles
stream.repartition(1).saveAsTextFiles(location)
On 21 Nov 2014 11:28, wrote:
> Hi I am also having similar problem.. any fix suggested..
>
>
>
> *Originally Posted by GaganBM*
>
> Hi,
Hi I am also having similar problem.. any fix suggested..
Originally Posted by GaganBM
Hi,
I am trying to persist the DStreams to text files. When I use the inbuilt API
'saveAsTextFiles' as :
stream.saveAsTextFiles(resultDirectory)
this creates a number of subdirectories, for each batch, and w
BufferedWriter
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
.
Any help on how to proceed with this ?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.c
