Thanks Akhil. For posterity, I ended up with:
https://gist.github.com/dokipen/aa07f351a970fe54fcff
I couldn't get rddToFilename() to work, but it's impl was pretty simple.
I'm a poet but I don't know it.
On Tue, Jun 9, 2015 at 3:10 AM Akhil Das wrote:
> like this?
>
> myDStream.foreachRD
like this?
myDStream.foreachRDD(rdd => rdd.saveAsTextFile("/sigmoid/", codec ))
Thanks
Best Regards
On Mon, Jun 8, 2015 at 8:06 PM, Bob Corsaro wrote:
> It looks like saveAsTextFiles doesn't support the compression parameter of
> RDD.saveAsTextFile. Is there a way to add the functionality in
It looks like saveAsTextFiles doesn't support the compression parameter of
RDD.saveAsTextFile. Is there a way to add the functionality in my client
code without patching Spark? I tried making my own saveFunc function and
calling DStream.foreachRDD but ran into trouble with invoking rddToFileName
an
eName
and making the RDD type parameter work properly. It's probably just do to my
lack of Scala knowledge. Can anyone give me a hand?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Saving-compressed-textFiles-from-a-DStream-in-Scala-tp23177.html
Sen
