You can use foreachRDD<http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations-on-dstreams> to get access to the batch API in streaming jobs.
From: Amir Rahnama [mailto:amirrahn...@gmail.com] Sent: Thursday, November 12, 2015 12:11 AM To: ayan guha <guha.a...@gmail.com> Cc: user <user@spark.apache.org> Subject: Re: How can you sort wordcounts by counts in stateful_network_wordcount.py example Hi Ayan, Thanks for help, Your example is not the streaming example. There we don't have sortByKey. On Wed, Nov 11, 2015 at 11:35 PM, ayan guha <guha.a...@gmail.com<mailto:guha.a...@gmail.com>> wrote: how about this? sorted = running_counts.map(lambda t: t[1],t[0]).sortByKey() Basically swap key and value of the RDD and then sort? On Thu, Nov 12, 2015 at 8:53 AM, Amir Rahnama <amirrahn...@gmail.com<mailto:amirrahn...@gmail.com>> wrote: Hey, Anybody knows how can one sort the result in the stateful example? Python would be prefered. https://github.com/apache/spark/blob/859dff56eb0f8c63c86e7e900a12340c199e6247/examples/src/main/python/streaming/stateful_network_wordcount.py -- Thanks and Regards, Amir Hossein Rahnama Tel: +46 (0) 761 681 102 Website: www.ambodi.com<http://www.ambodi.com> Twitter: @_ambodi<https://twitter.com/_ambodi> -- Best Regards, Ayan Guha -- Thanks and Regards, Amir Hossein Rahnama Tel: +46 (0) 761 681 102 Website: www.ambodi.com<http://www.ambodi.com> Twitter: @_ambodi<https://twitter.com/_ambodi>
