Is there anyone who use streaming join to filter spam as guide mentioned?

Hello guys, I'm just trying to use spark streaming features. I noticed that there is join example for filtering spam, so I just want to try. But, nothing happens after join, the output JavaPairDStream content is same as before. So, is there any examples that I can refer to? Thanks for any sug

Re: Is there anyone who use streaming join to filter spam as guide mentioned?

Hi TD, Eventually I found that I made a mistake - the RDD I used for join does not contain any content. Now it works. Thanks, Hawk On 2014年07月21日 17:58, Tathagata Das wrote: Could you share your code snippet so that we can take a look? TD On Mon, Jul 21, 2014 at 7:23 AM, hawkwang

Spark Streaming - How to save all items in batchs from beginning to a single stream rdd?

Hi guys, Is it possible to generate a single stream rdd which can be updated with new batch rdd content? I know that we can use updateStateByKey to make aggregation, but here just want to keep tracking all historical original content. I also noticed that we can save to redis or other storage