There is not in-built code in Spark Streaming to output to Kafka yet. However, I have heard people have use Twitter Storehaus with foreachRDD and Storehaus has a kafka output. Something that you might look into.
TD On Sun, May 4, 2014 at 11:45 PM, Weide Zhang <weo...@gmail.com> wrote: > Hi , > > Is there any code to implement a kafka output for spark streaming? My use > case is all the output need to be dumped back to kafka cluster again after > data is processed ? What will be guideline to implement such function ? I > heard foreachRDD will create one instance of producer per batch ? If so, > will that hurt performance ? > > Thanks, > > Weide > > >
