Re: Spark streaming kafka _output_

2014-04-02 Thread Benjamin Black
please no. On Wed, Apr 2, 2014 at 9:47 PM, Tathagata Das wrote: > If any body is interested is doing this, may I suggested taking a look at > Twitter's Storehaus. It presents an abstract interface for pushing data to > many different backends, including Kafka, mongodb, hbase, etc. Integrating >

Re: Spark streaming kafka _output_

2014-04-02 Thread Tathagata Das
If any body is interested is doing this, may I suggested taking a look at Twitter's Storehaus. It presents an abstract interface for pushing data to many different backends, including Kafka, mongodb, hbase, etc. Integrating DStream.foreachRDD with Storehaus maybe a very very useful thing to do. So

Re: Spark streaming kafka _output_

2014-04-02 Thread Soren Macbeth
Anybody? Seems like a reasonable thing to be able to do no? On Fri, Mar 21, 2014 at 3:58 PM, Benjamin Black wrote: > Howdy, folks! > > Anybody out there having a working kafka _output_ for Spark streaming? > Perhaps one that doesn't involve instantiating a new producer for every > batch? > > Th