Re: Global Variables in Spark Streaming

2014-09-10 Thread Ravi Sharma
Akhil, By using broadcast variable Will I be able to change the values of Broadcast variable? As per my understanding It will create final variable to access the value across the cluster. Please correct me if I'm wrong. Thanks, Cheers, Ravi Sharma On Wed, Sep 10, 2014 at 7:31 PM, Akhi

Global Variables in Spark Streaming

2014-09-10 Thread Ravi Sharma
essed(temporary stored) and upcoming events.* Cheers, Ravi Sharma

Re: Spark Streaming Output to DB

2014-08-27 Thread Ravi Sharma
stagi <https://twitter.com/mayur_rustagi> >> >> >> >> On Tue, Aug 26, 2014 at 6:45 PM, Akhil Das >> wrote: >> >>> Yes, you can open a jdbc connection at the beginning of the map method >>> then close this connection at the end of map() an

Spark Streaming - Small file in HDFS

2014-08-26 Thread Ravi Sharma
Hi People, I'm using java kafka spark streaming and saving the result file into hdfs. As per my understanding, spark streaming write every processed message or event to hdfs file. Reason to creating one file per message or event could be to ensure fault tolerance. Is there any way spark handle th

Re: Spark Streaming Output to DB

2014-08-26 Thread Ravi Sharma
> > Hello People, > > I'm using java spark streaming. I'm just wondering, Can I make simple jdbc > connection in JavaDStream map() method? > > Or > > Do I need to create jdbc connection for each JavaPairDStream, after map > task? > > Kindly give your thoughts. > > > Cheers, > Ravi Sharma >