Re: Manipulating RDDs within a DStream

2014-10-31 Thread Helena Edelson
toArray > >OPEN cassandra connection >store arr >CLOSE cassandra connection > > }) > > > Thanks > > > > - > Lalit Yadav > la...@sigmoidanalytics.com > -- > View this message in context: > http://apache-spark-use

Re: Manipulating RDDs within a DStream

2014-10-31 Thread Helena Edelson
Hi Harold, Yes, that is the problem :) Sorry for the confusion, I will make this clear in the docs ;) since master is work for the next version. All you need to do is use spark 1.1.0 as you have it already "com.datastax.spark" %% "spark-cassandra-connector" % "1.1.0-beta1” and assembly - not fr

Re: Manipulating RDDs within a DStream

2014-10-31 Thread Harold Nguyen
a connection > > }) > > > Thanks > > > > - > Lalit Yadav > la...@sigmoidanalytics.com > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Manipulating-RDDs-within-a-DStream-tp17740p17800.html > Sent from the A

Re: Manipulating RDDs within a DStream

2014-10-31 Thread Harold Nguyen
Hi Helena, Thanks very much ! I'm using Spark 1.1.0, and spark-cassandra-connector-assembly-1.2.0-SNAPSHOT Best wishes, Harold On Fri, Oct 31, 2014 at 10:31 AM, Helena Edelson < helena.edel...@datastax.com> wrote: > Hi Harold, > Can you include the versions of spark and spark-cassandra-connect

Re: Manipulating RDDs within a DStream

2014-10-31 Thread Helena Edelson
Hi Harold, Can you include the versions of spark and spark-cassandra-connector you are using? Thanks! Helena @helenaedelson On Oct 30, 2014, at 12:58 PM, Harold Nguyen wrote: > Hi all, > > I'd like to be able to modify values in a DStream, and then send it off to an > external source like C

Re: Manipulating RDDs within a DStream

2014-10-30 Thread lalit1303
pulating-RDDs-within-a-DStream-tp17740p17800.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail

Manipulating RDDs within a DStream

2014-10-30 Thread Harold Nguyen
Hi all, I'd like to be able to modify values in a DStream, and then send it off to an external source like Cassandra, but I keep getting Serialization errors and am not sure how to use the correct design pattern. I was wondering if you could help me. I'd like to be able to do the following: wor

Re: Manipulating RDDs within a DStream

2014-10-30 Thread Harold Nguyen
Hi, Sorry, there's a typo there: val arr = rdd.toArray Harold On Thu, Oct 30, 2014 at 9:58 AM, Harold Nguyen wrote: > Hi all, > > I'd like to be able to modify values in a DStream, and then send it off to > an external source like Cassandra, but I keep getting Serialization errors > and am n