Cassandra with Spark, select a few columns with case class

2018-03-25 Thread Guillermo Ortiz
I'm trying to get a few columns from a Cassandra table from Spark and put them in a case class. If I want all the columns that I have in my case class works. But, I only want to bring a few of them and don't have a specific case class for each case. I tried to overload constructor in the case clas

Re: replace dead node vs remove node

2018-03-25 Thread kurt greaves
Didn't read the blog but it's worth noting that if you replace the node and give it a *different* ip address repairs will not be necessary as it will receive writes during replacement. This works as long as you start up the replacement node before HH window ends. https://issues.apache.org/jira/bro

Deserialize Map[Int, UDT] to a case class from Spark Connector

2018-03-25 Thread Guillermo Ortiz
Hello, I'm working with UDT's and spark connector with these dependencies: 2.11.12 2.0.2 2.0.7 3.4.0 org.apache.spark spark-core_2.11 ${spark.version} org.apache.spark spark-streaming_2.11 ${spark.version} com.datastax.spark spark-cassandra-connector_2.11 ${cassandra-conector.version}

Measuring eventual consistency latency

2018-03-25 Thread Jeronimo de A. Barros
I'd like to know if there is a reasonable method to measure how long take to have the data available across all replica nodes in a multi DC environment using LOCAL_ONE or LOCAL_QUORUM consistency levels. If already there be a study about this topic in some place and someone could point me the dire

Re: Deserialize Map[Int, UDT] to a case class from Spark Connector

2018-03-25 Thread Christophe Schmitz
Hi Guillermo Which version of Spark are you using? Starting with Version 2.0, Spark is built with Scala 2.11 by default. If you are using a prior version (which looks like it's the case since your error message mention scala 2.10), you might need to build it yourself from sources with Scala 2.11 s

Re: Measuring eventual consistency latency

2018-03-25 Thread Christophe Schmitz
Hi Jeronimo, I am not sure that will address your exact request, but did you look at this issue (resolved in 3.8) which adds a ned latency across DCs metrics? https://issues.apache.org/jira/browse/CASSANDRA-11569 Cheers, Christophe On 26 March 2018 at 10:01, Jeronimo de A. Barros wrote: > I'd

Re: Measuring eventual consistency latency

2018-03-25 Thread Jeff Jirsa
Probably closer to https://issues.apache.org/jira/browse/CASSANDRA-13289 Will be in 4.0 -- Jeff Jirsa > On Mar 25, 2018, at 4:44 PM, Christophe Schmitz > wrote: > > Hi Jeronimo, > > I am not sure that will address your exact request, but did you look at this > issue (resolved in 3.8) whic