Re: Supporting Hive features in Spark SQL Thrift JDBC server

Hello Shahab, I think CassandraAwareHiveContext in Calliopee is what you are looking for. Create CAHC instance and you should be able to run hive functions against

Re: Supporting Hive features in Spark SQL Thrift JDBC server

> > best, > /Shahab > > On Tue, Mar 3, 2015 at 5:41 PM, Rohit Rai wrote: > >> Hello Shahab, >> >> I think CassandraAwareHiveContext >> <https://github.com/tuplejump/calliope/blob/develop/sql/hive/src/main/scala/org/apache/spark/sql/hive/CassandraAwareHiveCo

Re: Using CQLSSTableWriter to batch load data from Spark to Cassandra.

Hi Gerard, What is the version of Spark, Hadoop, Cassandra and Calliope are you using. We never built Calliope to Hadoop2 as we/or our clients don't use Hadoop in their deployments or use it only as the Infra component for Spark in which case H1/H2 doesn't make a difference for them. I know atlea

Re: Using CQLSSTableWriter to batch load data from Spark to Cassandra.

> Will you be attending the Spark Summit? I'll be around. > > We'll be in touch in any case :-) > > -kr, Gerard. > > > > On Thu, Jun 26, 2014 at 11:03 AM, Rohit Rai wrote: > >> Hi Gerard, >> >> What is the version of Spark, Hadoop, Cassandra

Re: spark streaming actor receiver doesn't play well with kryoserializer

Alan/TD, We are facing the problem in a project going to production. Was there any progress on this? Are we able to confirm that this is a bug/limitation in the current streaming code? Or there is anything wrong in user scope? Regards, Rohit *Founder & CEO, **Tuplejump, Inc.* __

[ANN] SparkSQL support for Cassandra with Calliope

Hi All, An year ago we started this journey and laid the path for Spark + Cassandra stack. We established the ground work and direction for Spark Cassandra connectors and we have been happy seeing the results. With Spark 1.1.0 and SparkSQL release, we its time to take Calliope

Re: [ANN] SparkSQL support for Cassandra with Calliope

the error details. > java.lang.IncompatibleClassChangeError: > Found interface (hadoop 2.x) > org.apache.hadoop.mapreduce.TaskAttemptContext, but class (hadoop 1.x) was > expected > > com.tuplejump.calliope.hadoop.cql3.CqlRecordReader.initialize(CqlRecordReader.java:82) > >

Re: [incubating-0.9.0] Too Many Open Files on Workers

Hello Andy, This is a problem we have seen in using the CQL Java driver under heavy ready loads where it is using NIO and is waiting on many pending responses which causes to many open sockets and hence too many open files. Are you by any chance using async queries? I am the maintainer of Calliop

Re: spark/shark + cql3

urrently RDD can only be persisted in memory or dist level(CACHE LEVEL), > but > I always want to add a general methods for RDD, which can persisted on any > storage, such as HDFS level, Cassandra, HBase etc. > > any thoughts here? > > > > On Sun, Feb 23, 2014 at 6:54 PM

Re: [BLOG] Spark on Cassandra w/ Calliope

We are happy that you found Calliope useful and glad we could help. *Founder & CEO, **Tuplejump, Inc.* www.tuplejump.com *The Data Engineering Platform* On Sat, Mar 8, 2014 at 2:18 AM, Brian O'Neill wrote: > > FWIW - I posted some notes to help people get started q

Re: [BLOG] Spark on Cassandra w/ Calliope

v.chowd...@gmail.com> wrote: > +1 that we have been using calliope for few months and its working out > really great for us. Any plans on integrating into spark? > On Mar 10, 2014 1:58 PM, "Rohit Rai" wrote: > >> We are happy that you found Calliope useful and gl

Re: Spark usage patterns and questions

> > 3. In our usecase we read from Kafka, do some mapping and lastly persists > data to cassandra as well as pushes the data over remote actor for > realtime update in dashboard. I used below approaches > - First tried to use vary naive way like stream.map(...)*.foreachRDD( > pushes to actor)* >

Re: Announcing Spark SQL

Great work guys! Have been looking forward to this . . . In the blog it mentions support for reading from Hbase/Avro... What will be the recommended approach for this? Will it be writing custom wrappers for SQLContext like in HiveContext or using Hive's "EXTERNAL TABLE" support? I ask this becaus

Re: [BLOG] : Shark on Cassandra

Thanks a lot for this post Brian! It was on our todo list like forever! :) *Founder & CEO, **Tuplejump, Inc.* www.tuplejump.com *The Data Engineering Platform* On Wed, Mar 26, 2014 at 10:51 AM, Matei Zaharia wrote: > Very cool, thanks for posting this! > > Matei > >

Re: Announcing Spark SQL

t information about the Cassandra schema. And > it should let you write-back to Cassandra by giving a mapping of fields to > the respective cassandra columns. I think all of this would be fairly easy > to implement on SchemaRDD and likely will make it into Spark 1.1 > > - Patrick &

Re: writing booleans w Calliope

Hello Adrian, Calliope relies on transformers to convert from a given type to ByteBuffer which is the format that is required by Cassandra. RichByteBuffer's incompleteness is at fault here. We are working on increasing the types we support out of the box, and will support all types supported in

Re: Calliope Frame size larger than max length

Hello Eric, This happens when the data being fetched from Cassandra in single split is greater than the maximum framesize allowed in thrift (yes it still uses thrift underneath, until the next release when we will start using Native CQL). Generally, we do set the the Cassandra the framesize in Ca