Re: Virtual Nodes, lots of physical nodes and potentially increasing outage count?

Thanks for your thoughts guys. I agree that with vnodes total downtime is lessened. Although it also seems that the total number of outages (however small) would be greater. But I think downtime is only lessened up to a certain cluster size. I'm thinking that as the cluster continues to grow:

Consistency QUORUM does not work anymore (hector:Could not fullfill request on this host)

hi list, I am using Cassandra with 3 data centers, each DC has more than 10 nodes. the schema for a keyspace: {DC1:3, DC2:3, DC3:3} now, I put some rows using hector with CL Local_Quorum in DC1 ，and then I get a row with the same CL Local_Quorum in DC1，some exceptions were occured: Cassandra wit

Cassandra on EC2 - describe_ring() is giving private IPs

We have a 4 node cluster in us-east region in two different AZ's. Clients connect to this cluster from our datacenter which is not on AWS. Hector clients are initialized with public DNS names, then listern_address is with private ip and rpc_address is with 0.0.0.0. Having issues with Node Auto Di

Re: Virtual Nodes, lots of physical nodes and potentially increasing outage count?

Hi Eric, The time to recover one node is limited by that node, but the time to recover that's most important is just the time to replicate the data that is missing from that node. This is the removetoken operation (called removenode in 1.2), and this gets faster the more nodes you have. Richard.

Re: Vnode migration path

Hi Mike, There's also the shuffle utility (in the bin directory) that can incrementally move ranges around to migrate to vnodes. Richard. On 11 December 2012 08:47, Michael Kjellman wrote: > So I'm wondering if anyone has given thought to their migration path to > Vnodes. Other than having a

Re: Why Secondary indexes is so slowly by my test?

Hi, Secondary index lookups are more complicated than normal queries so will be slower. Items have to first be queried in the index, then retrieved from their actual location. Also, inserting into indexed CFs will be slower (but will get substantially faster in 1.2 due to CASSANDRA-2897). If yo

Re: Multiple Data Center shows very uneven load

Keith Wright wrote > I have base Cassandra 1.1.7 installed in two data centers with 3 nodes > each using a PropertyFileSnitch as outlined below. When I run a nodetool > ring, I see a very uneven load. Any idea what I could be going on? I have > not added/removed any nodes or changed the replication

Re: multiget_slice SlicePredicate

Each node is doing it's thing in parallel….they on purpose do NOT co-ordinate as they do not need to so each one is doing it's scan on the rows it has individually. If all rows "happen" to be on the same server, sure some may be done sequentially depending on number of rows vs. thread pool size

Re: Selecting rows efficiently from a Cassandra CF containing time series data

We use PlayOrm to do something similar We have an object like this(typing all this from memory)…. TimeSeries { @NoSqlPartitionedByField private long beginOfMonth; @NoSqlIndexed Private long timestamp; } Then we just use the ScalableSQL to query into the partition itself. This is

Re: Diagnosing memory issues

Thank you for the response. Since the time of this question, we've identified a number of areas that needed improving and have helped things along quite a bit. To answer your question, we were seeing both ParNew and CMS. There were no errors in the log, and all the nodes have been up. Howe

Re: Vnode migration path

Awesome (and very welcomed news), what kind of failure conditions can we expect if a node goes down during the migration? From: Richard Low mailto:r...@acunu.com>> Reply-To: "user@cassandra.apache.org" mailto:user@cassandra.apache.org>> Date: Tuesday, December 1

Re: Virtual Nodes, lots of physical nodes and potentially increasing outage count?

Ok, thanks Richard. That's good to hear. However, I still contend that as node count increases to infinity, the probability of there being at least two node failures in the cluster at any time would increase to 100%. I think of this as somewhat analogous to RAID -- I would not be comfortable wit

Re: Selecting rows efficiently from a Cassandra CF containing time series data

I would consider to use wide rows. If you add timestamp to your column name you have naturally sorted data. You can easily select any time range without any indexes. Thank you, Andrey On Tue, Dec 11, 2012 at 6:23 AM, Chin Ko wrote: > I would like to get some opinions on how to select an incr

Re: Selecting rows efficiently from a Cassandra CF containing time series data

Wide rows does not work well if you start getting past 10,000,000 columns though so be very very careful there. PlayOrm does some wide row indices for us and each row length is as large as the number of rows in a partition so without playorm you could do partitioning yourself by the way….It's a

Re: Filter data on row key in Cassandra Hadoop's Random Partitioner

You may want to look into CQL3 as I hear there may be a way to specify the query so only those rows are map/reduced. I am not sure if that is out yet or not but I remember someone from datastax telling me about it. Dean On 12/11/12 9:46 AM, "Ayush V." wrote: >I'm working on Cassandra Hadoop in

cassandra vs couchbase benchmark

http://www.slideshare.net/Couchbase/benchmarking-couchbase#btnNext

Re: Multiple Data Center shows very uneven load

I would check the logs for Dropped Message alerts, and run repair if you have not. I would also look at the nodetool CF stats on each node to check the row size. It may be the case that you have some very wide rows stored on nodes 10.56.92.196, 10.28.91.8, 10.56.92.198 and 10.28.91.2 Hope th

Re: Multiple Data Center shows very uneven load

Hi Sergey, I think you have forgotten to include some information in your email. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 12/12/2012, at 3:00 AM, Sergey Olefir wrote: > Keith Wright wrote >> I have base Cass

Re: multiget_slice SlicePredicate

I tend to caution against making very large batch mutations or multi gets, by which I mean 100's of rows at a time. Each row request becomes a task and they can temporarily fill the mutation or read thread pool. Meaning overall *client* request throughout drops while a big request is chewed th

Re: Virtual Nodes, lots of physical nodes and potentially increasing outage count?

> Is it possible to configure or write a snitch that would create separate > distribution zones within the cluster? (e.g. 144 nodes in cluster, split > into 12 zones. Data stored to node 1 could only be replicated to one of 11 > other nodes in the same distribution zone). This is kind of what

Re: Selecting rows efficiently from a Cassandra CF containing time series data

Couple of ideas, one is to multiplex the even log stream (using flume or kafka) and feed it straight into your secondary system. The event system should allow you to rate limit inserts if that is a concern. The other is to use partitioning. Group the log entries per user into some sensible par

Re: Slow Reads in Cassandra with Hadoop

First I would try to simplify your architecture. Get everything onto the same OS. Then change the topology so you have 1 job tracker, and 4 nodes that ran both Cassandra and Hadoop tasks. So that reading and mapping the data is happening on the same nodes. Reads from cassandra happen as range

Re: Multiple Data Center shows very uneven load

I am running repairs now. I checked CF stats and they all appear to have very similar max and average row sizes between the lowest loaded node and the highest. One thing I did notice is that nodetool netstats shows files for a column family I dropped named payload and they never appear to go away

Re: Consistency QUORUM does not work anymore (hector:Could not fullfill request on this host)

> Caused by: TimedOutException() Means the nodes involved in the request did not return to the co ordinator before the rpc_timeout expired. Check the logs on the servers to see if they are overloaded and dropping messages. Also check that you are not asking for too much data. Cheers ---

Primary/secondary index question / best practices?

Hi folks - I'm doing an informal proof-of-concept with Cassandra and I've been getting some conflicting information about how my data layout should go. Perhaps somebody could point me in the right direction. I have a column family that will have billions of rows of data. The data do not have

Re: Primary/secondary index question / best practices?

Have to help out on a design without specifics but here is some advice based on the limited information Primary key : yes, must be cluster unique. TimeUUID or UUID….PlayOrm has very unique TimeUUID like keys as in this one 7AL2S8Y.b1 (b1 is the hostname and the prefix is a "unique" timestamp b

Re: Primary/secondary index question / best practices?

That was supposed to read "Hard to help out" oops. On 12/11/12 2:56 PM, "Hiller, Dean" wrote: >Have to help out on a design without specifics but here is some advice >based on the limited information > >Primary key : yes, must be cluster unique. TimeUUID or UUIDŠ.PlayOrm has >very unique TimeUU

Re: Cassandra on EC2 - describe_ring() is giving private IPs

> Though I configured the listen_address with public dns, still I had the same > issue. Internally the public DNS resolves to the private IP. > looks like describe_ring() is the one which provides the details. describe_ring() returns includes the registered RPC addresses for the nodes. Trying

bug with cqlsh for foreign charater

I have a column family with composite column CompositeType(UTF8Type, UTF8Type, LongType, UTF8Type) It stores (firstName, LastName, userID, meaningfulColumnName), if I insert the record with foreign characters,  looks like that cqlsh -3 interprets the values wrong. I can get the values back from

RE: Primary/secondary index question / best practices?

Dean, thank you for your response. To the second half of the query, I'm a little concerned about the secondary index approach since the indexes that I want to create are columns with high entropy. For example, I would like to query by User name and IP address, values which are decidedly NOT

Re: Multiple Data Center shows very uneven load

aaron morton wrote > Hi Sergey, I think you have forgotten to include some information in your > email. Ah, I used Nable's markup and it seems to have eaten text somehow. Anyway, here it is without formatting (much harder to read though)> I have a very similar issue myself and would love to know

Re: Primary/secondary index question / best practices?

Is there any column that would be a good qualifer as a partition key? Some people partition by time like every month or every day, and then you can either have your own secondary indexes that you query into(high entropy is NOT a big deal here) or PlayOrm can do some for you or you could use CQL

Re: Primary/secondary index question / best practices?

Oh, and one last thingŠthere is no limit on number of partitions, just on partition size really. Dean On 12/11/12 4:26 PM, "Hiller, Dean" wrote: >Is there any column that would be a good qualifer as a partition key? > >Some people partition by time like every month or every day, and then you >c

Re: Consistency QUORUM does not work anymore (hector:Could not fullfill request on this host)

Thanks aaron. more information is I can read data correctly using Aqulies with Local_Quorum. I just check the system.log which is normal on 172.16.74.31 and the number of RPC time out is 10s. the client exception occured on 2012-12-05. and the all log on server on 2012-12-05 were: WARN [pool-

Re: Why Secondary indexes is so slowly by my test?

Thanks to Low. We use CompositeColumn to substitue it in single not-equality and definite equalitys query. And we will give up cassandra because of the weak query ability and unstability. Many times, we found our data in confusion without definite cause in our cluster. For example, only two row

Re: Consistency QUORUM does not work anymore (hector:Could not fullfill request on this host)

hi aaron. There is no problem with get_slices but get_range_slices failed. the way I use this method is: sliceRangeQuery.setFamily(family); sliceRangeQuery.setKeys(rowkey,rowkey); sliceRangeQuery.setRange(Character.Min_Value, Character.Max_Value, false, Integer.Max_Value); sliceRangeQuery.setRowC