Re: Lots of deletions results in death by GC

On Wed, Feb 5, 2014 at 2:52 AM, srmore wrote: > Dropped messages are the sign that Cassandra is taking heavy that's the load > shedding mechanism. I would love to see some sort of back-pressure > implemented. +1 for back pressure in general with Cassandra

Re: No deletes - is periodic repair needed? I think not...

Hi, In a distributed system, such as Cassandra, things can happen (node down, stop the world GC, hardware issue, ...) and desynchronize replicas, isn't repairing also a needed operation to keep replicas up to date at least once a week or once a month ? It is a strong and reliable process to keep t

Re: Adding datacenter for move to vnodes

Hi, we did this exact same operation here too, with no issue. Contrary to Paulo we did not modify our snitch. We simply added a "dc_suffix" in the property in cassandra-rackdc.properties conf file for nodes in the new cluster : # Add a suffix to a datacenter name. Used by the Ec2Snitch and Ec2Mu

Sporadic gossip exception on add node

Environment : linux, cassandra 2.0.4, 3 node, embedded, byte ordered, LCS When I add a node to the existing 3 node cluster I sometimes get the exception 'Unable to gossip with any seeds ' listed below. If I just restart it without any change then mostly it works. Must be some timing issue. The

Re: GC taking a long time

Hi Robert, The heap, and GC are things a bit tricky to tune, I recently read a post about heap, explaining how heap works and how to tune it : http://tech.shift.com/post/74311817513/cassandra-tuning-the-jvm-for-read-heavy-workloads There is plenty of this kind of blogs or articles on the web. B

Re: GC taking a long time

for the restart issue see CASSANDRA-6008 and 6086 On Thu, Feb 6, 2014 at 12:19 PM, Alain RODRIGUEZ wrote: > Hi Robert, > > The heap, and GC are things a bit tricky to tune, > > I recently read a post about heap, explaining how heap works and

FW: exception during add node due to test beforeAppend on SSTableWriter

Also, these nodes and data are entirely created by a 2.0.4 code, so should not really be a 1.1.x related bug. Also, I restarted the whole test, thus completely new database, and I get similar problems. From: Desimpel, Ignace Sent: vrijdag 31 januari 2014 18:02 To: user@cassandra.apache.org Subje

Re: Periodic rpc_timeout errors on select query

Hi Steve, It looks like it will be pretty easy for us to do some testing with the new client version. I'm going to give it a shot and keep my fingers crossed. Thanks again, Chap On 5 Feb 2014, at 18:10, Steven A Robenalt wrote: Hi Chap, If you have the ability to test the 2.0.0rc2 driver,

Re: Question: ConsistencyLevel.ONE with multiple datacenters

I think the scenario you outlined is correct. The DES handles multiple DCs poorly and the LOCAL_ONE hammer is the best bet. On 01/31/2014 12:40 PM, Paulo Ricardo Motta Gomes wrote: Hey, When adding a new data center to our production C* datacenter using the procedure described in [1], some of

Re: Question about local reads with multiple data centers

On 01/29/2014 08:07 PM, Donald Smith wrote: My question: will the read process try to read first locally from the datacenter DC2 I specified in its connection string? I presume so. (I doubt that it uses the client's IP address to decide which datacenter is closer. And I am unaware of anot

exceptions all around in clean cluster

Hi, I am running a small 2 DC cluster of 3 nodes (each DC). I use 3 replicas in both DCs (all 6 nodes have everything) on Cassandra 1.2.11. I populated the cluster via cqlsh pipelined with a series of inserts. I use the cluster for tests, the dataset is pretty small (hundreds of thousands of recor

Re: what tool will create noncql columnfamilies in cassandra 3a

On 02/05/2014 04:57 AM, Sylvain Lebresne wrote: >How will users adjust the meta data of non cql column families The rational for removing cassandra-cli is mainly that maintaining 2 fully featured command line interface is a waste of the project resources in the long run. It's just a tool using t

Re: First SSTable file is not being compacted

On 02/06/2014 01:17 AM, Sameer Farooqui wrote: I'm running C* 2.0.4 and when I have a handful of SSTable files and trigger a manual compaction with 'nodetool compact' the first SSTable file doesn't get compacted away. Is there something special about the first SSTable that it remains even after

Re: exceptions all around in clean cluster

I ran nodetool scrub on nodes in the less corrupted datacenter and tried nodetool rebuild from this datacenter. This is the result: 2014-02-06 15:04:24.645+0100 [Thread-83] [ERROR] CassandraDaemon.java(191) org.apache.cassandra.service.CassandraDaemon: Exception in thread Thread[Thread-83,5,main]

Re: Question: ConsistencyLevel.ONE with multiple datacenters

Cool. I actually changed the consistency level to LOCAL_ONE and things worked as expected. Cheers! On Thu, Feb 6, 2014 at 11:31 AM, Chris Burroughs wrote: > I think the scenario you outlined is correct. The DES handles multiple > DCs poorly and the LOCAL_ONE hammer is the best bet. > > > On 01

Bootstrap failure on C* 1.2.13

Hello, One the nodes of our cluster failed and I performed the "replace a dead node procedure" described in [1]. After about 5 or 6 hours of streaming during bootstrap, the node fails with the exception "Unable to fetch range for keyspace foobar from any hosts." [2]. I haven't found any thread or

Re: First SSTable file is not being compacted

Yeah, it's definitely repeatable. I have a lab environment set up where the issue is occurring and I've recreated the lab environment 4 - 5 times and it's occurred each time. In my demodb.users CF I currently have 2 data SSTables on disk (demodb-users-jb-1-Data.db and demodb-users-jb-6-Data.db). H

Re: Adding datacenter for move to vnodes

Thank you Alain! That was exactly what I was looking for. I was worried I'd have to do a rolling restart to change the snitch. Katriel On Thu, Feb 6, 2014 at 1:10 PM, Alain RODRIGUEZ wrote: > Hi, we did this exact same operation here too, with no issue. > > Contrary to Paulo we did not modify

Re: Adding datacenter for move to vnodes

Glad it helps. Good luck with this. Cheers, Alain 2014-02-06 17:30 GMT+01:00 Katriel Traum : > Thank you Alain! That was exactly what I was looking for. I was worried > I'd have to do a rolling restart to change the snitch. > > Katriel > > > > On Thu, Feb 6, 2014 at 1:10 PM, Alain RODRIGUEZ w

Re: exceptions all around in clean cluster

Update: I dropped the keyspace, the system keyspace, deleted all the data and started from fresh state. Now it behaves correctly. The previously reported state is therefore the result of the keyspace being dropped beforehand and recreated with no compression on sstables - maybe some sstables were l

Re: First SSTable file is not being compacted

Sounds like you have done some solid test work. I suggest reading https://issues.apache.org/jira/browse/CASSANDRA-6568 and if you think your issue is the same adding your reproduction case there, otherwise create your own ticket. On 02/06/2014 10:53 AM, Sameer Farooqui wrote: Yeah, it's defi

Re: One of my nodes is in the wrong datacenter - help!

Thanks for the advice. I did use “removenode” as I was aware of the replace_token problems. I haven’t run into the issue in CASSANDRA-6615 yet, and I don’t believe I’m at risk for it. I’m actually running into a different problem. Having done a remove node on the node with the incorrect dat

Re: exception during add node due to test beforeAppend on SSTableWriter

I'm seeing the same with cassandra-2.0.4 during compaction, after lot of sstable files are streamed after bootstrap/repair. Strange thing is, the 'Last written key >= current key' exception during compaction of L0, L1 sstables, goes away after restarting cassandra. But, then see those warnings a

ring describe returns only public ips

We are using Cassandra 1.2.13 in a multi-datacenter setup. We are using Astyanax as the client, and we’d like to enable its token aware connection pool type and ring describe node discovery type. Unfortunately, I’ve found that both thrift’s describe_ring and `nodetool ring` only report the publi

CQL list command

TL;DR Is there a CQL equivalent of the CLI List command ? yes or No? Long version I often use the CLI command LIST for debugging or when teaching students showing them what’s going on under the hood of CQL. I see that CLI swill be removed in Cassandra 3 and we will lose this ability. It wou

Re: CQL list command

try SELECT * FROM my_table LIMIT 100; On February 6, 2014 at 4:02:26 PM, Andrew Cobley (a.e.cob...@dundee.ac.uk) wrote: TL;DR Is there a CQL equivalent of the CLI List command ? yes or No? Long version I often use the CLI command LIST for debugging or when teaching students showing t

Auto-Bootstrap not Auto-Bootstrapping?

Hi all, We recently needed/wanted to reconfigure the disks for our 3-node C*2.0.4 Cassandra setup and rebuild the server at the same time. Upon adding the newly rebuilt server into the cluster, it immediately started serving read requests with no data! Then because the latency is so "good" the vas

Re: Auto-Bootstrap not Auto-Bootstrapping?

Is it a seed node? My understanding is that they do not bootstrap On Feb 6, 2014 4:23 PM, Thunder Stumpges wrote: Hi all, We recently needed/wanted to reconfigure the disks for our 3-node C*2.0.4 Cassandra setup and rebuild the server at the same time. Upon adding the newly rebuilt server int

Re: Auto-Bootstrap not Auto-Bootstrapping?

I guess so, it likely was listed in the seeds in cassandra.yaml as all three of the existing servers were, and the rebuilt one used the same name and IP. On Thu, Feb 6, 2014 at 1:42 PM, Keith Wright wrote: > Is it a seed node? My understanding is that they do not bootstrap > On Feb 6, 2014 4:

Re: Adding datacenter for move to vnodes

Hello, My question is why would you need another DC to migrate to Vnodes? How about decommissioning each node in turn, changing the cassandra.yaml accordingly, delete the data and bring the node back in the cluster and let it bootstrap from the others? We did that recently with our demo cluster.

Re: Adding datacenter for move to vnodes

My understanding is you can't mix vnodes and regular nodes in the same DC. Is it correct? On Thu, Feb 6, 2014 at 2:16 PM, Vasileios Vlachos < vasileiosvlac...@gmail.com> wrote: > Hello, > > My question is why would you need another DC to migrate to Vnodes? How > about decommissioning each node i

Re: Cassandra 2.0 with Hadoop 2.x?

Thunder, Cyril, thanks for your responses. FWIW I started working on writing a set of InputFormat and OutputFormat classes that use the DataStax Java driver instead of the thrift client. That seems to be going pretty well. I am not sure if I can eliminate all of the thrift code, but at least the

Re: Cassandra 2.0 with Hadoop 2.x?

On Thu, Feb 6, 2014 at 3:50 PM, Clint Kelly wrote: > I can > post a link if anyone is curious (once it is done). > I'm curious... thanks -- :- a) @al3xandru

Re: Cassandra 2.0 with Hadoop 2.x?

I am as well. Thanks, Steve On Thu, Feb 6, 2014 at 4:13 PM, Alex Popescu wrote: > > On Thu, Feb 6, 2014 at 3:50 PM, Clint Kelly wrote: > >> I can >> post a link if anyone is curious (once it is done). >> > > I'm curious... thanks > > > -- > > :- a) > > > @al3xandru > -- Steve Robenalt So

Re: Cassandra 2.0 with Hadoop 2.x?

Okay neat, hopefully it will look reasonable by the end of the month or so! :) On Thu, Feb 6, 2014 at 4:15 PM, Steven A Robenalt wrote: > I am as well. > > Thanks, > Steve > > > > On Thu, Feb 6, 2014 at 4:13 PM, Alex Popescu wrote: >> >> >> On Thu, Feb 6, 2014 at 3:50 PM, Clint Kelly wrote: >>

Re: exceptions all around in clean cluster

On Thu, Feb 6, 2014 at 8:39 AM, Ondřej Černoš wrote: > Update: I dropped the keyspace, the system keyspace, deleted all the data > and started from fresh state. Now it behaves correctly. The previously > reported state is therefore the result of the keyspace being dropped > beforehand and recreat

Re: exceptions all around in clean cluster

This is a known issue until Cassandra 2.1 https://issues.apache.org/jira/browse/CASSANDRA-5202 -Tupshin On Feb 6, 2014 10:05 PM, "Robert Coli" wrote: > On Thu, Feb 6, 2014 at 8:39 AM, Ondřej Černoš wrote: > >> Update: I dropped the keyspace, the system keyspace, deleted all the data >> and sta

Re: CQL list command

On Thu, Feb 6, 2014 at 9:01 PM, Andrew Cobley wrote: > I often use the CLI command LIST for debugging or when teaching students > showing them what's going on under the hood of CQL. I see that CLI swill be > removed in Cassandra 3 and we will lose this ability. It would be nice if > CQL retai