RE: Advice on memory warning

2013-04-21 Thread moshe.kranc
My experience (running C* 1.2.2): 1. I also observe that this occurs during compaction. 2. I have never yet seen a node recover from this state. Once it starts complaining about heap, it starts a death spiral, i.e., futile attempts to fix the situation. Eventually the node starts running GC fo

Re: Prepared Statement - cache duration (CQL3 - Cassandra 1.2.4)

2013-04-21 Thread Stuart Broad
Hi Sorin, Thanks very much for your response. From the sounds of it I think I can share the prepared statements as long as I handle the case when they disappear (out of LSU cache or due to server restart). To identify this I think I need to catch InvalidRequestException and look at the message (

Re: Prepared Statement - cache duration (CQL3 - Cassandra 1.2.4)

2013-04-21 Thread Sorin Manolache
On 2013-04-19 13:57, Stuart Broad wrote: Hi, I am using Cassandra.Client prepare_cql3_query/execute_prepared_cql3_query to create and run some prepared statements. It is working well but I am unclear as to how long the server side 'caches' the prepared statements. Should a prepared statement b

Re: Advice on memory warning

2013-04-21 Thread Derek Williams
The CMS compaction threshold is usually set to 75% as well, it might help to set it lower to 70% to see if that resolves these warnings as Cassandra will start CMS GC before it hits the 75% warning. There is also a setting to lower the max amount of memory used for compacting each row. This may ca

Re: Ec2Snitch to Ec2MultiRegionSnitch

2013-04-21 Thread aaron morton
> So I guess we have to switch to Ec2MultiRegionSnitch. It depends on how you are connecting the regions. If the nodes can directly communicate with each other, say through a VPN, you may not need to change it. If they are behind a NAT you will need to use it. When you change the snitch test f

Re: How to make compaction run faster?

2013-04-21 Thread aaron morton
> You are suggesting to go back to STCS and increase the compaction_throughput > step by step to see if compaction catch up with write traffic? As a personal approach, when so many config settings are changed it becomes impossible to understand cause and effect. So I try to return to a known ba

Re: Using map type with composite primary key causes significant performance decrease

2013-04-21 Thread aaron morton
> Write performance decreases. > Check the logs for WARN messages from the GCInspector. With 2Gb and only 2 Cores you may be seeing ParNew compaction which pauses the server. > Sometimes I have to wait 3-4 seconds to get a count even though there're only > couple of thousand small entries in

Re: How to configure Cassandra params to handle heavy write, light read, short TTL scenario

2013-04-21 Thread aaron morton
Use nodetool cfhistograms to get a better understanding of the latency. It will also tell you the number of SSTables involved in the read, if it's above 3 it may indicate the data model needs changes. If you have rows written to for a long time they can become very fragmented. Also check the d

How to find total number of rows in Cassandra databaase?

2013-04-21 Thread Techy Teck
I have inserted 1000 rows in Cassandra database. Now I am trying to find out how many rows have been inserted in Cassandra database using the CLI mode. In rdbms, I can do this sql- * SELECT count(*) from TABLE;* And this will give me total count for that table; How to do the same thing i

Re: Unable to drop secondary index

2013-04-21 Thread aaron morton
>> [default@production] describe Users; show schema; is the cli equivalent of describe in cqlsh. There was a schema related issue in 1.1 CASSANDRA-4561 that was fixed. This is a tricky one to diagnose remotely. I could try using nodetool resetlocalschema on each node, it's just wild guess incas

Re: Moving cluster

2013-04-21 Thread aaron morton
Sorry i do not understand you question. What are the two solutions ? Cheers - Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 20/04/2013, at 3:43 AM, Kais Ahmed wrote: > Hello and thank you for your answers. > > The first

Re: Key-Token mapping in cassandra

2013-04-21 Thread aaron morton
> There are many more such CFs all with prefixes of "grpId". By hashing grpId > to cassandra's token, I thought we can co-locate all the group's data into > one set of replicated nodes. > > Is there a way to achieve this? No. Rows with the same row key are stored on the same nodes. Rows with d

Re: How to find total number of rows in Cassandra databaase?

2013-04-21 Thread Utkarsh Sengar
Using cqlsh you can do: SELECT COUNT(*) FROM columnfamily LIMIT 5000; Does that help? Read more: http://www.datastax.com/docs/1.0/references/cql/SELECT Thanks, -Utkarsh On Sun, Apr 21, 2013 at 1:04 PM, Techy Teck wrote: > I have inserted 1000 rows in Cassandra database. Now I am trying to

Re: Batch get queries

2013-04-21 Thread aaron morton
> This is very acceptable but wanted to get everyone's take as I have seen > messages about this "starving" the request pool. The issue with sending large mutli gets or batch mutations is that it can reduce overall request throughput. Every row in your 10K multi becomes RF number of tasks that

Re: index filter

2013-04-21 Thread aaron morton
In 1.2 there is also a -Sample file which contains a sample of -Index at the last index_interval. This is done to (dramatically) improve startup time. Cheers - Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 20/04/2013, at 7

Re: How to find total number of rows in Cassandra databaase?

2013-04-21 Thread Techy Teck
Yeah it helps a lot. I always have this doubt with me. What is the difference between CLI and CQL? On Sun, Apr 21, 2013 at 1:30 PM, Utkarsh Sengar wrote: > Using cqlsh you can do: > > SELECT COUNT(*) FROM columnfamily LIMIT 5000; > > Does that help? > > Read more: http://www.datastax.com/docs/1

Re: differences between DataStax Community Edition and Cassandra package

2013-04-21 Thread aaron morton
https://www.getpantheon.com/news/inside-pantheon-valhalla-filesystem http://weblog.leapster.org/archives/250-cassafs-a-fuse-based-filesystem-using-apache-cassandra-as-a-backend.html Cheers - Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpi

Re: Datastax Java Driver connection issue

2013-04-21 Thread aaron morton
Make sure that the server rpc_address is set to 127.0.0.1 Cheers - Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 20/04/2013, at 1:47 PM, Techy Teck wrote: > I am also running into this problem. I have already enabled > st

Re: Building SSTables using SSTableSimpleUnsortedWriter (v. 1.2.3)

2013-04-21 Thread aaron morton
The key to your problem is likely the row key. Take a look in at the table schema / sample data in the cassandra-cli to see how CQL uses composites also http://thelastpickle.com/2013/01/11/primary-keys-in-cql/ The simple thing to do is use COMPACT STORAGE but that may not suite all use cases

Re: Moving cluster

2013-04-21 Thread Michael Theroux
I believe the two solutions that are being referred to is the "lift and shift" vs. upgrading by replacing a node and letting it restore from the cluster. I don't think there are any more "risks" per-say on the upgrading by replacing, as long as you can make sure your new node is configured prope

Re: com.datastax.driver.core.exceptions.InvalidQueryException using Datastax Java driver

2013-04-21 Thread Techy Teck
Can anyone help me out here? The thing that I wanted to know is whether I can insert into Column Family (that I created from CLI mode) using Datastax Java driver or not? As soon as I am trying to insert into Column Family which I have created in CLI mode, I always get the above exception. But If I

Re: How to find total number of rows in Cassandra databaase?

2013-04-21 Thread Utkarsh Sengar
Difference b/w cqlsh and cli is documented by the datastax guys here nicely: http://www.datastax.com/support-forums/topic/cli-vs-cql Thanks, -Utkarsh On Sun, Apr 21, 2013 at 1:39 PM, Techy Teck wrote: > Yeah it helps a lot. I always have this doubt with me. What is the > difference between CLI

Re: Building SSTables using SSTableSimpleUnsortedWriter (v. 1.2.3)

2013-04-21 Thread David McNelis
I figured that the primary key and how to define it was the issue. What I don't get is how to structure my SSTableSimpleUnsortedWriter.newRow() call to support the CQL3 style composite primary keys. It takes only a ByteBuffer as an argument... I guess I'm looking for some kind of example of a ne

Re: Thrift message length exceeded

2013-04-21 Thread Oleksandr Petrov
If you're using Cassandra 1.2.3, and new Hadoop interface, that would make a call to next(), you'll have an eternal loop reading same things all over again from your cassandra nodes (you may see it if you enable Debug output). next() is clearing key() which is required for Wide Row iteration. Set