Re: Why "select count("*) from .." hangs ?

2014-03-26 Thread shahab
Thanks for the hints. I got a better picture of how to deal with "count" queries. On Tue, Mar 25, 2014 at 7:01 PM, Robert Coli wrote: > On Tue, Mar 25, 2014 at 8:36 AM, shahab wrote: > >> But after iteration 8, (i.e. inserting 150 sensor data), the >> "select count(') ...) throws time-o

Thrift -> CQL

2014-03-26 Thread rubbish me
Hi all, We have been using Cassandra for more than 3 years and now we have a cluster in production still running on 1.1.x contains dynamic-columned column-families - with hector as client. We are trying to update to the latest 1.2.x and considering to use datastax client in order to utilise

Re: Thrift -> CQL

2014-03-26 Thread Peter Lin
Hector has round robin and failover. Is there a particular kind of failover you're looking for? by default Hector will try another node if the first node it connects to is down. It's been that way since the 1.x client if I'm not mistaken. On Wed, Mar 26, 2014 at 9:41 AM, rubbish me wrote: > Hi

Re: Thrift -> CQL

2014-03-26 Thread Sylvain Lebresne
> > - *Is there any way to do insert / update at all on a good old wide cf > using CQL? Based on what we read back out, we have tried:* > > > INSERT INTO cf_name(key, column1, value) VALUES ('key1', > 'columnName1','columnValue2') > > > But we ended up with "Unknown identifier column1" > What d

unstable write performance

2014-03-26 Thread Jiaan Zeng
Hi, I am doing some performance benchmarks in a *single* node cassandra 1.2.4. BTW, the machine is dedicated to run one cassandra instance. The workload is 100% write. The throughput varies dramatically and sometimes even drops to 0. I have tried several things below and still got the same observa

Re: Kernel keeps killing cassandra process - OOM

2014-03-26 Thread prem yadav
Thanks Robert. That seems to be the issue. however the fix mentioned there doesn't work. I downgraded Java to jdk6_37 and that seems to have done the trick. Thanks for pointing me to that Jira ticket. On Mon, Mar 24, 2014 at 6:48 PM, Robert Coli wrote: > On Mon, Mar 24, 2014 at 4:11 AM, prem ya

nodetool scrub throws exception FileAlreadyExistsException

2014-03-26 Thread Donald Smith
% time nodetool scrub -s as_reports data_report_info_2011 xss = -ea -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms8192M -Xmx8192M -Xmn2048M -XX:+HeapDumpOnOutOfMemoryError -Xss256k Exception in thread "main" FSWriteError in /mnt/cas

Re: unstable write performance

2014-03-26 Thread Marcin Cabaj
ParNew GC (used by default in cassandra) uses 'stop-the-world' algorithm, which means your application has to be stopped to do gc. You can run jstat command to monitor gc activity and check if your write performance is related to GC, eg: $ jstat -gc 1s But it shouldn't drop throughtput to 0 ops/s.

memory usage spikes

2014-03-26 Thread prem yadav
Hi, in another thread, I has mentioned that we had issue with Cassandra getting killed by kernel due to OOM. Downgrading to jdk6_37 seems to have fixed it. However, even now, after every couple of hours, the nodes are showing a spike in memory usage. For ex: on a 8GB ram machine, once the usage re

Question about how compaction and partition keys interact

2014-03-26 Thread Donald Smith
In CQL we need to decide between using ((customer_id,type),date) as the CQL primary key for a reporting table, versus ((customer_id,date),type). We store reports for every day. If we use (customer_id,type) as the partition key (physical key), then we have a WIDE ROW where each date's data is s

Re: memory usage spikes

2014-03-26 Thread Marcin Cabaj
Hi, RSS or VIRT? Could you paste output of: $ ps -p `jps | awk '/CassandraDaemon/ {print $1}'` uww please? On Wed, Mar 26, 2014 at 5:20 PM, prem yadav wrote: > Hi, > in another thread, I has mentioned that we had issue with Cassandra > getting killed by kernel due to OOM. Downgrading to jdk6_

Re: memory usage spikes

2014-03-26 Thread prem yadav
here: ps -p `/usr/java/jdk1.6.0_37/bin/jps | awk '/Dse/ {print $1}'` uww SER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND 497 20450 0.9 31.0 4727620 2502644 ? SLl 06:55 3:28 /usr/java/jdk1.6.0_37//bin/java -ea -javaagent:/usr/share/dse/cassandra/lib/jamm-0.2.5.

RE: memory usage spikes

2014-03-26 Thread Donald Smith
Prem, Did you follow the instructions at http://www.datastax.com/documentation/cassandra/2.0/cassandra/install/installRecommendSettings.html?scroll=reference_ds_sxl_gf3_2k And did you install jna-3.2.7.jar into /usr/share/java, as per http://www.datastax.com/documentation/cassandra/2.0/mobile/c

Re: memory usage spikes

2014-03-26 Thread prem yadav
Thanks Don, Yes have followed those steps. Except jna. The version I am using is 3.2.4. The link you have shared is for Cassandra 2.0. I am using 1.1. Let me install jna 3.2.7 and see if that helps. Thanks On Wed, Mar 26, 2014 at 5:38 PM, Donald Smith < donald.sm...@audiencescience.com> wrote:

Re: memory usage spikes

2014-03-26 Thread Marcin Cabaj
You can try to dump memory mapping of the cassandra process during spike using pmap, eg: $ pmap -x and paste here. On Wed, Mar 26, 2014 at 5:47 PM, prem yadav wrote: > Thanks Don, > Yes have followed those steps. Except jna. The version I am using is > 3.2.4. The link you have shared is for Ca

Re: Question about how compaction and partition keys interact

2014-03-26 Thread Jonathan Lacefield
Don, What is the underlying question? Are trying to figure out what's going to be faster for reads or are you really concerned about storage? The recommendation typically provided is to suggest that tables are modeled based on query access, to enable the fastest read performance. In your

RE: Question about how compaction and partition keys interact

2014-03-26 Thread Donald Smith
My underlying question is about the effects of the partitioning key on compaction. Specifically, would having date as part of the partitioning key make compaction easier (because compaction wouldn't have to merge wide rows over multiple days)? According to the person on irc, it wouldn't make

Re: Kernel keeps killing cassandra process - OOM

2014-03-26 Thread Robert Coli
On Wed, Mar 26, 2014 at 8:35 AM, prem yadav wrote: > Thanks Robert. That seems to be the issue. however the fix mentioned there > doesn't work. I downgraded Java to jdk6_37 and that seems to have done the > trick. Thanks for pointing me to that Jira ticket. > If the workaround on that ticket doe

Rearranging commitlog and saved_cache directories on a live cluster.

2014-03-26 Thread Redmumba
I currently have a group of about 51 hosts on Cassandra 1.2.15, 17 in each EC2 AZ (us-east-1a, 1d, 1e). These are m2.4xlarge machines, so they have basically a 10G partition on /, and then two ~800G partitions on /dev/sdb and /dev/sdc. When I first started, I was expecting the commitlog to take u

Re: unstable write performance

2014-03-26 Thread ssiv...@gmail.com
280 sec: 865658 operations; 2661.5 current ops/sec; [INSERT AverageLatency(us)=3640.16] 290 sec: 865658 operations; 0 current ops/sec; It also may indicate that C* trying to finished active tasks and your write requests have been in the queue all 10 sec. Try to monitor C* doing*$watch nodetool

Re: Why "select count("*) from .." hangs ?

2014-03-26 Thread Arthur Zubarev
I faced the same nuance in my early days with C*, specifically I got RPC timeouts on selecting data from CFs larger than 300 GB. The typical remedy is to implement paging. So instead of using the CLI resort to a custom built client app. Regards, Arthur From: shahab Sent: Wednesday, March 26