Re: compaction strategy

2011-05-09 Thread Sylvain Lebresne
On Sat, May 7, 2011 at 7:20 PM, Terje Marthinussen wrote: > This is an all ssd system. I have no problems with read/write performance > due to I/O. > I do have a potential with the crazy explosion you can get in terms of disk > use if compaction cannot keep up. > > As things falls behind and you g

Re: compaction strategy

2011-05-09 Thread David Boxenhorn
"I'm also not too much in favor of triggering major compactions, because it mostly have a nasty effect (create one huge sstable)." If that is the case, why can't major compactions create many, non-overlapping SSTables? In general, it seems to me that non-overlapping SSTables have all the advantag

Re: Adding a new node

2011-05-09 Thread aaron morton
Gossip should help them converge on the truth. Can you give an example of the different views from nodetool ring ? Also check the logs to see if there is anything been logged about endpoints. Hope that helps. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://

Re: Migrating all rows from 0.6.13 to 0.7.5 over thrift?

2011-05-09 Thread aaron morton
That was my initial thought, just wanted to see if there was anything else going on. Sounds like Henrik has a workaround so all is well. Cheers - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 9 May 2011, at 18:10, Jonathan Ellis wrote:

Re: compaction strategy

2011-05-09 Thread Terje Marthinussen
Yes, agreed. I actually think cassandra has to. And if you do not go down to that single file, how do you avoid getting into a situation where you can very realistically end up with 4-5 big sstables each having its own copy of the same data massively increasing disk requirements? Terje On Mon,

Re: RequestResponseStage Assertion Error

2011-05-09 Thread Eric tamme
On Sun, May 8, 2011 at 7:17 PM, aaron morton wrote: > What version are you on ? > > Check the nodetool ring from each node in your cluster to check they have the > same view. I am running 0.7.3. I checked nodetool ring on all hosts and it all comes back the same. I had some funky business whe

Re: RequestResponseStage Assertion Error

2011-05-09 Thread aaron morton
You can check the schema using cassandra-cli, run "describe cluster" it will tell you how many schemas are defined. I think the best approach when you discover bad schemas is to drain then stop the affected node, remove the Location, Migrations and Schema files in the System data directory, re

Re: RequestResponseStage Assertion Error

2011-05-09 Thread Eric tamme
On Mon, May 9, 2011 at 7:18 AM, aaron morton wrote: > You can check the schema using cassandra-cli, run "describe cluster" it will > tell you how many schemas are defined. > I think the best approach when you discover bad schemas is to drain then > stop the affected node, remove the Location, Migr

Re: compaction strategy

2011-05-09 Thread David Boxenhorn
If they each have their own copy of the data, then they are *not* non-overlapping! If you have non-overlapping SSTables (and you know the min/max keys), it's like having one big SSTable because you know exactly where each row is, and it becomes easy to merge a new SSTable in small batches, rather

Re: compaction strategy

2011-05-09 Thread Terje Marthinussen
Sorry, I was referring to the claim that "one big file" was a problem, not the non-overlapping part. If you never compact to a single file, you never get rid of all generations/duplicates. With non-overlapping files covering small enough token ranges, compacting down to one file is not a big issue

Re: RequestResponseStage Assertion Error

2011-05-09 Thread Jonathan Ellis
Fixed since 0.7.4. You should upgrade. https://issues.apache.org/jira/browse/CASSANDRA-2282 On Sun, May 8, 2011 at 2:37 PM, Eric tamme wrote: > I have a 4 node ring that was  setup with tokens a,b,c,d using NTS and > 2 nodes in each of 2 datacenters with a replication of DC1:1, DC2:1. > I was get

Does anyone have Cassandra running on OpenSolaris?

2011-05-09 Thread Jeffrey Kesselman
I get this error: bin/cassandra: syntax error at line 29: `system_memory_in_mb=$' unexpected Thanks JK -- It's always darkest just before you are eaten by a grue.

AW: Does anyone have Cassandra running on OpenSolaris?

2011-05-09 Thread Roland Gude
Use bash as a shell #bash bin/cassandra -f -Ursprüngliche Nachricht- Von: Jeffrey Kesselman [mailto:jef...@gmail.com] Gesendet: Montag, 9. Mai 2011 17:12 An: user@cassandra.apache.org Betreff: Does anyone have Cassandra running on OpenSolaris? I get this error: bin/cassandra: syntax

Re: Does anyone have Cassandra running on OpenSolaris?

2011-05-09 Thread Jeffrey Kesselman
Ah. That solved it. ty. On Mon, May 9, 2011 at 11:29 AM, Roland Gude wrote: > > Use bash as a shell > > #bash bin/cassandra -f > > > -Ursprüngliche Nachricht- > Von: Jeffrey Kesselman [mailto:jef...@gmail.com] > Gesendet: Montag, 9. Mai 2011 17:12 > An: user@cassandra.apache.org > Betref

Index interval tuning

2011-05-09 Thread Héctor Izquierdo Seliva
Hi everyone. I have a few sstables with around 500 million keys, and memory usage has grown a lot, I suppose because of the indexes. This sstables are comprised of skinny rows, but a lot of them. Would tuning index interval make the memory usage go down? And what would the performance hit be? I

Re: Index interval tuning

2011-05-09 Thread Peter Schuller
> I have a few sstables with around 500 million keys, and memory usage has > grown a lot, I suppose because of the indexes. This sstables are > comprised of skinny rows, but a lot of them. Would tuning index interval > make the memory usage go down? And what would the performance hit be? Assuming

Re: Index interval tuning

2011-05-09 Thread Héctor Izquierdo Seliva
El lun, 09-05-2011 a las 17:58 +0200, Peter Schuller escribió: > > I have a few sstables with around 500 million keys, and memory usage has > > grown a lot, I suppose because of the indexes. This sstables are > > comprised of skinny rows, but a lot of them. Would tuning index interval > > make the

Re: Native heap leaks?

2011-05-09 Thread Hannes Schmidt
> I have not looked into smaps before. But it actually seems odd that that > mmaped Index files are taking up so *little memory*.  Are they only a > few kb on disk? The sum of the sizes of all *-Index.db files in /var/lib/cassandra is 2924kb. > Is this a snapshot taken shortly after the process >

Re: Native heap leaks?

2011-05-09 Thread Hannes Schmidt
On Thu, May 5, 2011 at 5:56 PM, Benjamin Coverston wrote: > How many column families do you have? We have 10 key spaces, each with 2 column families. > > On 5/4/11 12:50 PM, Hannes Schmidt wrote: >> >> Hi, >> >> We are using Cassandra 0.6.12 in a cluster of 9 nodes. Each node is >> 64-bit, has 4

Re: Native heap leaks?

2011-05-09 Thread Hannes Schmidt
On Thu, May 5, 2011 at 4:16 PM, aaron morton wrote: > Hannes, >        To get a baseline of behaviour set disk_access to standard. You will > probably want to keep it like that if you want better control over the memory > on the box. I'll do a test with standard and report back. > >        Als

Re: datacenter ShardStrategy

2011-05-09 Thread Anurag Gujral
Jonathan thanks for your email. If I use datacenter shard strategy in cassandra how will it effect the ring structure of the cassandra cluster can you please explain. Thanks Anurag On Sun, May 8, 2011 at 11:13 PM, Jonathan Ellis wrote: > Step 0: Upgrade to 0.7 and read about NetworkTopologyStra

Re: New node not joining

2011-05-09 Thread Sanjeev Kulkarni
Thanks! On Sun, May 8, 2011 at 3:40 PM, aaron morton wrote: > Ah, I see the case you are talking about. > > If the node will auto bootstrap on startup if when it joins the ring: it is > not already bootstrapped, auto bootstrap is enabled, and the node is not in > it's own seed list. > > In the au

Re: Memory Usage During Read

2011-05-09 Thread Sanjeev Kulkarni
Hi Adam, We have been facing some similar issues of late. Wondering if Jonathan's suggestions worked for you. Thanks! On Sat, May 7, 2011 at 6:37 PM, Jonathan Ellis wrote: > The live:serialized size ratio depends on what your data looks like > (small columns will be less efficient than large blo

Re: datacenter ShardStrategy

2011-05-09 Thread aaron morton
If you are using 0.7 the recommended approach is to use the NetworkTopologyStrategy. Here is a recent discussion on setting the tokens in a multi DC deployment http://www.mail-archive.com/user@cassandra.apache.org/msg12898.html Can you move to 0.7 ? Cheers - Aaron Morton Free

logged out: #

2011-05-09 Thread Suan Aik Yeo
I have a Cassandra 0.7.0, 3 node cluster with logging set to DEBUG. A few days ago, and I'm not sure what triggered this, the logs started showing messages like DEBUG 17:37:30,399 logged out: # every second or so, regardless whether there was Cassandra activity. Today I just upgraded to 0.7.5,

Re: Ec2 Stress Results

2011-05-09 Thread Alex Araujo
On 5/6/11 9:47 PM, Jonathan Ellis wrote: On Fri, May 6, 2011 at 5:13 PM, Alex Araujo wrote: I raised the default MAX_HEAP setting from the AMI to 12GB (~80% of available memory). This is going to make GC pauses larger for no good reason. Good point - only doing writes at the moment. I will r

Re: logged out: #

2011-05-09 Thread Tyler Hobbs
It just means a client connection was closed. On Mon, May 9, 2011 at 5:41 PM, Suan Aik Yeo wrote: > I have a Cassandra 0.7.0, 3 node cluster with logging set to DEBUG. A few > days ago, and I'm not sure what triggered this, the logs started showing > messages like > DEBUG 17:37:30,399 logged

cassandra not reading keyspaces defined in cassandra.yaml

2011-05-09 Thread Anurag Gujral
Hi All, I have following in my cassandra.yaml keyspaces: - column_families: - column_metadata: [] column_type: Standard compare_with: BytesType gc_grace_seconds: 86400 key_cache_save_period_in_seconds: 14400 keys_cached: 0.0 max_compaction_threshold: 32 memt

Re: cassandra not reading keyspaces defined in cassandra.yaml

2011-05-09 Thread Tyler Hobbs
http://wiki.apache.org/cassandra/FAQ#no_keyspaces On Mon, May 9, 2011 at 7:51 PM, Anurag Gujral wrote: > Hi All, >I have following in my cassandra.yaml > keyspaces: > - column_families: > - column_metadata: [] > column_type: Standard > compare_with: BytesType > gc_grace_

Re: logged out: #

2011-05-09 Thread Suan Aik Yeo
Ah, must be the status check that I set up. Thanks! On Mon, May 9, 2011 at 7:42 PM, Tyler Hobbs wrote: > It just means a client connection was closed. > > > On Mon, May 9, 2011 at 5:41 PM, Suan Aik Yeo wrote: > >> I have a Cassandra 0.7.0, 3 node cluster with logging set to DEBUG. A few >> days

A query in deletioelo

2011-05-09 Thread anuya joshi
Hello, I am unclear on Why deleting a row in Cassandra does not delete a row key? Is an empty row never deleted from Column Family? It would be of great help if someone can elaborate on this. Thanks, Anuya

A query in deletion of a row

2011-05-09 Thread anuya joshi
Hello, > > I am unclear on Why deleting a row in Cassandra does not delete a row key? > Is an empty row never deleted from Column Family? > > It would be of great help if someone can elaborate on this. > > Thanks, > Anuya >

Re: A query in deletioelo

2011-05-09 Thread Jonathan Ellis
http://wiki.apache.org/cassandra/FAQ#range_ghosts On Mon, May 9, 2011 at 9:24 PM, anuya joshi wrote: > Hello, > > I am unclear on Why deleting a row in Cassandra does not delete a row key? > Is an empty row never deleted from Column Family? > > It would be of great help if someone can elaborate o

Re: Ec2 Stress Results

2011-05-09 Thread Jonathan Ellis
On Mon, May 9, 2011 at 5:58 PM, Alex Araujo > How many replicas are you writing? > > Replication factor is 3. So you're actually spot on the predicted numbers: you're pushing 20k*3=60k "raw" rows/s across your 4 machines. You might get another 10% or so from increasing memtable thresholds, but bo

Renaming cluster

2011-05-09 Thread Shaun Newman
Hi, I am trying to rename my cluster which has several keyspaces running on cassandra 0.7.5. When I try to remove the system files as suggested by http://wiki.apache.org/cassandra/FAQ#clustername_mismatch , I get "Could not read system table. Did you change partitioners?" error. If I remove all t

Re: Renaming cluster

2011-05-09 Thread aaron morton
Can you provide the full error stack, it will show where it failed when starting up. AFAIK this i the correct process. I just did a quick test on a singe 0.7 node and it could start up after removing the locations SSTables. If you go ahead with removing all the system sstables you can re-recr

Re: cassandra not reading keyspaces defined in cassandra.yaml

2011-05-09 Thread Narendra Sharma
Look for "Where are my keyspaces?" on following page: *http://wiki.apache.org/cassandra/StorageConfiguration * On Mon, May 9, 2011 at 5:51 PM, Anurag Gujral wrote: > Hi All, >I have following in my cassandra.yaml > keyspaces: > - column_families: > - column_metadata: [] > column_

Re: cassandra not reading keyspaces defined in cassandra.yaml

2011-05-09 Thread Paul Loy
http://wiki.apache.org/cassandra/FAQ#no_keyspaces On Tue, May 10, 2011 at 6:21 AM, Narendra Sharma wrote: > Look for "Where are my keyspaces?" on following page: > *http://wiki.apache.org/cassandra/StorageConfiguration > * > On Mon, May 9, 2011 at 5:51 PM, Anurag Gujral wrote: > >> Hi All, >>

Re: logged out: #

2011-05-09 Thread Stu Hood
As a side note, be aware that running with DEBUG logging enabled can make your cluster run a full order of magnitude slower. On Mon, May 9, 2011 at 6:54 PM, Suan Aik Yeo wrote: > Ah, must be the status check that I set up. Thanks! > > > On Mon, May 9, 2011 at 7:42 PM, Tyler Hobbs wrote: > >> It