Re: running two rings on the same subnet

2012-03-13 Thread aaron morton
If you are on Ubuntu it may be this http://wiki.apache.org/cassandra/FAQ#ubuntu_hangs otherwise I would look for GC problems. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 13/03/2012, at 7:53 PM, Tamar Fraenkel wrote: > Done it. Now i

Re: running two rings on the same subnet

2012-03-12 Thread Tamar Fraenkel
Done it. Now it generally runs ok, till one of the nodes get's stuck with 100% cpu and I need to reboot it. Last lines in the system.log just before are: INFO [OptionalTasks:1] 2012-03-13 07:36:43,850 MeteredFlusher.java (line 62) flushing high-traffic column family CFS(Keyspace='tok', ColumnFami

Re: running two rings on the same subnet

2012-03-06 Thread aaron morton
Reduce these settings for the CF row_cache (disable it) key_cache (disable it) Increase these settings for the CF bloom_filter_fp_chance Reduce these settings in cassandra.yaml flush_largest_memtables_at memtable_flush_queue_size sliced_buffer_size_in_kb in_memory_compaction_limit_in_mb concurre

Re: running two rings on the same subnet

2012-03-06 Thread Tamar Fraenkel
Arron, Thanks for your response. I was afraid this is the issue. Can you give me some direction regarding the fine tuning of my VMs, I would like to explore that option some more. Thanks! *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +97

Re: running two rings on the same subnet

2012-03-06 Thread aaron morton
You do not have enough memory allocated to the JVM and are suffering from excessive GC as a result. There are some tuning things you can try, but 480MB is not enough. 1GB would be a better start, 2 better than that. Consider using https://github.com/pcmanus/ccm for testing multiple instances o

Re: running two rings on the same subnet

2012-03-06 Thread Tamar Fraenkel
I have some more info, after couple of hours running the problematic node became again 100% CPU and I had to reboot it, last lines from log show it did GC: INFO [ScheduledTasks:1] 2012-03-06 10:28:00,880 GCInspector.java (line 122) GC for Copy: 203 ms for 1 collections, 185983456 used; max is 513

Re: running two rings on the same subnet

2012-03-05 Thread Tamar Fraenkel
Works.. But during the night my setup encountered a problem. I have two VMs on my cluster (running on VmWare ESXi). Each VM has1GB memory, and two Virtual Disks of 16 GB They are running on a small server with 4CPUs (2.66 GHz), and 4 GB memory (together with two other VMs) I put cassandra data on

Re: running two rings on the same subnet

2012-03-05 Thread aaron morton
Create nodes that do not share seeds, and give the clusters different names as a safety measure. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 6/03/2012, at 12:04 AM, Tamar Fraenkel wrote: > I want tow separate clusters. > Tamar Fraenke

Re: Rationale behind incrementing all tokens by one in a different datacenter (was: running two rings on the same subnet)

2012-03-05 Thread Jeremiah Jordan
There is a requirement that all nodes have a unique token. There is still one global cluster/ring that each node needs to be unique on. The logically seperate rings that NetworkTopologyStrategy puts them into is hidden from the rest of the code. -Jeremiah On 03/05/2012 05:13 AM, Hontvári Jó

Rationale behind incrementing all tokens by one in a different datacenter (was: running two rings on the same subnet)

2012-03-05 Thread Hontvári József Levente
I am thinking about the frequent example: dc1 - node1: 0 dc1 - node2: large...number dc2 - node1: 1 dc2 - node2: large...number + 1 In theory using the same tokens in dc2 as in dc1 does not significantly affect key distribution, specifically the two keys on the border will move to the next on

Re: running two rings on the same subnet

2012-03-05 Thread Tamar Fraenkel
I want tow separate clusters. *Tamar Fraenkel * Senior Software Engineer, TOK Media [image: Inline image 1] ta...@tok-media.com Tel: +972 2 6409736 Mob: +972 54 8356490 Fax: +972 2 5612956 On Mon, Mar 5, 2012 at 12:48 PM, aaron morton wrote: > Do you want to create two separate cluster

Re: running two rings on the same subnet

2012-03-05 Thread aaron morton
Do you want to create two separate clusters or a single cluster with two data centres ? If it's the later, token selection is discussed here http://www.datastax.com/docs/1.0/install/cluster_init#token-gen-cassandra > Moreover all tokens must be unique (even across datacenters), although - fro

Re: running two rings on the same subnet

2012-03-05 Thread Hontvári József Levente
You have to use PropertyFileSnitch and NetworkTopologyStrategy to create a multi-datacenter setup with two circles. You can start reading from this page: http://www.datastax.com/docs/1.0/cluster_architecture/replication#about-replica-placement-strategy Moreover all token

Re: running two rings on the same subnet

2012-03-05 Thread aaron morton
> Would the rings be separate? Yes. But I would recommend you give them different cluster names. It's a good protections against nodes accidentally joining the wrong cluster. cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 5/03/2012, at 1

running two rings on the same subnet

2012-03-05 Thread Tamar Fraenkel
Hi! I have a Cassandra cluster with two nodes nodetool ring -h localhost Address DC RackStatus State LoadOwns Token 85070591730234615865843651857942052864 10.0.0.19 datacenter1 rack1 Up Normal 488.74 KB 50.00% 0 10.0.0.28 datac