Re: RandomPartitioner to Murmur3Partitioner

2013-01-03 Thread Edward Capriolo
By the way "10% faster" does not necessarily mean 10% more requests. https://issues.apache.org/jira/browse/CASSANDRA-2975 https://issues.apache.org/jira/browse/CASSANDRA-3772 Also if you follow the tickets "My tests show that Murmur3Partitioner actually is worse than MD5 with high cardinalit

Re: RandomPartitioner to Murmur3Partitioner

2013-01-03 Thread Sylvain Lebresne
On Thu, Jan 3, 2013 at 10:21 AM, Alain RODRIGUEZ wrote: > > Does this mean that there absolutely no way to switch to the new > partitioner for people that are already using Cassandra ? > Yes, that is what this means. -- Sylvain

Re: RandomPartitioner and the token limits

2012-10-08 Thread aaron morton
AFAIK in the code the minimum exclusive value token is -1, so as a signed integer the maxmium value is 2**127 Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 4/10/2012, at 3:19 AM, Carlos Pérez Miguel wrote: > Hello, > > Reading the wiki

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-25 Thread Safdar Kureishy
Got it. Thanks Jake. Will do. Safdar On Mon, Jun 25, 2012 at 4:16 PM, Jake Luciani wrote: > Hi Sarfar, > > Yes you should make it a multiple. The issue is each shard 'sticks' to a > given node but there is no way to guarantee 5 random keys will equally > distribute across 5 nodes. The idea

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-25 Thread Jake Luciani
Hi Sarfar, Yes you should make it a multiple. The issue is each shard 'sticks' to a given node but there is no way to guarantee 5 random keys will equally distribute across 5 nodes. The idea is eventually they will as you add more and more keys. So increasing shards at once can make that happe

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-24 Thread Safdar Kureishy
Hi Jake, Thanks. Yes, I forgot to mention also that I had raised the solandra.shards.at.once param from 4 to 5 (to match the # of nodes). Should I have raised it to 10 or 15 (multiple of 5)? I have added all the documents that I needed to the index now. It appears the distribution became more even

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-24 Thread Jake Luciani
Hi Safdar, If you want to get better utilization of the cluster raise the solandra.shards.at.once param in solandra.properties -Jake On Sun, Jun 24, 2012 at 11:00 AM, Safdar Kureishy wrote: > Hi, > > I've searched online but was unable to find any leads for the problem > below. This mailing

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-24 Thread Safdar Kureishy
Thanks. Oh, I forgot to mention that I'm using cassandra 1.1.0-beta2...in case that question comes up. Hoping someone can offer some more feedback on the likelyhood of this behavior ... Thanks again, Safdar On Jun 24, 2012 9:22 PM, "Dave Brosius" wrote: > Well it sounds like this doesn't apply t

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-24 Thread Dave Brosius
Well it sounds like this doesn't apply to you. if you had set up your column family in cql as PRIMARY KEY (domain_name, path) or something like that and where looking at lots and lots of url pages (domain_name + path), but from a very small number domain_names, then the partitioner be

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-24 Thread Safdar Kureishy
Hi Dave, Would you mind elaborating a bit more on that, preferably with an example? AFAIK, Solandra uses the unique id of the Solr document as the input for calculating the md5 hash for shard/node assignment. In this case the ids are just millions of varied web URLs that do *not* adhere to any reg

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-24 Thread Dave Brosius
If i read what you are saying, you are _not_ using composite keys? That's one thing that could do it, if the first part of the composite key had a very very low cardinality. On 06/24/2012 11:00 AM, Safdar Kureishy wrote: Hi, I've searched online but was unable to find any leads for the proble

Re: RandomPartitioner is providing a very skewed distribution of keys across a 5-node Solandra cluster

2012-06-24 Thread Safdar Kureishy
An additional detail is that the CPU utilization on those nodes is proportional to the load below, so machines 9.9.9.1 and 9.9.9.3 experience a fraction of CPU load as compared to the remaining 3 nodes. This might further point to the possibility that the keys are hashing minimally to the token ran

Re: RandomPartitioner

2011-02-14 Thread Matthew Dennis
nodes contain data for (prevTokenInRing, nodesOwnToken] (i.e. exclusive from previous token to inclusive of the nodes token). So .179 will contain things that hash in the range (152896308109140433971537345591636551711,0] and .12 will contain things that hash in range (0,152896308109140433971537345

Re: RandomPartitioner

2011-02-14 Thread mcasandra
I installed cassandra and started it in multi-node. I set the InitialToken to 0. I ran nodetool and see: $ nodetool -h localhost ring Address Status State LoadOwnsToken 152896308109140433971537345591636551711 172.16.

Re: RandomPartitioner

2011-02-14 Thread mcasandra
Dan Kuebrich wrote: > > You may find this part of the wiki helpful: > http://wiki.apache.org/cassandra/Operations#Range_changes > > "If you explicitly specify an InitialToken in the configuration, the new > node will bootstrap to that position on the ring. Otherwise, it will pick > a > Token th

Re: RandomPartitioner

2011-02-14 Thread Dan Kuebrich
You may find this part of the wiki helpful: http://wiki.apache.org/cassandra/Operations#Range_changes "If you explicitly specify an InitialToken in the configuration, the new node will bootstrap to that position on the ring. Otherwise, it will pick a Token that will give it half the keys from the

Re: RandomPartitioner doubts

2010-04-21 Thread Jonathan Ellis
For each "page" of results, start with the key that was last in the previous iteration, and you will get all the keys back. The order is random but consistent. On Wed, Apr 21, 2010 at 7:55 PM, Lucas Di Pentima wrote: > Hello, > > I'm using Cassandra 0.6.1 and ruby's library. I did some tests on