Hi, This is just a quick reply since this is somewhat a current topic on the ML.
On 24 May 2017, at 12:57, Denis Gudtsov <dg.65...@gmail.com> wrote: > Hello > > We have 6-nodes cluster with ring size 128 configured. The problem is that > two partitions has replicas only on two nodes rather than three as required > (n_val=3). We have tried several times to clean leveldb and ring directories > and then rebuild cluster, but this issue is still present. There was a fairly long discussion about this very issue recently (see http://lists.basho.com/pipermail/riak-users_lists.basho.com/2017-May/019281.html) I ran a little code and the following {RingSize, NodeCount, IsViolated} tuples were the result. If you built any of these clusters from scratch (i.e. you started NodeCount nodes, and used riak-admin cluster join, riak-admin cluster plan, riak-admin cluster commit to create a cluster of NodeCount from scratch) then you have tail violations in your ring. [{16,3,true}, {16,5,true}, {16,7,true}, {16,13,true}, {16,14,true}, {32,3,true}, {32,5,true}, {32,6,true}, {32,10,true}, {64,3,true}, {64,7,true}, {64,9,true}, {128,3,true}, {128,5,true}, {128,6,true}, {128,7,true}, {128,9,true}, {128,14,true}, {256,3,true}, {256,5,true}, {256,11,true}, {512,3,true}, {512,5,true}, {512,6,true}, {512,7,true}, {512,10,true}] > How can we diagnose where the issue is and fix it? WRT your problem, a quick experiment looks like adding 2 new nodes will solve your problem, just adding one doesn’t look like it does. I tried just adding one new node and still had a single violated preflist, but I have just thrown a little experiment together so I could well be wrong. It doesn’t actually build any clusters, and uses the claim code out of context, ymmv > Is there any way how we > can assign partition to node manually? I don’t know of a way, but that would be very useful. Do you remember if this cluster was built all at once as a 6-node cluster, or has it grown over time? Have you run the command riak-admin diag ring_preflists as documented here http://docs.basho.com/riak/kv/2.2.3/setup/upgrading/checklist/#confirming-configuration-with-riaknostic? Sorry I can’t be more help Cheers Russell > > Please find output of member-status below and screen from riak control ring > status: > [root@riak01 ~]# riak-admin member-status > ================================= Membership > ================================== > Status Ring Pending Node > ------------------------------------------------------------------------------- > valid 17.2% -- 'riak@riak01. > valid 17.2% -- 'riak@riak02. > valid 16.4% -- 'riak@riak03. > valid 16.4% -- 'riak@riak04. > valid 16.4% -- 'riak@riak05. > valid 16.4% -- 'riak@riak06. > ------------------------------------------------------------------------------- > Valid:6 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 > > <http://riak-users.197444.n3.nabble.com/file/n4035179/10.png> > > Thank you. > > > > -- > View this message in context: > http://riak-users.197444.n3.nabble.com/Issues-with-partition-distribution-across-nodes-tp4035179.html > Sent from the Riak Users mailing list archive at Nabble.com. > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
