I just closed https://github.com/basho/riak/issues/667
riak-2.0.5 will address it Thanks for the report and sorry for the inconvenience. On 21 Jan 2015, at 19:28, Alexander Popov <mogada...@gmail.com> wrote: > Actually, problematic node is production. but when shit happens i was migrate > all data ( export/import) to clean setup. > so i can experiment with old node. > It have not source installation, from package. > I will try compile it tomorrow. > > On Wed, Jan 21, 2015 at 9:01 PM, Russell Brown <russell.br...@me.com> wrote: > There’s a branch of riak_dt here https://github.com/basho/riak_dt/pull/111 > > Since you’re running against a source install (I assume in development, or > test?) would you be so kind as to try this branch and see if it solves your > issue? > > Many thanks > > Russell > > On 20 Jan 2015, at 15:12, Alexander Popov <mogada...@gmail.com> wrote: > > > Thanks. > > > > On Tue, Jan 20, 2015 at 5:07 PM, Russell Brown <russell.br...@me.com> wrote: > > FYI https://github.com/basho/riak/issues/667 > > > > I’ll get to work on it at once, we might be able to hit the 2.0.5 build > > date at the end of the week. > > > > On 20 Jan 2015, at 14:04, Russell Brown <russell.br...@me.com> wrote: > > > > > I’ll open a ticket for it. We changed the internal structure of both Maps > > > and Sets, and tested that they were backwards compatible, but clearly > > > missed something. > > > > > > On 20 Jan 2015, at 13:54, Alexander Popov <mogada...@gmail.com> wrote: > > > > > >> Upgraded riak 2.0.2-> 2.0.4 > > >> trying to add nodes to cluster( was single node ) > > >> all new nodes located on another host, but multi-nodes installation from > > >> source > > >> each new node have config settings tuned like > > >> > > >> nodename = riak1@10.0.0.28 > > >> platform_data_dir = /var/lib/riak1 > > >> listener.http.internal = 0.0.0.0:18098 > > >> listener.protobuf.internal = 0.0.0.0:18087 > > >> handoff.port = 18099 > > >> search.solr.port = 18093 > > >> search.solr.jmx_port = 18985 > > >> search=on > > >> > > >> after adding this nodes to cluster for long time was nothing was > > >> transferred for new nodes. > > >> I was down this nodes and force-remove it from cluster. > > >> > > >> After this strange things happens. > > >> On new nodes: > > >> Even after restart cleaning entire data dir and start again, in folder > > >> yz indexes somehow appears again. > > >> > > >> logs contains only > > >> 2015-01-20 13:15:04.815 [info] <0.352.0>@riak_core:wait_for_service:483 > > >> Wait complete for service riak_kv (10 seconds) > > >> 2015-01-20 13:15:06.660 [info] <0.547.0>@yz_index:local_create:189 > > >> Created index contacts_index with schema contacts > > >> 2015-01-20 13:15:08.683 [info] <0.547.0>@yz_index:local_create:189 > > >> Created index users_index with schema users > > >> > > >> > > >> Main Node now have bunch of errors on start and periodically after: > > >> > > >> 015-01-20 13:29:00.382 [error] > > >> <0.871.0>@riak_core_vnode:vnode_command:348 riak_kv_vnode command failed > > >> {{badrecord,dict},[{dict,filter_dict,2,[{file,"dict.erl"},{line,464}]},{riak_dt_map,'-filter_unique/4-fun-1-',4,[{file,"src/ria > > >> k_dt_map.erl"},{line,466}]},{sets,fold_bucket,3,[{file,"sets.erl"},{line,313}]},{sets,fold_seg,4,[{file,"sets.erl"},{line,309}]},{sets,fold_segs,4,[{file,"sets.erl"},{line,305}]},{riak_dt_map,merge,2,[{file,"src/riak_dt_map.erl"},{l > > >> ine,454}]},{riak_kv_crdt,'-merge_value/2-fun-0-',5,[{file,"src/riak_kv_crdt.erl"},{line,204}]},{orddict,update,4,[{file,"orddict.erl"},{line,170}]}]} > > >> 2015-01-20 13:29:00.382 [error] > > >> <0.650.0>@riak_core_vnode:vnode_command:348 riak_kv_vnode command failed > > >> {{badrecord,dict},[{dict,filter_dict,2,[{file,"dict.erl"},{line,464}]},{riak_dt_map,'-filter_unique/4-fun-1-',4,[{file,"src/ria > > >> k_dt_map.erl"},{line,466}]},{sets,fold_bucket,3,[{file,"sets.erl"},{line,313}]},{sets,fold_seg,4,[{file,"sets.erl"},{line,309}]},{sets,fold_segs,4,[{file,"sets.erl"},{line,305}]},{riak_dt_map,merge,2,[{file,"src/riak_dt_map.erl"},{l > > >> ine,454}]},{riak_kv_crdt,'-merge_value/2-fun-0-',5,[{file,"src/riak_kv_crdt.erl"},{line,204}]},{orddict,update,4,[{file,"orddict.erl"},{line,170}]}]} > > >> 2015-01-20 13:29:00.383 [error] <0.4520.0> gen_fsm <0.4520.0> in state > > >> waiting_local_vnode terminated with reason: no case clause matching > > >> {vnode_error,{{badrecord,dict},[{dict,filter_dict,2,[{file,"dict.erl"},{line,464}]},{riak_dt_ > > >> map,'-filter_unique/4-fun-1-',4,[{file,"src/riak_dt_map.erl"},{line,466}]},{sets,fold_bucket,3,[{file,"sets.erl"},{line,313}]},{sets,fold_seg,4,[{file,"sets.erl"},{line,309}]},{sets,fold_segs,4,[{file,"sets.erl"},{line,305}]},{riak_ > > >> dt_map,merge,2,[{file,"src/riak_dt_map.erl"},{line,454}]},{riak_kv_crdt,'-merge_value/2-fun-0-',5,[{file,"src/riak_kv_crdt.erl"},{line,204}]},{orddict,update,4,[{file,"or..."},...]}]}} > > >> in riak_kv_put_fsm:waiting_local_vnode/2 line 5 > > >> 67 > > >> 2015-01-20 13:29:00.384 [error] <0.4520.0> CRASH REPORT Process > > >> <0.4520.0> with 0 neighbours exited with reason: no case clause matching > > >> {vnode_error,{{badrecord,dict},[{dict,filter_dict,2,[{file,"dict.erl"},{line,464}]},{riak_dt_ma > > >> p,'-filter_unique/4-fun-1-',4,[{file,"src/riak_dt_map.erl"},{line,466}]},{sets,fold_bucket,3,[{file,"sets.erl"},{line,313}]},{sets,fold_seg,4,[{file,"sets.erl"},{line,309}]},{sets,fold_segs,4,[{file,"sets.erl"},{line,305}]},{riak_dt > > >> _map,merge,2,[{file,"src/riak_dt_map.erl"},{line,454}]},{riak_kv_crdt,'-merge_value/2-fun-0-',5,[{file,"src/riak_kv_crdt.erl"},{line,204}]},{orddict,update,4,[{file,"or..."},...]}]}} > > >> in riak_kv_put_fsm:waiting_local_vnode/2 line 567 > > >> in gen_fsm:terminate/7 line 622 > > >> 2015-01-20 13:29:00.384 [error] <0.4521.0> gen_fsm <0.4521.0> in state > > >> waiting_local_vnode terminated with reason: no case clause matching > > >> {vnode_error,{{badrecord,dict},[{dict,filter_dict,2,[{file,"dict.erl"},{line,464}]},{riak_dt_ > > >> map,'-filter_unique/4-fun-1-',4,[{file,"src/riak_dt_map.erl"},{line,466}]},{sets,fold_bucket,3,[{file,"sets.erl"},{line,313}]},{sets,fold_seg,4,[{file,"sets.erl"},{line,309}]},{sets,fold_segs,4,[{file,"sets.erl"},{line,305}]},{riak_ > > >> dt_map,merge,2,[{file,"src/riak_dt_map.erl"},{line,454}]},{riak_kv_crdt,'-merge_value/2-fun-0-',5,[{file,"src/riak_kv_crdt.erl"},{line,204}]},{orddict,update,4,[{file,"or..."},...]}]}} > > >> in riak_kv_put_fsm:waiting_local_vnode/2 line 5 > > >> 67 > > >> 2015-01-20 13:29:00.384 [error] <0.4521.0> CRASH REPORT Process > > >> <0.4521.0> with 0 neighbours exited with reason: no case clause matching > > >> {vnode_error,{{badrecord,dict},[{dict,filter_dict,2,[{file,"dict.erl"},{line,464}]},{riak_dt_ma > > >> p,'-filter_unique/4-fun-1-',4,[{file,"src/riak_dt_map.erl"},{line,466}]},{sets,fold_bucket,3,[{file,"sets.erl"},{line,313}]},{sets,fold_seg,4,[{file,"sets.erl"},{line,309}]},{sets,fold_segs,4,[{file,"sets.erl"},{line,305}]},{riak_dt > > >> _map,merge,2,[{file,"src/riak_dt_map.erl"},{line,454}]},{riak_kv_crdt,'-merge_value/2-fun-0-',5,[{file,"src/riak_kv_crdt.erl"},{line,204}]},{orddict,update,4,[{file,"or..."},...]}]}} > > >> in riak_kv_put_fsm:waiting_local_vnode/2 line 567 > > >> in gen_fsm:terminate/7 line 622 > > >> 2015-01-20 13:29:00.385 [error] <0.871.0> gen_fsm <0.871.0> in state > > >> active terminated with reason: bad record dict in dict:filter_dict/2 > > >> line 464 > > >> 2015-01-20 13:29:00.385 [error] <0.871.0> CRASH REPORT Process <0.871.0> > > >> with 1 neighbours exited with reason: bad record dict in > > >> dict:filter_dict/2 line 464 in gen_fsm:terminate/7 line 622 > > >> 2015-01-20 13:29:00.385 [error] <0.167.0> Supervisor riak_core_vnode_sup > > >> had child undefined started with {riak_core_vnode,start_link,undefined} > > >> at <0.871.0> exit with reason bad record dict in dict:filter_dict/2 line > > >> 464 in context > > >> child_terminated > > >> 2015-01-20 13:29:00.385 [error] <0.988.0> gen_fsm <0.988.0> in state > > >> ready terminated with reason: bad record dict in dict:filter_dict/2 line > > >> 464 > > >> 2015-01-20 13:29:00.385 [error] <0.988.0> CRASH REPORT Process <0.988.0> > > >> with 10 neighbours exited with reason: bad record dict in > > >> dict:filter_dict/2 line 464 in gen_fsm:terminate/7 line 622 > > >> 2015-01-20 13:29:00.385 [error] <0.650.0> gen_fsm <0.650.0> in state > > >> active terminated with reason: bad record dict in dict:filter_dict/2 > > >> line 464 > > >> 2015-01-20 13:29:00.386 [error] <0.650.0> CRASH REPORT Process <0.650.0> > > >> with 1 neighbours exited with reason: bad record dict in > > >> dict:filter_dict/2 line 464 in gen_fsm:terminate/7 line 622 > > >> 2015-01-20 13:29:00.386 [error] <0.989.0> Supervisor > > >> {<0.989.0>,poolboy_sup} had child riak_core_vnode_worker started with > > >> riak_core_vnode_worker:start_link([{worker_module,riak_core_vnode_worker},{worker_args,[593735040165679310520 > > >> 246963290989976735222595584,...]},...]) at undefined exit with reason > > >> bad record dict in dict:filter_dict/2 line 464 in context shutdown_error > > >> 2015-01-20 13:29:00.386 [error] <0.989.0> gen_server <0.989.0> > > >> terminated with reason: bad record dict in dict:filter_dict/2 line 464 > > >> 2015-01-20 13:29:00.386 [error] <0.989.0> CRASH REPORT Process <0.989.0> > > >> with 0 neighbours exited with reason: bad record dict in > > >> dict:filter_dict/2 line 464 in gen_server:terminate/6 line 744 > > >> 2015-01-20 13:29:00.386 [error] <0.167.0> Supervisor riak_core_vnode_sup > > >> had child undefined started with {riak_core_vnode,start_link,undefined} > > >> at <0.650.0> exit with reason bad record dict in dict:filter_dict/2 line > > >> 464 in context > > >> child_terminated > > >> 2015-01-20 13:29:00.386 [error] <0.696.0> gen_fsm <0.696.0> in state > > >> ready terminated with reason: bad record dict in dict:filter_dict/2 line > > >> 464 > > >> 2015-01-20 13:29:00.386 [error] <0.696.0> CRASH REPORT Process <0.696.0> > > >> with 10 neighbours exited with reason: bad record dict in > > >> dict:filter_dict/2 line 464 in gen_fsm:terminate/7 line 622 > > >> 2015-01-20 13:29:00.386 [error] <0.702.0> Supervisor > > >> {<0.702.0>,poolboy_sup} had child riak_core_vnode_worker started with > > >> riak_core_vnode_worker:start_link([{worker_module,riak_core_vnode_worker},{worker_args,[228359630832953580969 > > >> 325755111919221821239459840,...]},...]) at undefined exit with reason > > >> bad record dict in dict:filter_dict/2 line 464 in context shutdown_error > > >> 2015-01-20 13:29:00.386 [error] <0.702.0> gen_server <0.702.0> > > >> terminated with reason: bad record dict in dict:filter_dict/2 line 464 > > >> 2015-01-20 13:29:00.386 [error] <0.702.0> CRASH REPORT Process <0.702.0> > > >> with 0 neighbours exited with reason: bad record dict in > > >> dict:filter_dict/2 line 464 in gen_server:terminate/6 line 744 > > >> > > >> > > >> > > >> any advice? > > >> > > >> _______________________________________________ > > >> riak-users mailing list > > >> riak-users@lists.basho.com > > >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > > > > _______________________________________________ > > > riak-users mailing list > > > riak-users@lists.basho.com > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > _______________________________________________ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
