Thanks Dmitri. I’ll try it with the modified reip script!
On Wed, Aug 26, 2015 at 10:39 PM, Dmitri Zagidulin <dzagidu...@basho.com> wrote: > Hi Sujay, > > Checking into it, it seems like there is a bug with 'reip' in 1.4.2. > > You have a couple of options, then. > > 1) Since you're just trying to verify the data, you can actually restore > all the files except for the 'ring' directory. And then just form a new > cluster of 5 nodes, with different IPs (don't forget about the cookie). The > new cluster would still have the data from the old cluster, though. It will > go through a lot of partition transfers, but the data should be preserved. > > 2) You can fix the 'reip' issue yourself, by doing the following: > Edit the 'riak-admin' shell script (you can see where it lives on your OS > by doing 'which riak-admin'.) > Change the following line: > https://github.com/basho/riak/blob/riak-1.4.2/rel/files/riak-admin#L382 > (line 382) to read 'node_down_check' instead of 'node_up_check'. > Then save and re-try the 'reip' command. > > 3) Even better: upgrade the test cluster (that you're using to restore the > data) to version 1.4.12, it has the 'reip' problem fixed. And then follow > the same procedure. > > > > On Wed, Aug 26, 2015 at 2:18 AM, Sujay Mansingh <su...@editd.com> wrote: > >> Thanks for your reply Dmitri. >> >> ubuntu@test-riak-products-hetzner-04:~$ sudo riak ping >> Node 'riak@172.16.16.211' not responding to pings. >> ubuntu@test-riak-products-hetzner-04:~$ ps -ef | grep erl >> ubuntu 13786 13709 0 07:12 pts/0 00:00:00 grep --color=auto erl >> ubuntu@test-riak-products-hetzner-04:~$ sudo riak-admin reip >> riak@192.168.3.8 riak@172.16.16.211 >> Node is not running! >> ubuntu@test-riak-products-hetzner-04:~$ riak version >> 1.4.2 >> >> It’s odd, because it looks like the reip command requires the node to be >> running. >> >> Riak doesn’t seem to be running. >> >> Sujay >> >> >> >> On Tue, Aug 25, 2015 at 6:31 PM, Dmitri Zagidulin <dzagidu...@basho.com> >> wrote: >> >>> Can you paste me the command-line output of 'riak-admin reip', and the >>> error message? As far as I know, 'reip' requires the node to be not >>> running. (This was the case at least as far back as Riak 1.3, and probably >>> earlier). >>> >>> I don't think there are ways of verifying the backup while the nodes are >>> not running. I'm confident we can sort this out, though, and get them up >>> and running. >>> >>> On Tue, Aug 25, 2015 at 6:17 AM, Sujay Mansingh <su...@editd.com> wrote: >>> >>>> Thanks Dmitri. >>>> >>>> I tried that, but with no luck. I replaced the ring and bitcask data >>>> directories. >>>> >>>> But I can’t run riak-admin reip ... because it complains that riak >>>> isn’t running. >>>> >>>> However, I can’t start riak (I get the following in >>>> /var/log/riak/error.log). >>>> (The existing cluster is the 192.168.3.x range, and the new one is the >>>> 172.16.16.x range.) >>>> >>>> 2015-08-25 11:13:39.217 [error] <0.161.0> gen_server riak_core_capability >>>> terminated with reason: no function clause matching >>>> orddict:fetch('riak@172.16.16.211', >>>> [{'riak@192.168.3.5',[{{riak_control,member_info_version},[v1,v0]},{{riak_core,resizable_ring},...},...]},...]) >>>> line 72 >>>> 2015-08-25 11:13:39.217 [error] <0.161.0> CRASH REPORT Process >>>> riak_core_capability with 0 neighbours exited with reason: no function >>>> clause matching orddict:fetch('riak@172.16.16.211', >>>> [{'riak@192.168.3.5',[{{riak_control,member_info_version},[v1,v0]},{{riak_core,resizable_ring},...},...]},...]) >>>> line 72 in gen_server:terminate/6 line 747 >>>> 2015-08-25 11:13:39.219 [error] <0.137.0> Supervisor riak_core_sup had >>>> child riak_core_capability started with riak_core_capability:start_link() >>>> at <0.161.0> exit with reason no function clause matching >>>> orddict:fetch('riak@172.16.16.211', >>>> [{'riak@192.168.3.5',[{{riak_control,member_info_version},[v1,v0]},{{riak_core,resizable_ring},...},...]},...]) >>>> line 72 in context child_terminated >>>> 2015-08-25 11:13:39.219 [error] <0.135.0> CRASH REPORT Process <0.135.0> >>>> with 0 neighbours exited with reason: >>>> {{function_clause,[{orddict,fetch,['riak@172.16.16.211',[{'riak@192.168.3.5',[{{riak_control,member_info_version},[v1,v0]},{{riak_core,resizable_ring},[true,false]},{{riak_core,staged_joins},[true,false]},{{riak_core,vnode_routing},[proxy,legacy]},{{riak_kv,anti_entropy},[enabled_v1,disabled]},{{riak_kv,crdt},[[pncounter],[]]},{{riak_kv,handoff_data_encoding},[encode_raw,encode_zlib]},{{riak_kv,index_backpressure},[true,false]},{{riak_kv,legacy_keylisting},[false]},{{riak_kv,listkeys_backpressure},...},...]},...]],...},...]},...} >>>> in application_master:init/4 line 138 >>>> >>>> At the moment, it looks like I can’t restore the cluster. Is there any >>>> other way of verifying the backup? Perhaps I can simply pull out all the >>>> keys in the bitcask data dump? >>>> >>>> Thanks, >>>> Sujay >>>> >>>> >>>> On Mon, Aug 24, 2015 at 1:43 PM, Dmitri Zagidulin <dzagidu...@basho.com >>>> > wrote: >>>> >>>>> Hi Sujay, >>>>> >>>>> This is where we get into the fact that maintaining docs across many >>>>> versions is a hard problem :) >>>>> >>>>> You'll want to follow the instructions laid out in >>>>> http://docs.basho.com/riak/latest/ops/running/nodes/renaming/#Clusters-from-Backups >>>>> (the >>>>> Clusters from Backup section, specifically). That outlines the >>>>> instructions >>>>> on renaming the ring on an existing new cluster from backup. (And keep in >>>>> mind what I said earlier about renaming the Erlang cookie in vm.args). >>>>> >>>>> Since it's written for Riak version 2, you'll want to cross-reference >>>>> it with the slightly older version of the doc, that you're looking at, >>>>> http://docs.basho.com/riak/1.4.2/ops/running/nodes/renaming/ . The >>>>> procedure should be largely the same, just the names of the config files >>>>> are different. >>>>> >>>>> >>>>> On Monday, August 24, 2015, Sujay Mansingh <su...@editd.com> wrote: >>>>> >>>>>> Hi guys >>>>>> >>>>>> I am looking at the instructions here: >>>>>> http://docs.basho.com/riak/1.4.2/ops/running/nodes/renaming/ >>>>>> >>>>>> However, these are instructions for renaming an existing cluster >>>>>> ‘in-place’. >>>>>> >>>>>> What I have is an existing 5 node cluster. >>>>>> I have brought up a completely new (and separate) 5 node cluster. >>>>>> I am copying over the bitcask data and /var/lib/riak/ring >>>>>> directories from each existing node to the new cluster. (i.e. from >>>>>> existing-01 to new-01, existing-02 to new-02, etc) >>>>>> >>>>>> The instructions above mention to join the cluster, but I don’t wish >>>>>> to do that (as it would join a new node to the existing cluster). >>>>>> >>>>>> At the moment I have not formed the new cluster (all 5 riak nodes are >>>>>> standalone). >>>>>> What do I need to do in order to rename the ring on the nodes in the >>>>>> new cluster? >>>>>> >>>>>> Sujay >>>>>> >>>>>> >>>>>> On Tue, Aug 11, 2015 at 4:47 PM, Dmitri Zagidulin < >>>>>> dzagidu...@basho.com> wrote: >>>>>> >>>>>>> Hi Sujay, >>>>>>> >>>>>>> Yes, riak.conf is a riak 2 thing. If you're running 1.4, you would >>>>>>> change the -setcookie in vm.args, exactly. >>>>>>> >>>>>>> And no, the node name doesn't have to match the cookie. The two are >>>>>>> independent. >>>>>>> >>>>>>> On Tue, Aug 11, 2015 at 3:22 PM, Sujay Mansingh <su...@editd.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Oh and also, does the first part of the riak node name have to >>>>>>>> match the cookie? >>>>>>>> I.e. If I change the cookie to riaktest, does the node name have to >>>>>>>> be riaktest@{{ ip_addr }} ? >>>>>>>> >>>>>>>> >>>>>>>> On Tuesday, August 11, 2015, Sujay Mansingh <su...@editd.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Dmitri >>>>>>>>> >>>>>>>>> When you say the cookie must be modified in /etc/riak/riak.conf, >>>>>>>>> is that a riak 2 thing? >>>>>>>>> I can see a -setcookie riak line in /etc/riak/vm.args, is that >>>>>>>>> what you mean? >>>>>>>>> >>>>>>>>> Sujay >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Aug 6, 2015 at 2:11 PM, Dmitri Zagidulin < >>>>>>>>> dzagidu...@basho.com> wrote: >>>>>>>>> >>>>>>>>>> Sujay, >>>>>>>>>> >>>>>>>>>> You're right - the best way to verify the backup is to bring up a >>>>>>>>>> separate 5 node cluster, and restore it from the backup files. >>>>>>>>>> The procedure is slightly more involved than untar-ing, though. >>>>>>>>>> The backed up ring directories from the original cluster will >>>>>>>>>> contain the >>>>>>>>>> node ids (which rely on their IP addresses, etc). Since the new >>>>>>>>>> re-hydrated >>>>>>>>>> cluster is likely to have different IPs from the original one, >>>>>>>>>> there's a >>>>>>>>>> few more steps you need to take. >>>>>>>>>> >>>>>>>>>> The procedure of standing up a new cluster from backups is >>>>>>>>>> outlined here: >>>>>>>>>> http://docs.basho.com/riak/latest/ops/running/nodes/renaming/#Clusters-from-Backups >>>>>>>>>> >>>>>>>>>> There is one other important step to remember. If, by any chance, >>>>>>>>>> you're bringing up the new cluster on the same network as the old >>>>>>>>>> cluster >>>>>>>>>> is running on, be sure to modify the Erlang cookie in the new >>>>>>>>>> cluster (so >>>>>>>>>> that as far as Erlang is concerned, they're existing on different >>>>>>>>>> networks). >>>>>>>>>> The Erlang cookie must be modified in /etc/riak/riak.conf so the >>>>>>>>>> new cluster does not conflict with any existing cluster. >>>>>>>>>> >>>>>>>>>> Let us know if you have any further questions. >>>>>>>>>> >>>>>>>>>> Dmitri >>>>>>>>>> >>>>>>>>>> On Thu, Aug 6, 2015 at 8:40 AM, Sujay Mansingh <su...@editd.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Magnus & John. >>>>>>>>>>> >>>>>>>>>>> Yes certainly I will test it on a separate cluster first! Which >>>>>>>>>>> is related to another question I have. >>>>>>>>>>> >>>>>>>>>>> If I want to backup I can archive the directories on the nodes >>>>>>>>>>> as described here: >>>>>>>>>>> http://docs.basho.com/riak/latest/ops/running/backups/#OS-Specific-Directory-Locations >>>>>>>>>>> >>>>>>>>>>> But in order to verify the backup (or perform operations on the >>>>>>>>>>> cluster in 'offline' mode), can I simply bring up a separate 5 node >>>>>>>>>>> cluster >>>>>>>>>>> and untar the backup files? >>>>>>>>>>> (Probably not the /etc/riak directory, but the data and ring >>>>>>>>>>> directories.) >>>>>>>>>>> >>>>>>>>>>> I want to do that, and then try adding a riak 2.0.6 node to the >>>>>>>>>>> test riak 1.4.2 cluster and see if things are ok. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Sujay >>>>>>>>>>> >>>>>>>>>>> On Thu, Aug 6, 2015 at 9:31 AM, Magnus Kessler < >>>>>>>>>>> mkess...@basho.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 5 August 2015 at 18:53, John Daily <jda...@basho.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> That’s correct: upgrades to either 2.0.x or 2.1.x are >>>>>>>>>>>>> supported from the 1.4 series. >>>>>>>>>>>>> >>>>>>>>>>>>> Side note: I definitely recommend testing the upgrade process >>>>>>>>>>>>> in a QA environment first. >>>>>>>>>>>>> >>>>>>>>>>>>> -John >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Sujay, >>>>>>>>>>>> >>>>>>>>>>>> The latest release in the 2.0 series is 2.0.6 [0]. Please use >>>>>>>>>>>> this version if you upgrade to 2.0. >>>>>>>>>>>> >>>>>>>>>>>> Please also review the documentation about the new 'riak.conf' >>>>>>>>>>>> configuration file [1][2]. 2.x installations should use the new >>>>>>>>>>>> format, but >>>>>>>>>>>> you can continue to use the 'app.config' format from Riak 1.x. To >>>>>>>>>>>> maintain >>>>>>>>>>>> complete backwards compatibility when using 'app.config', please >>>>>>>>>>>> add >>>>>>>>>>>> >>>>>>>>>>>> [{default_bucket_props, >>>>>>>>>>>> [{allow_mult,false}, %% have Riak resolve conflicts and do not >>>>>>>>>>>> return siblings >>>>>>>>>>>> {dvv_enabled,false}]}, %% use vector clocks for conflict >>>>>>>>>>>> resolution >>>>>>>>>>>> %% other settings >>>>>>>>>>>> ] >>>>>>>>>>>> >>>>>>>>>>>> to 'app.config'. This will ensure that your existing >>>>>>>>>>>> application continues to work exactly as before. When using >>>>>>>>>>>> 'riak.conf', >>>>>>>>>>>> these settings will be applied automatically. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Magnus >>>>>>>>>>>> >>>>>>>>>>>> [0] http://docs.basho.com/riak/2.0.6/downloads/ >>>>>>>>>>>> [1] >>>>>>>>>>>> http://docs.basho.com/riak/latest/intro-v20/#Simplified-Configuration-Management >>>>>>>>>>>> [2] >>>>>>>>>>>> http://docs.basho.com/riak/latest/ops/advanced/configs/configuration-files/ >>>>>>>>>>>> >>>>>>>>>>>> On Aug 5, 2015, at 12:13 PM, Sujay Mansingh <su...@editd.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hello all, I have a 5 node riak cluster, all nodes running >>>>>>>>>>>>> 1.4.2. >>>>>>>>>>>>> >>>>>>>>>>>>> I want to upgrade to riak 2.x >>>>>>>>>>>>> >>>>>>>>>>>>> According to this: >>>>>>>>>>>>> http://docs.basho.com/riak/latest/ops/upgrading/rolling-upgrades/ >>>>>>>>>>>>> I can perform a rolling upgrade (a mixed cluster) >>>>>>>>>>>>> as long as the versions aren't more than 2 versions apart. >>>>>>>>>>>>> >>>>>>>>>>>>> There is no riak 1.5 so would riak 1.4.2 -> 2.0.5 count as 1 >>>>>>>>>>>>> version apart? >>>>>>>>>>>>> >>>>>>>>>>>>> Sujay >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> riak-users mailing list >>>>>>>>>>>>> riak-users@lists.basho.com >>>>>>>>>>>>> >>>>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> riak-users mailing list >>>>>>>>>>>>> riak-users@lists.basho.com >>>>>>>>>>>>> >>>>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Magnus Kessler >>>>>>>>>>>> Client Services Engineer @ Basho >>>>>>>>>>>> >>>>>>>>>>>> Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg >>>>>>>>>>>> 07970431 >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> riak-users mailing list >>>>>>>>>>>> riak-users@lists.basho.com >>>>>>>>>>>> >>>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> riak-users mailing list >>>>>>>>>>> riak-users@lists.basho.com >>>>>>>>>>> >>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> riak-users mailing list >>>>>>>>>> riak-users@lists.basho.com >>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>> >>> >> >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
