Hi, You guys may want to give the following command a try:
riak-admin wait-for-service riak_kv Credit to Dizzy for bringing it to my attention yesterday [0]. [0] https://twitter.com/dizzyd/status/308710172136710144 -- Hector On Tue, Mar 5, 2013 at 7:57 AM, Shane McEwan <sh...@mcewan.id.au> wrote: > On 05/03/13 11:36, Daniel Iwan wrote: >> >> What is the correct approach to make sure nodes are ready and Riak service >> is fully up and running and can take request? >> SInce our app depends on Riak we need to somehow make sure Riak is ready >> before our services start. > > > G'day! > > We run 256 vnodes on 4 nodes. Our average Riak startup time in our > virtualised test environment is about 60-70 seconds although it sometimes > can take longer (I've seen over 300 seconds before I moved to faster disk!). > > I watch the Riak console.log files and wait for the "Wait complete for > service riak_kv" message to appear on each node before starting our > application. > > I have a nightly cron job that updates our test Riak database from > production. The cron job probably does most of what your test framework is > doing: stops the application, stops Riak, rsyncs the production database > files, starts Riak, waits for Riak to settle, starts the application. > > I use 'dsh'[http://sourceforge.net/projects/dsh/] to ssh to the four Riak > nodes to start Riak. On each Riak node I have the following script: > > ################################################################ > #!/bin/bash > > pattern=$1 > file=$2 > > if [[ -z $pattern || -z $file ]]; then > echo "Usage: $0 <pattern> <file>" > exit 1 > fi > > # Wait for string and then exit > grep -q "$pattern" <( exec tail -n 0 -f $file ); kill $! > ################################################################# > > This script will wait for a string to appear in a file and then exit. > > I run it from my cron job like this: > > dsh -r ssh -c -m node1 -m node2 -m node3 -m node4 -- > /usr/local/bin/waitforstring '"Wait complete for service riak_kv"' > /var/log/riak/console.log > > Using the above command in your test framework allows you to wait for your > Riak nodes to start before starting your application. 'dsh' won't exit until > the commands on all four nodes have completed. > > Of course, there's a potential race condition if Riak starts faster than you > can ssh and run the "waitforstring" script. If the string you're waiting for > has already appeared in the log file before you start looking for it you'll > never see it and end up waiting forever. As always, YMMV. > > An alternative is to have a loop running "riak-admin ringready" and > "riak-admin transfers" and exit when they are both showing desired output. > However, watching the log file is a nice, low impact method that has been > working great for me. > > Hope this helps! > > Shane. > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
