Re: Getting largest key
Hi Alex, Thanks a lot for your answer. Very often (we upgrade an application from BerkeleyDB to RIAK). E.g.: A new key should be be the largest key + 1. Is there an other trick to workaround this problem? Thanks, Markus Gesendet: Mittwoch, 18. September 2013 um 21:53 Uhr Von: "Alex Moore" An: riak-users@lists.basho.com, "Markus Doppelbauer" Betreff: Re: Hi Markus, With Riak 1.4, 2i results are sorted in ascending order. So a range query from 0 to or to 0 with a max result of 1 will always return the lowest index value. In your case you may want to store the largest key # somewhere, or if you need to find it infrequently you may also try a map reduce job. How often will you need to know this max key? Thanks, Alex Moore On September 18, 2013 at 5:56:22 AM, Markus Doppelbauer (doppelba...@gmx.net) wrote: Hello, Is there a chance to get the biggest key of a bucket or secondary index? E.g.: A bucket (or 2i) contains the keys: 10, 20, 30, 40, 50 The result should be: 50 Could this problem be solved, by quering a secondary index from back, e.g.: curl http://localhost:8098/buckets/mybucket/index/field_int/999/0?max_results=1 Thanks a lot Markus ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Getting largest key
You can store revertIndex = (MAX_KEY_VALUE - keyvaluefromberkley) in Riak as a secondary index for every object. Then get a full range for that index limiting results to 1. In this way you'll get one result with max keyvaluefromberkley. Reversing order in a nutshell, because I think values for 2i in Riak are always ascending. Regards Daniel -- View this message in context: http://riak-users.197444.n3.nabble.com/no-subject-tp4029126p4029153.html Sent from the Riak Users mailing list archive at Nabble.com. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Getting largest key
On September 19, 2013 10:12:06 AM Markus Doppelbauer wrote: > Hi Alex, > > Thanks a lot for your answer. > Very often (we upgrade an application from BerkeleyDB to RIAK). > > E.g.: A new key should be be the largest key + 1. Is there an other > trick to workaround this problem? > > Thanks, > Markus > Hi Markus, Because Riak is an eventually consistent system, it is very hard/next to impossible to guarantee this property using Riak alone. For instance, if you have a cluster split, then several inserts you will end up with multiple records with the same id. This occusrs since both sides of the cluster will only see its new keys, and believe that the ids are all unique. Instead, you could use something like Twitter's snowflake (available here: https://github.com/twitter/snowflake ) (if you don't want java, there are various re-implementations of it around). Snowflake will generate ids that are generally incrementing, but will also leave gaps. You can run multiple instances (safely) to get redundancy or capacity. Snowflake works without having to synchronize its state amongst several servers, so it would continue to work in the face of cluster splits. -- Matthew smime.p7s Description: S/MIME cryptographic signature ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: ring resizing feature?
Dear Alexander, dear Jordan, Both of your answers proved very helpful, thank you very much. Glad to see an active mailing list working. All the best, PN > Message du 19/09/13 05:34 > De : "Alexander Sicular" > A : "pseudo-n...@laposte.net" > Copie à : "riak-users@lists.basho.com" > Objet : Re: ring resizing feature? > > That's about the rights of it. As I was reading your message I did some quick > math and sure enough you already put that same number down. I've heard told > that the max size for any Erlang network, of which Riak is most assuredly, is > 200 Erlang VM's. Extrapolating from there, considering that I've also heard > told that a good sizing rule of thumb is 10 vnodes per node, you find > yourself with ~2000 vnodes. Accounting for another wives' tale that the ring > size should be powers of 2, 2048 looks to be the upper bound. > > Wives' tales, rumors and general misinformation aside, I've not tested those > numbers in any way whatsoever but I would love to see an EC2 script give it a > shot. > > -Alexander > > @siculars > http://siculars.posthaven.com > > Sent from my iRotaryPhone > > On Sep 18, 2013, at 3:44, pseudo-n...@laposte.net wrote: > > > I have seen mentioned that 2048 and higher could importantly increase the > > volume of gossiping, what is your experience with it? > Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ? Je crée ma boîte mail www.laposte.net ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Using /etc/hosts entry and not ip in configuration files ..
hi.
we are trying to run Riak in the Amazon cloud (using OpsWorks).
With every restart of the servers the ip addresses change so using an entry
in the /etc/hosts instead of an IP looks like a great idea.
Google's answer is:
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-December/006912.html
Just to make sure: Riak only supports IP addresses and not DNS names in the
config files.
Though in your docs it still says:
"Riak identifies other machines in the ring using Erlang identifiers
(, ex: riak@10.9.8.7)."
The error logged when changing our config from something like 172.31.41.138
to riaknode2-ext is:
2013-09-19 21:49:53.383 [error] <0.113.0> Supervisor riak_core_sup had
child "http_riaknode2-ext:8098" started with
webmachine_mochiweb:start([{name,"http_riaknode2-ext:8098"},{ip,"riaknode2-ext"},{port,8098},{nodelay,true},{log_dir,"log"},...])
at undefined exit with reason
{'EXIT',{{badmatch,{error,einval}},[{mochiweb_socket_server,parse_options,2},{mochiweb_socket_server,start,1},{supervisor,do_start_child,2},{supervisor,start_children,3},{supervisor,init_children,2},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}}
in context start_error
Thanks a bunch.
Have a great day.
Michael
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
VNodes distribution on the ring
Is there anywhere a pseudo-code or description of the algorithm how vnodes (primaries and replicas) would be distributed if I had 3, 4 and more nodes in the cluster? Does it depend in any way on the node name or any other setting, or is it only a function of number of physical nodes? Regards Daniel ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Using /etc/hosts entry and not ip in configuration files ..
Hi Mike,Just to make sure: Riak only supports IP addresses and not DNS names in the config files.Though in your docs it still says:"Riak identifies other machines in the ring using Erlang identifiers (, ex: riak@10.9.8.7)."Riak only accepts ip addresses in the app.config file, but the vm.args "-name" parameter can use either full hostnames or ip addresses. You can read about the -name parameter at http://www.erlang.org/doc/reference_manual/distributed.html.The error logged when changing our config from something like 172.31.41.138 to riaknode2-ext is:2013-09-19 21:49:53.383 [error] <0.113.0> Supervisor riak_core_sup had child "http_riaknode2-ext:8098" started with webmachine_mochiweb:start([{name,"http_riaknode2-ext:8098"},{ip,"riaknode2-ext"},{port,8098},{nodelay,true},{log_dir,"log"},...]) at undefined exit with reason {'EXIT',{{badmatch,{error,einval}},[{mochiweb_socket_server,parse_options,2},{mochiweb_socket_server,start,1},{supervisor,do_start_child,2},{supervisor,start_children,3},{supervisor,init_children,2},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}} in context start_errorIf you've previously started the node with another "-name" parameter, you will have to do the renaming procedure instead of just changing the "-name" value: http://docs.basho.com/riak/latest/ops/running/nodes/renaming/. Thanks,AlexOn September 19, 2013 at 6:52:29 PM, Mike Nathe (mna...@fathom-i.com) wrote:hi.we are trying to run Riak in the Amazon cloud (using OpsWorks).With every restart of the servers the ip addresses change so using an entry in the /etc/hosts instead of an IP looks like a great idea.Google's answer is:http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-December/006912.htmlJust to make sure: Riak only supports IP addresses and not DNS names in the config files.Though in your docs it still says:"Riak identifies other machines in the ring using Erlang identifiers (, ex: riak@10.9.8.7)."The error logged when changing our config from something like 172.31.41.138 to riaknode2-ext is:2013-09-19 21:49:53.383 [error] <0.113.0> Supervisor riak_core_sup had child "http_riaknode2-ext:8098" started with webmachine_mochiweb:start([{name,"http_riaknode2-ext:8098"},{ip,"riaknode2-ext"},{port,8098},{nodelay,true},{log_dir,"log"},...]) at undefined exit with reason {'EXIT',{{badmatch,{error,einval}},[{mochiweb_socket_server,parse_options,2},{mochiweb_socket_server,start,1},{supervisor,do_start_child,2},{supervisor,start_children,3},{supervisor,init_children,2},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}} in context start_errorThanks a bunch.Have a great day.Michael
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: VNodes distribution on the ring
It's best ignored, truthfully. That being said, if you really want to know, there is a good description in Replication - Understanding Replication by Example [1] Concepts - Replication [2] Little Riak Book - Replication and Partitions [3] [1]: http://docs.basho.com/riak/latest/theory/concepts/Replication/#Understanding-replication-by-example [2]: http://docs.basho.com/riak/latest/theory/concepts/#Replication [3]: http://littleriakbook.com/#toc_7 --- Jeremiah Peschka - Founder, Brent Ozar Unlimited MCITP: SQL Server 2008, MVP Cloudera Certified Developer for Apache Hadoop On Thu, Sep 19, 2013 at 4:24 PM, Daniel Iwan wrote: > Is there anywhere a pseudo-code or description of the algorithm how > vnodes (primaries and replicas) would be distributed if I had 3, 4 and more > nodes in the cluster? > > Does it depend in any way on the node name or any other setting, or is it > only a function of number of physical nodes? > > Regards > Daniel > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Using /etc/hosts entry and not ip in configuration files ..
Hey Mike,Another option for your app.config would be to bind everything to "0.0.0.0", which will make Riak listen on all interfaces. This combined with using a domain name in the vm.args should prevent you from having to do renames/changing ips in the config files. If you do this, please consider using a VPC to limit access to the machines. A good resource to read over is Amazon's Riak Whitepaper: http://media.amazonwebservices.com/AWS_NoSQL_Riak.pdf, as it goes through some operation considerations and tuning points that might be useful to you.Thanks,AlexOn September 19, 2013 at 6:52:29 PM, Mike Nathe (mna...@fathom-i.com) wrote: hi.we are trying to run Riak in the Amazon cloud (using OpsWorks).
With every restart of the servers the ip addresses change so using an entry in the /etc/hosts instead of an IP looks like a great idea.
Google's answer is:http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-December/006912.html
Just to make sure: Riak only supports IP addresses and not DNS names in the config files.
Though in your docs it still says:"Riak identifies other machines in the ring using Erlang identifiers (, ex: riak@10.9.8.7)."
The error logged when changing our config from something like 172.31.41.138 to riaknode2-ext is:
2013-09-19 21:49:53.383 [error] <0.113.0> Supervisor riak_core_sup had child "http_riaknode2-ext:8098" started with webmachine_mochiweb:start([{name,"http_riaknode2-ext:8098"},{ip,"riaknode2-ext"},{port,8098},{nodelay,true},{log_dir,"log"},...]) at undefined exit with reason {'EXIT',{{badmatch,{error,einval}},[{mochiweb_socket_server,parse_options,2},{mochiweb_socket_server,start,1},{supervisor,do_start_child,2},{supervisor,start_children,3},{supervisor,init_children,2},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}} in context start_error
Thanks a bunch.
Have a great day.Michael
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Using /etc/hosts entry and not ip in configuration files ..
hi.
we are trying to run Riak in the Amazon cloud (using OpsWorks).
With every restart of the servers the ip addresses change so using an entry
in the /etc/hosts instead of an IP looks like a great idea.
Google's answer is:
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-December/006912.html
Just to make sure: Riak only supports IP addresses and not DNS names in the
config files.
Though in your docs it still says:
"Riak identifies other machines in the ring using Erlang identifiers
(, ex: riak@10.9.8.7)."
The error logged when changing our config from something like 172.31.41.138
to riaknode2-ext is:
2013-09-19 21:49:53.383 [error] <0.113.0> Supervisor riak_core_sup had
child "http_riaknode2-ext:8098" started with
webmachine_mochiweb:start([{name,"http_riaknode2-ext:8098"},{ip,"riaknode2-ext"},{port,8098},{nodelay,true},{log_dir,"log"},...])
at undefined exit with reason
{'EXIT',{{badmatch,{error,einval}},[{mochiweb_socket_server,parse_options,2},{mochiweb_socket_server,start,1},{supervisor,do_start_child,2},{supervisor,start_children,3},{supervisor,init_children,2},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]}}
in context start_error
Thanks a bunch.
Have a great day.
Michael
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
