AAE problems
Hi guys,
I just noticed looking at the disk utilization that AAE is consuming a
significant amount of space on the hard drive.
du -hs /var/lib/riak/*| grep anti
68G /var/lib/riak/anti_entropy
The entire dataset is idempotent and immutable, so there is not even a
slightest chance that we are ending up with different values on
different nodes for the same key in the same bucket. It seems that
anti-entropy still finds problems:
/var/log/riak/console.log.4:2014-06-11 06:11:41.756 [info]
<0.6776.6003>@riak_kv_exchange_fsm:key_exchange:206 Repaired 1 keys
during active anti-entropy exchange of
{536645132457440915277915524513010171279912730624,3} between
{548063113999088594326381812268606132370974703616,'riak@10.1.11.120'}
and {559481095540736273374848100024202093462036676608,'riak@10.1.11.121'}
My question would be:
Is there any reason to let AAE running if we don't mutate the data in place?
Is there any way knowing what is causing the difference according to
AAE between two nodes? I was thinking about how this could potentially
happen and I am wondering if the Java client pb interface supports R
and W values, so I could make sure that a write goes in with W=(the
number of nodes we have).
Thanks in advance,
Istvan
--
the sun shines for all
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak-admin backup - Backup documentation out of date?
Toby, While there is mention of it on the riak-admin page [1], in the upcoming 2.0 release `riak-admin backup` has been deprecated. The documentation you linked contains the suggested ways for backing up your cluster at this time. There are many ongoing discussions about providing a first-class backup solution as part of riak(-admin) again but, right now, I do not have many more details on how it would work or when it will be available. [1] http://docs.basho.com/riak/1.4.9/ops/running/tools/riak-admin/#backup On Mon, Jun 16, 2014 at 6:16 PM, Toby Corkindale wrote: > Hi, > The documentation here, apparently the latest, doesn't appear to > mention the riak-admin backup command at all, nor is it mentioned on > the command-line tools page for riak-admin. > http://docs.basho.com/riak/latest/ops/running/backups/ > http://docs.basho.com/riak/1.4.9/ops/running/tools/riak/ > > Is the command suitable for use yet? It'll simplify backups greatly if so. > > Cheers, > Toby > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: AAE problems
On Tue, Jun 17, 2014 at 9:46 AM, István wrote: > Is there any reason to let AAE running if we don't mutate the data in > place? > Yes. If your data is infrequently accessed after being written then no read-repair will take place. While values may not conflict, values may be missing (e.g. a write only successfully reached 2 of 3 nodes) or corrupt (bit rot on disk). > Is there any way knowing what is causing the difference according to > AAE between two nodes? This information is not currently exposed, unfortunately. Given the minimal number of repaired keys its likely a write failed to one node or raced with an AAE exchange between two pairs of nodes. > I was thinking about how this could potentially > happen and I am wondering if the Java client pb interface supports R > and W values, so I could make sure that a write goes in with W=(the > number of nodes we have). > While this is possible, although you'd want to set W=N, where N=number of replicas (n_val), it will affect the performance of your writes -- although that may not matter given the boost from turning AAE off. In the end, it is the operators decision whether or not the storage and performance overhead of AAE is worth the added safety*. If you are frequently accessing 100% of your data set then there is less need to increase the W per request or use AAE. However, many data sets have a large amount of data that is infrequently accessed and the added safety is worth the cost. Hope that helps. Cheers, Jordan * In the upcoming Riak 2.0 release, in order to write data to buckets w/ the consistent property set to true enabling AAE will be required ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: AAE problems
Thanks Jordan, Much appreciated! I think we are good now, I deleted the AAE data and restarted the service on one node and I am going to monitor the growth of the AAE folder over time. See what happens. Best regards, Istvan On Tue, Jun 17, 2014 at 10:48 AM, Jordan West wrote: > On Tue, Jun 17, 2014 at 9:46 AM, István wrote: > >> >> Is there any reason to let AAE running if we don't mutate the data in >> place? > > > Yes. If your data is infrequently accessed after being written then no > read-repair will take place. While values may not conflict, values may be > missing (e.g. a write only successfully reached 2 of 3 nodes) or corrupt > (bit rot on disk). > >> >> Is there any way knowing what is causing the difference according to >> AAE between two nodes? > > > This information is not currently exposed, unfortunately. Given the minimal > number of repaired keys its likely a write failed to one node or raced with > an AAE exchange between two pairs of nodes. > >> >> I was thinking about how this could potentially >> happen and I am wondering if the Java client pb interface supports R >> and W values, so I could make sure that a write goes in with W=(the >> number of nodes we have). > > > While this is possible, although you'd want to set W=N, where N=number of > replicas (n_val), it will affect the performance of your writes -- although > that may not matter given the boost from turning AAE off. In the end, it is > the operators decision whether or not the storage and performance overhead > of AAE is worth the added safety*. If you are frequently accessing 100% of > your data set then there is less need to increase the W per request or use > AAE. However, many data sets have a large amount of data that is > infrequently accessed and the added safety is worth the cost. > > Hope that helps. Cheers, > Jordan > > * In the upcoming Riak 2.0 release, in order to write data to buckets w/ > the consistent property set to true enabling AAE will be required -- the sun shines for all ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
+stbt nnts
Hi all, I am trying to set the Erlang scheduler bind type to nnts by adding the following to vm.args: +stbt nnts In theory stbt is supposed to handle lack of support properly, but instead my server fails to start. However it starts with just +sbt nnts. Here is the Erlang doc on the flag: http://erlang.org/doc/man/erl.html#+sbt Has anyone run into this? How can I see what's failing? Cheers, Alain ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: +stbt nnts
Hi Alain, You're likely running an erlang version that's too early. +sbt was added earlier than +stbt. They're otherwise equivalent, so using +sbt should be OK. On Tue, Jun 17, 2014 at 4:16 PM, Alain Rodriguez wrote: > Hi all, > > I am trying to set the Erlang scheduler bind type to nnts by adding the > following to vm.args: > > +stbt nnts > > In theory stbt is supposed to handle lack of support properly, but instead > my server fails to start. However it starts with just +sbt nnts. > > Here is the Erlang doc on the flag: http://erlang.org/doc/man/erl.html#+sbt > > Has anyone run into this? How can I see what's failing? > > Cheers, > > Alain > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
