Re: Migrating to riak 2.1.4, selecting a specific multi-backend from the Java Client

2016-08-10 Thread Magnus Kessler
On 9 August 2016 at 15:42, Henning Verbeek  wrote:

> We're slowly migrating our application from Riak KV 1.4 to Riak KV
> 2.1.4, and at the same time riak-java-client 1.4.x to riak-java-client
> 2.0.6. We are doing this by adding seven new Riak KV 2.1 nodes to the
> ring, and later removing the seven Riak KV 1.4 nodes from the ring.
> For a few days, the cluster will consist of a mix.
>

Hi Henning,

This plan will generate a lot of unnecessary data shuffling in the entire
cluster. Each time a node is added or removed from the ring, the claim
algorithm will re-assign partitions to/from all nodes. Please consider
updating the existing nodes in place. For more information see the
documentation[0]. Please let us know if any of the documentation is
unclear, so that we can make it better.


> [...]
>

Kind Regards,

Magnus


[0]: http://docs.basho.com/riak/kv/2.1.4/setup/upgrading/cluster/

-- 
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


Re: Migrating to riak 2.1.4, selecting a specific multi-backend from the Java Client

2016-08-10 Thread Henning Verbeek
Hi Magnus,

On Wed, Aug 10, 2016 at 9:21 AM, Magnus Kessler  wrote:
> On 9 August 2016 at 15:42, Henning Verbeek  wrote:
>>
>> We're slowly migrating our application from Riak KV 1.4 to Riak KV
>> 2.1.4, and at the same time riak-java-client 1.4.x to riak-java-client
>> 2.0.6. We are doing this by adding seven new Riak KV 2.1 nodes to the
>> ring, and later removing the seven Riak KV 1.4 nodes from the ring.
>> For a few days, the cluster will consist of a mix.
>
> This plan will generate a lot of unnecessary data shuffling in the entire
> cluster. Each time a node is added or removed from the ring, the claim
> algorithm will re-assign partitions to/from all nodes. Please consider
> updating the existing nodes in place.

Thanks. We anticipated that, and it's basically two ring changes (add
all new nodes, remove all old nodes). The balancing takes about 12h
for the 15TB or so to get redistributed. We've taken this opportunity
to move our data onto newer, bigger machines that have become
available at our hosting provider.

Cheers,
Henning

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


Re: Migrating to riak 2.1.4, selecting a specific multi-backend from the Java Client

2016-08-10 Thread Henning Verbeek
Good morning Alex,

On Tue, Aug 9, 2016 at 8:47 PM, Alex Moore  wrote:
>>
>> Is this equivalent to specifying a bucket type?
>
>
> Not exactly, that code will set the backend for that bucket, which is in the
> default bucket type.
> If you need to specify a bucket-type, it's just an extra parameter on the
> Namespace ctor:
>
> Namespace ns = new Namespace(bucketType, bucketName);
>
> If you'd like everything in a bucket-type to be on the memory backend, you
> should be able to specify that backend in the bucket type properties when
> creating it with riak-admin. The bucket type'd namespace wont' be available
> to the old Riak nodes though :/

Thanks. From this I understand that my "old" way (no bucket types)
will continue to work, right? At least for the short while where I
have a mixed bucket.

I still struggle with bucket types. In my mind, it is not ideal to
have to switch to the console (riak-admin) to set application-specific
properties. I'd much prefer if I could set/query/confirm such
properties in the initialisation code of my application; that way it
can't be forgotten. riak-admin is a manual operation and hence
error-prone.

Looking at 
http://docs.basho.com/riak/kv/2.1.4/using/cluster-operations/bucket-types/,
it is not clear to me, if actions get propagated throughout the
cluster, or if it must be repeated on each cluster node. "bucket
properties are gossiped around", but what about bucket types? Is there
a way to detect what bucket-types exist, what has been activated, and
with what properties?
I find the blue box ("how many buckets can I have") difficult to
follow. After some time I understand from it: don't modify default
bucket properties, rather define the desired properties in a bucket
type, then use the bucket type when interacting with the bucket.

It is not clear to me, if bucket types can later be associated with
buckets that used the default bucket-type in the past. If bucket-types
are part of the namespace, I guess no. Maybe worth mentioning this in
the migration section?

I'm also looking for documentation, what properties can be set in a bucket type.

>>  How does a cluster behave where some nodes have the multibackend
>> configured (in riak.conf), and others not (because you forgot)?
>> What happens if you configure the multibackend afterwards? Is the
>> bucket with all its data being migrated? Can you detect in which
>> *backend* an object is located?
>
>
> Anything without the multibackend config should automatically save/load to
> whatever the configured (or default) backend is.   If you configure it
> afterwards, the data won't be available b/c it's on a backend it's now not
> configured for.

Thanks, that makes it very clear.

Cheers,
Henning

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com