Re: Default values vs non-existing keys for CRDTs
It’s a tricky one Timur. CRDTs are based on Join Semi-Lattices, and they have the concept of a bottom value. That is what you are seeing. Implicitly all keys exist at the bottom value until you operate on them. Since you cannot do a compare-and-set operation with a counter anyway (that is you can’t “put counter with value 0” at that key), why do you need to distinguish between 0 as bottom and 0 as value? On 20 Aug 2015, at 01:59, Timur Fayruzov wrote: > Hello, > > It seems that Riak Datatype API does not allow to distinguish between > non-existing keys and default values. For example, if I query for > non-existing key as follows: > > val fetchOp = new FetchCounter.Builder(key).build() > val c = client.execute(fetchOp).getDatatype > > I'll get a counter that holds 0. Now, if I put counter with value 0 at this > key and run the query, I get the same result. Is there any way to distinguish > between these two different states? > > Note: when working with sets there is a context that I can check. If I fetch > a set and the context is null, it means that the set does not exist under > this key. This trick does not work for counters though, as they do not > maintain context. Is it a valid trick to use though? > > I posted this to StackOverflow a while ago, but no luck: > http://stackoverflow.com/questions/31845164/riak-dataypes-default-values-vs-non-existing-keys > > Thanks, > > Timur > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: s3cmd error: access to bucket was denied
somebody watching on this? On Wed, Aug 19, 2015 at 9:01 AM, changmao wang wrote: > Matthew, > > I used s3cmd --configure to generate ".s3cfg" config file and then access > RIAK service by s3cmd. > The access_key and secret_key from ".s3cfg" is same as admin_key > and admin_secret from "/etc/riak-cs/app.config". > > However, I got error as below using s3cmd to access one bucket. > > root@cluster-s3-hd1:~# s3cmd -c /root/.s3cfg ls > s3://pipeline/article/111.pdf > ERROR: Access to bucket 'pipeline' was denied > > By the way, I used Riak and Riak-CS 1.4.2 on Ubuntu. Current production > cluster is a legacy system without documents for co-workers. > > Attached file is "s3cfg" generated by "s3cmd --configure". > -- > Amao Wang > Best & Regards > -- Amao Wang Best & Regards ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: s3cmd error: access to bucket was denied
2015-08-20 13:57 GMT+05:00 changmao wang : > somebody watching on this? Do you set up same domain in riak-cs.conf and in .s3cfg? I got such error in this case. > On Wed, Aug 19, 2015 at 9:01 AM, changmao wang > wrote: >> >> Matthew, >> >> I used s3cmd --configure to generate ".s3cfg" config file and then access >> RIAK service by s3cmd. >> The access_key and secret_key from ".s3cfg" is same as admin_key and >> admin_secret from "/etc/riak-cs/app.config". >> >> However, I got error as below using s3cmd to access one bucket. >> >> root@cluster-s3-hd1:~# s3cmd -c /root/.s3cfg ls >> s3://pipeline/article/111.pdf >> ERROR: Access to bucket 'pipeline' was denied >> >> By the way, I used Riak and Riak-CS 1.4.2 on Ubuntu. Current production >> cluster is a legacy system without documents for co-workers. >> >> Attached file is "s3cfg" generated by "s3cmd --configure". >> -- >> Amao Wang >> Best & Regards > > > > > -- > Amao Wang > Best & Regards > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > -- Stanislav ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: s3cmd error: access to bucket was denied
stanislav, what's your meaning of domain name of /etc/riak-cs/app.config and ~/.s3cfg? I guess it's cs_root_host parameter from /etc/riak-cs/app.config and host_base from '~/.s3cfg'. If so, there're same as "api2.cloud-datayes.com". However, I can not ping this host from localhost. On Thu, Aug 20, 2015 at 5:23 PM, Stanislav Vlasov wrote: > 2015-08-20 13:57 GMT+05:00 changmao wang : > > somebody watching on this? > > Do you set up same domain in riak-cs.conf and in .s3cfg? > I got such error in this case. > > > On Wed, Aug 19, 2015 at 9:01 AM, changmao wang > > wrote: > >> > >> Matthew, > >> > >> I used s3cmd --configure to generate ".s3cfg" config file and then > access > >> RIAK service by s3cmd. > >> The access_key and secret_key from ".s3cfg" is same as admin_key and > >> admin_secret from "/etc/riak-cs/app.config". > >> > >> However, I got error as below using s3cmd to access one bucket. > >> > >> root@cluster-s3-hd1:~# s3cmd -c /root/.s3cfg ls > >> s3://pipeline/article/111.pdf > >> ERROR: Access to bucket 'pipeline' was denied > >> > >> By the way, I used Riak and Riak-CS 1.4.2 on Ubuntu. Current production > >> cluster is a legacy system without documents for co-workers. > >> > >> Attached file is "s3cfg" generated by "s3cmd --configure". > >> -- > >> Amao Wang > >> Best & Regards > > > > > > > > > > -- > > Amao Wang > > Best & Regards > > > > ___ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > -- > Stanislav > -- Amao Wang Best & Regards ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: s3cmd error: access to bucket was denied
2015-08-20 14:47 GMT+05:00 changmao wang :
> what's your meaning of domain name of /etc/riak-cs/app.config and ~/.s3cfg?
> I guess it's cs_root_host parameter from /etc/riak-cs/app.config and
> host_base from '~/.s3cfg'.
> If so, there're same as "api2.cloud-datayes.com".
Yes, is that i mean, but i see, it is not your case
Try to set {level, debug} in lager_file_backend section for console.log.
> However, I can not ping this host from localhost.
It's ok, if you write proper proxy_host and proxy_port in .s3cfg
> On Thu, Aug 20, 2015 at 5:23 PM, Stanislav Vlasov
> wrote:
>>
>> 2015-08-20 13:57 GMT+05:00 changmao wang :
>> > somebody watching on this?
>>
>> Do you set up same domain in riak-cs.conf and in .s3cfg?
>> I got such error in this case.
>>
>> > On Wed, Aug 19, 2015 at 9:01 AM, changmao wang
>> > wrote:
>> >>
>> >> Matthew,
>> >>
>> >> I used s3cmd --configure to generate ".s3cfg" config file and then
>> >> access
>> >> RIAK service by s3cmd.
>> >> The access_key and secret_key from ".s3cfg" is same as admin_key and
>> >> admin_secret from "/etc/riak-cs/app.config".
>> >>
>> >> However, I got error as below using s3cmd to access one bucket.
>> >>
>> >> root@cluster-s3-hd1:~# s3cmd -c /root/.s3cfg ls
>> >> s3://pipeline/article/111.pdf
>> >> ERROR: Access to bucket 'pipeline' was denied
>> >>
>> >> By the way, I used Riak and Riak-CS 1.4.2 on Ubuntu. Current production
>> >> cluster is a legacy system without documents for co-workers.
>> >>
>> >> Attached file is "s3cfg" generated by "s3cmd --configure".
>> >> --
>> >> Amao Wang
>> >> Best & Regards
>> >
>> >
>> >
>> >
>> > --
>> > Amao Wang
>> > Best & Regards
>> >
>> > ___
>> > riak-users mailing list
>> > riak-users@lists.basho.com
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>>
>>
>>
>> --
>> Stanislav
>
>
>
>
> --
> Amao Wang
> Best & Regards
--
Stanislav
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: s3cmd error: access to bucket was denied
Just now, I used "admin_key" and "admin_secret" from
/etc/riak-cs/app.config to run "s3cmd -c s3-stock ls
s3://stock/XSHE/0/000600"
and I got the below error:
ERROR: Access to bucket 'stock' was denied
Below is abstract from "/var/log/riak-cs/console.log"
2015-08-20 18:40:22.790 [debug]
<0.28085.18>@riak_cs_s3_auth:calculate_signature:129 STS:
["GET","\n",[],"\n",[],"\n","\n",[["x-amz-date",":",<<"Thu, 20 Aug 2015
10:40:22 +">>,"\n"]],["/stock/",[]]]
2015-08-20 18:40:32.861 [error]
<0.28153.18>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user
record for s3 failed. Reason: no_user_key
2015-08-20 18:40:32.861 [debug]
<0.28153.18>@riak_cs_wm_common:post_authentication:452 No user key
2015-08-20 18:40:32.969 [debug] <0.28189.18>@riak_cs_get_fsm:prepare:406
Manifest:
{lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,122,47,83,90,48,48,50,49,55,53,67,78,47,50,48,49,48,95,50,48,49,48,45,48,52,45,50,52,95,229,133,172,229,143,184,231,171,160,231,168,139,239,188,136,50,48,49,48,229,185,180,52,230,156,136,239,188,137,46,80,68,70>>},[],"2013-12-16T23:01:12.000Z",<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,255387,<<"application/pdf">>,<<55,41,141,170,187,226,47,223,183,95,105,129,155,154,210,202>>,active,{1387,234872,598819},{1387,234872,918555},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,234872,598546}},[],undefined}
2015-08-20 18:40:33.043 [debug]
<0.28189.18>@riak_cs_lfs_utils:range_blocks:118 InitialBlock: 0,
FinalBlock: 0
2015-08-20 18:40:33.043 [debug]
<0.28189.18>@riak_cs_lfs_utils:range_blocks:120 SkipInitial: 0, KeepFinal:
255387
2015-08-20 18:40:33.050 [debug]
<0.28189.18>@riak_cs_get_fsm:waiting_continue_or_stop:229 Block Servers:
[<0.28191.18>]
2015-08-20 18:40:33.079 [debug]
<0.28189.18>@riak_cs_get_fsm:waiting_chunks:307 Retrieved block
{<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0}
2015-08-20 18:40:33.079 [debug]
<0.28189.18>@riak_cs_get_fsm:perhaps_send_to_user:280 Returning block
{<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0} to client
2015-08-20 18:40:38.218 [error]
<0.28086.18>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user
record for s3 failed. Reason: no_user_key
2015-08-20 18:40:38.218 [debug]
<0.28086.18>@riak_cs_wm_common:post_authentication:452 No user key
2015-08-20 18:40:38.226 [debug] <0.28210.18>@riak_cs_get_fsm:prepare:406
Manifest:
{lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,104,47,83,72,54,48,48,55,53,48,67,78,47,50,48,48,55,95,50,48,48,55,45,49,49,45,50,49,95,230,177,159,228,184,173,232,141,175,228,184,154,229,133,179,228,186,142,73,66,69,95,53,232,141,175,229,147,129,232,142,183,229,190,151,228,186,140,230,156,159,228,184,180,229,186,138,230,137,185,230,150,135,231,154,132,229,133,172,229,145,138,229,143,138,233,163,142,233,153,169,230,143,144,231,164,186,46,112,100,102>>},[],"2013-12-15T09:04:48.000Z",<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,89863,<<"application/pdf">>,<<139,151,203,173,6,111,222,48,17,81,102,170,216,66,193,77>>,active,{1387,98288,545827},{1387,98288,618409},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,98288,545618}},[],undefined}
2015-08-20 18:40:38.280 [debug]
<0.28210.18>@riak_cs_lfs_utils:range_blocks:118 InitialBlock: 0,
FinalBlock: 0
2015-08-20 18:40:38.280 [debug]
<0.28210.18>@riak_cs_lfs_utils:range_blocks:120 SkipInitial: 0, KeepFinal:
89863
2015-08-20 18:40:38.280 [debug]
<0.28210.18>@riak_cs_get_fsm:waiting_continue_or_stop:229 Block Servers:
[<0.28212.18>]
2015-08-20 18:40:38.343 [debug]
<0.28210.18>@riak_cs_get_fsm:waiting_chunks:307 Retrieved block
{<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,0}
2015-08-20 18:40:38.344 [debug]
<0.28210.18>@riak_cs_get_fsm:perhaps_send_to_user:280 Returning block
{<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,0} to
client
On Thu, Aug 20, 2015 at 6:04 PM, Stanislav Vlasov
wrote:
> 2015-08-20 14:47 GMT+05:00 changmao wang :
>
> > what's your meaning of domain name of /etc/riak-cs/app.config and
> ~/.s3cfg?
> > I guess it's cs_root_host parameter from /etc/riak-cs/app.config and
> > host_base from '~/.s3cfg'.
> > If so, there're same as "api2.cloud-datayes.com".
>
> Yes, is that i mean, but i see, it is not your case
> Try to set {level, debug} in lager_file_backend section for consol
Re: s3cmd error: access to bucket was denied
Hi Changmao,
It seems your s3cmd config should include 2 items:
signature_v2 = True
host_base = api2.cloud-datayes.com
Riak CS requires "signature_v2 = True" since Riak CS has not supported
s3 authentication version 4 yet.
You can find a sample configuration of s3cmd here to interact with Riak CS [1].
[1]:
http://docs.basho.com/riakcs/2.0.1/cookbooks/configuration/Configuring-an-S3-Client/#Sample-s3cmd-Configuration-File-for-Production-Use
Thanks,
On Thu, Aug 20, 2015 at 7:44 PM, changmao wang wrote:
> Just now, I used "admin_key" and "admin_secret" from /etc/riak-cs/app.config
> to run "s3cmd -c s3-stock ls s3://stock/XSHE/0/000600"
> and I got the below error:
> ERROR: Access to bucket 'stock' was denied
>
> Below is abstract from "/var/log/riak-cs/console.log"
> 2015-08-20 18:40:22.790 [debug]
> <0.28085.18>@riak_cs_s3_auth:calculate_signature:129 STS:
> ["GET","\n",[],"\n",[],"\n","\n",[["x-amz-date",":",<<"Thu, 20 Aug 2015
> 10:40:22 +">>,"\n"]],["/stock/",[]]]
> 2015-08-20 18:40:32.861 [error]
> <0.28153.18>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user
> record for s3 failed. Reason: no_user_key
> 2015-08-20 18:40:32.861 [debug]
> <0.28153.18>@riak_cs_wm_common:post_authentication:452 No user key
> 2015-08-20 18:40:32.969 [debug] <0.28189.18>@riak_cs_get_fsm:prepare:406
> Manifest:
> {lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,122,47,83,90,48,48,50,49,55,53,67,78,47,50,48,49,48,95,50,48,49,48,45,48,52,45,50,52,95,229,133,172,229,143,184,231,171,160,231,168,139,239,188,136,50,48,49,48,229,185,180,52,230,156,136,239,188,137,46,80,68,70>>},[],"2013-12-16T23:01:12.000Z",<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,255387,<<"application/pdf">>,<<55,41,141,170,187,226,47,223,183,95,105,129,155,154,210,202>>,active,{1387,234872,598819},{1387,234872,918555},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,234872,598546}},[],undefined}
> 2015-08-20 18:40:33.043 [debug]
> <0.28189.18>@riak_cs_lfs_utils:range_blocks:118 InitialBlock: 0, FinalBlock:
> 0
> 2015-08-20 18:40:33.043 [debug]
> <0.28189.18>@riak_cs_lfs_utils:range_blocks:120 SkipInitial: 0, KeepFinal:
> 255387
> 2015-08-20 18:40:33.050 [debug]
> <0.28189.18>@riak_cs_get_fsm:waiting_continue_or_stop:229 Block Servers:
> [<0.28191.18>]
> 2015-08-20 18:40:33.079 [debug]
> <0.28189.18>@riak_cs_get_fsm:waiting_chunks:307 Retrieved block
> {<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0}
> 2015-08-20 18:40:33.079 [debug]
> <0.28189.18>@riak_cs_get_fsm:perhaps_send_to_user:280 Returning block
> {<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0} to client
> 2015-08-20 18:40:38.218 [error]
> <0.28086.18>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user
> record for s3 failed. Reason: no_user_key
> 2015-08-20 18:40:38.218 [debug]
> <0.28086.18>@riak_cs_wm_common:post_authentication:452 No user key
> 2015-08-20 18:40:38.226 [debug] <0.28210.18>@riak_cs_get_fsm:prepare:406
> Manifest:
> {lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,104,47,83,72,54,48,48,55,53,48,67,78,47,50,48,48,55,95,50,48,48,55,45,49,49,45,50,49,95,230,177,159,228,184,173,232,141,175,228,184,154,229,133,179,228,186,142,73,66,69,95,53,232,141,175,229,147,129,232,142,183,229,190,151,228,186,140,230,156,159,228,184,180,229,186,138,230,137,185,230,150,135,231,154,132,229,133,172,229,145,138,229,143,138,233,163,142,233,153,169,230,143,144,231,164,186,46,112,100,102>>},[],"2013-12-15T09:04:48.000Z",<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,89863,<<"application/pdf">>,<<139,151,203,173,6,111,222,48,17,81,102,170,216,66,193,77>>,active,{1387,98288,545827},{1387,98288,618409},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","ef38ca69e145a40c1f837
8633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,98288,545618}},[],undefined}
> 2015-08-20 18:40:38.280 [debug]
> <0.28210.18>@riak_cs_lfs_utils:range_blocks:118 InitialBlock: 0, FinalBlock:
> 0
> 2015-08-20 18:40:38.280 [debug]
> <0.28210.18>@riak_cs_lfs_utils:range_blocks:120 SkipInitial: 0, KeepFinal:
> 89863
> 2015-08-20 18:40:38.280 [debug]
> <0.28210.18>@riak_cs_get_fsm:waiting_continue_or_stop:229 Block Servers:
> [<0.28212.18>]
> 2015-08-20 18:40:38.343 [debug]
> <0.28210.18>@riak_cs_get_fsm:waiting_chunks:307 Retrieved block
> {<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,0}
> 2015-08-20 18:40:38.344 [d
Re: Default values vs non-existing keys for CRDTs
Forgot to include the list in the initial response.
Thanks, Dmitri. I'm writing in Scala so I use riak java client. That seem
to make the trick:
val fetchOp = new
FetchValue.Builder(key.location).withOption(FetchValue.Option.HEAD,
java.lang.Boolean.TRUE).build()
val res = module.client.execute(fetchOp)
res.isNotFound should equal(true)
But also means that I need to issue a separate (albeit lightweight) request
to check whether the key exists before fetching a CRDT value. I'm also not
entirely sure whether it is compatible with CRDT, based on the Russel's
comment below.
On Wed, Aug 19, 2015 at 8:48 PM, Dmitri Zagidulin
wrote:
> From what I understand, this is a limitation of that particular client
> (what language is that, by the way?). Feel free to open an issue on Github
> for it.
>
> The HTTP API, at least, does distinguish between a non-existent counter
> and a counter whose value happens to be 0.
>
> For example, here's the result of trying to access an existing key (with a
> value set to 0):
>
> curl
> http://localhost:8098/types/counters/buckets/room_occupancy/datatypes/room-215
> {"type":"counter","value":0}
>
> And here's the result of retrieving the value of a non-existent key:
>
> curl
> http://localhost:8098/types/counters/buckets/room_occupancy/datatypes/non-existent-room
> {"type":"counter","error":"notfound"}
>
> There's a workaround you can do with your existing client, however (until
> the issue with the FetchCounter request gets fixed).
>
> You can issue a regular Fetch Object operation (preferably setting the
> 'HEAD only' option to true) for that counter. A Riak counter (or any other
> data type) still exists as a regular object, and you can issue a HEAD
> request to it (as opposed to a Fetch Counter or whatever), and it'll return
> a 404 as expected.
>
> On Wed, Aug 19, 2015 at 8:59 PM, Timur Fayruzov
> wrote:
>
>> Hello,
>>
>> It seems that Riak Datatype API does not allow to distinguish between
>> non-existing keys and default values. For example, if I query for
>> non-existing key as follows:
>>
>> val fetchOp = new FetchCounter.Builder(key).build()
>> val c = client.execute(fetchOp).getDatatype
>>
>> I'll get a counter that holds 0. Now, if I put counter with value 0 at
>> this key and run the query, I get the same result. Is there any way to
>> distinguish between these two different states?
>>
>> Note: when working with sets there is a context that I can check. If I
>> fetch a set and the context is null, it means that the set does not exist
>> under this key. This trick does not work for counters though, as they do
>> not maintain context. Is it a valid trick to use though?
>>
>> I posted this to StackOverflow a while ago, but no luck:
>> http://stackoverflow.com/questions/31845164/riak-dataypes-default-values-vs-non-existing-keys
>>
>> Thanks,
>>
>> Timur
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Default values vs non-existing keys for CRDTs
Forgot to include the list in the initial response. > On Thu, Aug 20, 2015 at 12:31 AM, Russell Brown > wrote: > >> It’s a tricky one Timur. CRDTs are based on Join Semi-Lattices, and they >> have the concept of a bottom value. That is what you are seeing. Implicitly >> all keys exist at the bottom value until you operate on them. > > Thanks, Russell. I was under impression that key mechanism does not differ between 'regular' KV store and CRDT, that is checking key existence does not involve CRDT concepts. Is that not true? > > >> Since you cannot do a compare-and-set operation with a counter anyway >> (that is you can’t “put counter with value 0” at that key), why do you need >> to distinguish between 0 as bottom and 0 as value? > > I'm building a Scala wrapper for our services to use and trying to provide a uniform simple interface for CRDT counters and sets (maps are a whole different story). One of the fundamental operations is to be able to tell whether a key exists in the store. Although it's not possible to 'set' counter to 0, update '+1' followed by update '-1' at 'key' will result in 'key' with value 0, which is semantically different from answer "no 'key' found". Could you also comment whether null context for a set is a valid way to tell that set did not exist in store? It feels hacky. On 20 Aug 2015, at 01:59, Timur Fayruzov wrote: > > > Hello, > > > > It seems that Riak Datatype API does not allow to distinguish between > non-existing keys and default values. For example, if I query for > non-existing key as follows: > > > > val fetchOp = new FetchCounter.Builder(key).build() > > val c = client.execute(fetchOp).getDatatype > > > > I'll get a counter that holds 0. Now, if I put counter with value 0 at > this key and run the query, I get the same result. Is there any way to > distinguish between these two different states? > > > > Note: when working with sets there is a context that I can check. If I > fetch a set and the context is null, it means that the set does not exist > under this key. This trick does not work for counters though, as they do > not maintain context. Is it a valid trick to use though? > > > > I posted this to StackOverflow a while ago, but no luck: > http://stackoverflow.com/questions/31845164/riak-dataypes-default-values-vs-non-existing-keys > > > > Thanks, > > > > Timur > > > > ___ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: s3cmd error: access to bucket was denied
Kazuhiro,
Maybe that's not the key point. I'm using riak 1.4.2 and follow below docs
to configure "s3cfg" file.
http://docs.basho.com/riakcs/1.4.2/cookbooks/configuration/Configuring-an-S3-Client/#Sample-s3cmd-Configuration-File-for-Production-Use
There's no "signature_v2" parameter in "s3cfg". However, I added this
parameter to "s3cfg" and tried again with same errors.
On Thu, Aug 20, 2015 at 10:31 PM, Kazuhiro Suzuki wrote:
> Hi Changmao,
>
> It seems your s3cmd config should include 2 items:
>
> signature_v2 = True
> host_base = api2.cloud-datayes.com
>
> Riak CS requires "signature_v2 = True" since Riak CS has not supported
> s3 authentication version 4 yet.
> You can find a sample configuration of s3cmd here to interact with Riak CS
> [1].
>
> [1]:
> http://docs.basho.com/riakcs/2.0.1/cookbooks/configuration/Configuring-an-S3-Client/#Sample-s3cmd-Configuration-File-for-Production-Use
>
> Thanks,
>
> On Thu, Aug 20, 2015 at 7:44 PM, changmao wang
> wrote:
> > Just now, I used "admin_key" and "admin_secret" from
> /etc/riak-cs/app.config
> > to run "s3cmd -c s3-stock ls s3://stock/XSHE/0/000600"
> > and I got the below error:
> > ERROR: Access to bucket 'stock' was denied
> >
> > Below is abstract from "/var/log/riak-cs/console.log"
> > 2015-08-20 18:40:22.790 [debug]
> > <0.28085.18>@riak_cs_s3_auth:calculate_signature:129 STS:
> > ["GET","\n",[],"\n",[],"\n","\n",[["x-amz-date",":",<<"Thu, 20 Aug 2015
> > 10:40:22 +">>,"\n"]],["/stock/",[]]]
> > 2015-08-20 18:40:32.861 [error]
> > <0.28153.18>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user
> > record for s3 failed. Reason: no_user_key
> > 2015-08-20 18:40:32.861 [debug]
> > <0.28153.18>@riak_cs_wm_common:post_authentication:452 No user key
> > 2015-08-20 18:40:32.969 [debug] <0.28189.18>@riak_cs_get_fsm:prepare:406
> > Manifest:
> >
> {lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,122,47,83,90,48,48,50,49,55,53,67,78,47,50,48,49,48,95,50,48,49,48,45,48,52,45,50,52,95,229,133,172,229,143,184,231,171,160,231,168,139,239,188,136,50,48,49,48,229,185,180,52,230,156,136,239,188,137,46,80,68,70>>},[],"2013-12-16T23:01:12.000Z",<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,255387,<<"application/pdf">>,<<55,41,141,170,187,226,47,223,183,95,105,129,155,154,210,202>>,active,{1387,234872,598819},{1387,234872,918555},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,234872,598546}},[],undefined}
> > 2015-08-20 18:40:33.043 [debug]
> > <0.28189.18>@riak_cs_lfs_utils:range_blocks:118 InitialBlock: 0,
> FinalBlock:
> > 0
> > 2015-08-20 18:40:33.043 [debug]
> > <0.28189.18>@riak_cs_lfs_utils:range_blocks:120 SkipInitial: 0,
> KeepFinal:
> > 255387
> > 2015-08-20 18:40:33.050 [debug]
> > <0.28189.18>@riak_cs_get_fsm:waiting_continue_or_stop:229 Block Servers:
> > [<0.28191.18>]
> > 2015-08-20 18:40:33.079 [debug]
> > <0.28189.18>@riak_cs_get_fsm:waiting_chunks:307 Retrieved block
> > {<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0}
> > 2015-08-20 18:40:33.079 [debug]
> > <0.28189.18>@riak_cs_get_fsm:perhaps_send_to_user:280 Returning block
> > {<<192,71,150,153,181,181,77,61,186,41,100,32,5,91,197,166>>,0} to client
> > 2015-08-20 18:40:38.218 [error]
> > <0.28086.18>@riak_cs_wm_common:maybe_create_user:223 Retrieval of user
> > record for s3 failed. Reason: no_user_key
> > 2015-08-20 18:40:38.218 [debug]
> > <0.28086.18>@riak_cs_wm_common:post_authentication:452 No user key
> > 2015-08-20 18:40:38.226 [debug] <0.28210.18>@riak_cs_get_fsm:prepare:406
> > Manifest:
> >
> {lfs_manifest_v3,3,1048576,{<<"pipeline">>,<<100,97,116,97,121,101,115,47,112,105,112,101,108,105,110,101,47,100,97,116,97,47,114,101,112,111,114,116,47,115,104,47,83,72,54,48,48,55,53,48,67,78,47,50,48,48,55,95,50,48,48,55,45,49,49,45,50,49,95,230,177,159,228,184,173,232,141,175,228,184,154,229,133,179,228,186,142,73,66,69,95,53,232,141,175,229,147,129,232,142,183,229,190,151,228,186,140,230,156,159,228,184,180,229,186,138,230,137,185,230,150,135,231,154,132,229,133,172,229,145,138,229,143,138,233,163,142,233,153,169,230,143,144,231,164,186,46,112,100,102>>},[],"2013-12-15T09:04:48.000Z",<<201,247,249,158,95,22,64,242,161,118,253,64,120,187,205,105>>,89863,<<"application/pdf">>,<<139,151,203,173,6,111,222,48,17,81,102,170,216,66,193,77>>,active,{1387,98288,545827},{1387,98288,618409},[],undefined,undefined,undefined,undefined,{acl_v2,{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51","AVG2DHZ4UNUYFAZ8F4WR"},[{{"pipelinewrite","ef38ca69e145a40c1f8378633994192dace4539339315e6b42d7d1e6e2d2de51"},['FULL_CONTROL']},{'AllUsers',['READ']}],{1387,98288,545618}},[],undefined}
>
