Re: Create Bucket failed
Salman -
Please use "reply all" to include the mailing list in the discussion.
This is the first time you have mentioned a "failed server". Can you
go into more detail? How many servers are in this cluster?
--
Luke Bakken
Engineer
lbak...@basho.com
On Mon, Jul 18, 2016 at 2:39 AM, Salman Khaleghian wrote:
> Hello
> After the failed server come back and we add it, the problem was solved. But
> I do not understand the reason. any idea please.
>
>
> On Thu, 14 Jul 2016 17:25:34 +0430 Luke Bakken wrote
>
>
> Salman -
>
> Can you provide more detailed debugging logs from s3cmd or some way to
> reproduce this?
>
> --
> Luke Bakken
> Engineer
> lbak...@basho.com
>
>
> On Wed, Jul 13, 2016 at 3:34 AM, Salman Khaleghian
> wrote:
>> Hello
>> I use cloudberry and s3cmd both. Both of them show internal server error.
>> Bests
>>
>>
>> On Tue, 12 Jul 2016 19:33:37 +0430 Luke Bakken
>> wrote
>>
>>
>> What tool are you using to create buckets? If you can provide debug
>> output, it looks as though the message sent to Riak CS is bad ("error,
>> malformed_xml")
>>
>> --
>> Luke Bakken
>> Engineer
>> lbak...@basho.com
>>
>>
>> On Sat, Jul 9, 2016 at 1:11 AM, s251251251 wrote:
>>> Hello
>>> after some day after riak-cs Installation, I can not create bucket.
>>> Server
>>> Error is:
>>>
>>> 2016-07-09 12:36:15.401 [error] <0.796.0> Webmachine error at path
>>> "/buckets/test" :
>>>
>>>
>>> {error,{error,{badmatch,{error,malformed_xml}},[{riak_cs_s3_response,xml_error_code,1,[{file,"src/riak_cs_s3_response.erl"},{line,396}]},{riak_cs_s3_response,error_response,1,[{file,"src/riak_cs_s3_response.erl"},{line,273}]},{riak_cs_wm_bucket,accept_body,2,[{file,"src/riak_cs_wm_bucket.erl"},{line,130}]},{riak_cs_wm_common,accept_body,2,[{file,"src/riak_cs_wm_common.erl"},{line,342}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,...},...]}}
>>> in riak_cs_s3_response:xml_error_code/1 line 396
>>>
>>> however i can get and put files. stanchion started.
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Create Bucket failed
Hi Salman, Please re-read the Riak CS instructions carefully. You *must* only have one Stanchion service running in your entire cluster: https://docs.basho.com/riak/cs/2.1.1/cookbooks/installing/#installing-stanchion-on-a-node Based on your latest email, it sounds as though Stanchion is running on every node. -- Luke Bakken Engineer lbak...@basho.com On Mon, Jul 18, 2016 at 8:00 AM, Salman Khaleghian wrote: > We have 3 servers. One of them failed. And we set it as down server. As I > know and think it is should not be problem. So i did not mention it and > focuse on other parts. Today after the server come back suddenly the problem > was solved. Snachion service was ok on two working server. AND we set that > node as down. > > Original message > From: Luke Bakken > Date: 07/18/2016 7:26 PM (GMT+03:30) > To: Salman Khaleghian > Cc: riak-users > Subject: Re: Create Bucket failed > > Salman - > > Please use "reply all" to include the mailing list in the discussion. > > This is the first time you have mentioned a "failed server". Can you > go into more detail? How many servers are in this cluster? > > -- > Luke Bakken > Engineer > lbak...@basho.com > > On Mon, Jul 18, 2016 at 2:39 AM, Salman Khaleghian > wrote: >> Hello >> After the failed server come back and we add it, the problem was solved. >> But >> I do not understand the reason. any idea please. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak bitcask calculation
Hi Travis, Could you go into detail about how you're coming up with 9GiB per node? Is this from the output of the "free" command? Bitcask uses the operating system's buffers for file operations, and will happily use as much free ram as it can get to speed up operations. However, the OS will use that memory for other programs that need it should that need arise. If you're using Linux, these settings may improve bitcask performance in your cluster: http://docs.basho.com/riak/kv/2.1.4/using/performance/#optional-i-o-settings Benchmarking before and after making changes is the recommended way to proceed. Thanks - -- Luke Bakken Engineer lbak...@basho.com On Fri, Jul 15, 2016 at 8:48 AM, Travis Kirstine wrote: > I've put ~74 million objects in my riak cluster with a bucket size of 9 > bytes and keys size of 21 bytes. According the riak capacity calculator > this should require ~4 GiB of RAM per node. Right now my servers are > showing ~ 9 GiB used per node. Is this caused by hashing or something > else.. > > # capacity calculator output > > To manage your estimated 73.9 million key/bucket pairs where bucket names > are ~9 bytes, keys are ~21 bytes, values are ~36 bytes and you are setting > aside 16.0 GiB of RAM per-node for in-memory data management within a > cluster that is configured to maintain 3 replicas per key (N = 3) then Riak, > using the Bitcask storage engine, will require at least: > > 5 nodes > 3.9 GiB of RAM per node (19.7 GiB total across all nodes) > 11.4 GiB of storage space per node (56.8 GiB total storage space used across > all nodes) ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: riak bitcask calculation
Hi Travis - The calculation provided for bitcask memory consumption is only a rough guideline. Using more memory than the calculation suggests is normal and expected with Riak. As you increase load on this cluster memory use may go up further as the operating system manges disk operations and buffers. Is there a reason you're concerned about this usage? -- Luke Bakken Engineer lbak...@basho.com On Mon, Jul 18, 2016 at 8:28 AM, Travis Kirstine wrote: > Yes from the free command > > [root@riak1 ~]# free -g > totalusedfree shared buff/cache > available > Mem: 45 9 0 0 36 > 35 > Swap:23 0 22 > > Or from top > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND > 24421 riak 20 0 16.391g 8.492g 41956 S 82.7 18.5 10542:10 beam.smp > > > I don't think that we are IO bound > dstat > > total-cpu-usage -dsk/total- -net/total- ---paging-- ---system-- > usr sys idl wai hiq siq| read writ| recv send| in out | int csw > 0 0 99 0 0 0| 150k 633k| 0 0 | 1B 25B| 702 2030 > 0 5 95 0 0 0| 0 0 | 10k 2172B| 0 0 |1125 765 > 2 6 92 0 0 0| 0 0 | 213k 135k| 0 0 |2817 7502 > 2 5 92 0 0 0| 0 0 | 159k 88k| 0 0 |2758 9834 > 2 5 93 0 0 0| 0 4884k| 278k 70k| 0 0 |2923 7453 > 0 5 95 0 0 0|4096B 10M| 21k 1066B| 0 0 |3121 781 > 4 7 89 0 0 0| 010M| 258k 160k| 0 0 | 13k 16k > 0 5 95 0 0 0| 0 4096B| 200k 65k| 0 0 |1413 1589 > 1 5 92 1 0 0| 026k| 287k 206k| 0 0 |2124 4990 > 1 4 95 0 0 0| 0 2048B| 67k 78k| 0 0 |1667 4504 > 1 4 95 0 0 0| 0 1560k| 102k 105k| 0 0 |1639 4146 > 3 8 88 1 0 0| 086M| 453k 335k| 0 0 |609716k > 4 14 81 0 0 0| 015k| 635k 564k| 0 0 |538314k > 0 4 96 0 0 0| 0 0 | 29k 1697B| 0 0 |1121 769 > 4 7 89 0 0 0| 0 0 | 339k 376k| 0 0 |801715k > 5 16 79 0 0 0| 011M| 847k 824k| 0 0 | 13k 30k > 2 12 86 1 0 0|4096B 10M| 301k 272k| 0 0 |463911k > 3 10 87 0 0 0| 010M| 508k 610k| 0 0 |826017k > 2 9 87 2 0 0| 013k| 523k 354k| 0 0 |343210k > 0 4 96 0 0 0| 0 0 |3434B 1468B| 0 0 |1063 774 ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RE: riak bitcask calculation
Yes from the free command [root@riak1 ~]# free -g totalusedfree shared buff/cache available Mem: 45 9 0 0 36 35 Swap:23 0 22 Or from top PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 24421 riak 20 0 16.391g 8.492g 41956 S 82.7 18.5 10542:10 beam.smp I don't think that we are IO bound dstat total-cpu-usage -dsk/total- -net/total- ---paging-- ---system-- usr sys idl wai hiq siq| read writ| recv send| in out | int csw 0 0 99 0 0 0| 150k 633k| 0 0 | 1B 25B| 702 2030 0 5 95 0 0 0| 0 0 | 10k 2172B| 0 0 |1125 765 2 6 92 0 0 0| 0 0 | 213k 135k| 0 0 |2817 7502 2 5 92 0 0 0| 0 0 | 159k 88k| 0 0 |2758 9834 2 5 93 0 0 0| 0 4884k| 278k 70k| 0 0 |2923 7453 0 5 95 0 0 0|4096B 10M| 21k 1066B| 0 0 |3121 781 4 7 89 0 0 0| 010M| 258k 160k| 0 0 | 13k 16k 0 5 95 0 0 0| 0 4096B| 200k 65k| 0 0 |1413 1589 1 5 92 1 0 0| 026k| 287k 206k| 0 0 |2124 4990 1 4 95 0 0 0| 0 2048B| 67k 78k| 0 0 |1667 4504 1 4 95 0 0 0| 0 1560k| 102k 105k| 0 0 |1639 4146 3 8 88 1 0 0| 086M| 453k 335k| 0 0 |609716k 4 14 81 0 0 0| 015k| 635k 564k| 0 0 |538314k 0 4 96 0 0 0| 0 0 | 29k 1697B| 0 0 |1121 769 4 7 89 0 0 0| 0 0 | 339k 376k| 0 0 |801715k 5 16 79 0 0 0| 011M| 847k 824k| 0 0 | 13k 30k 2 12 86 1 0 0|4096B 10M| 301k 272k| 0 0 |463911k 3 10 87 0 0 0| 010M| 508k 610k| 0 0 |826017k 2 9 87 2 0 0| 013k| 523k 354k| 0 0 |343210k 0 4 96 0 0 0| 0 0 |3434B 1468B| 0 0 |1063 774 -Original Message- From: Luke Bakken [mailto:lbak...@basho.com] Sent: July-18-16 11:14 AM To: Travis Kirstine Cc: riak-users@lists.basho.com; ac...@jdbarnes.com Subject: Re: riak bitcask calculation Hi Travis, Could you go into detail about how you're coming up with 9GiB per node? Is this from the output of the "free" command? Bitcask uses the operating system's buffers for file operations, and will happily use as much free ram as it can get to speed up operations. However, the OS will use that memory for other programs that need it should that need arise. If you're using Linux, these settings may improve bitcask performance in your cluster: http://docs.basho.com/riak/kv/2.1.4/using/performance/#optional-i-o-settings Benchmarking before and after making changes is the recommended way to proceed. Thanks - -- Luke Bakken Engineer lbak...@basho.com On Fri, Jul 15, 2016 at 8:48 AM, Travis Kirstine wrote: > I've put ~74 million objects in my riak cluster with a bucket size of > 9 bytes and keys size of 21 bytes. According the riak capacity > calculator this should require ~4 GiB of RAM per node. Right now my > servers are showing ~ 9 GiB used per node. Is this caused by hashing > or something else.. > > # capacity calculator output > > To manage your estimated 73.9 million key/bucket pairs where bucket > names are ~9 bytes, keys are ~21 bytes, values are ~36 bytes and you > are setting aside 16.0 GiB of RAM per-node for in-memory data > management within a cluster that is configured to maintain 3 replicas > per key (N = 3) then Riak, using the Bitcask storage engine, will require at > least: > > 5 nodes > 3.9 GiB of RAM per node (19.7 GiB total across all nodes) > 11.4 GiB of storage space per node (56.8 GiB total storage space used > across all nodes) ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RE: riak bitcask calculation
Yes, the reason I'm concerned is that we projected much lower memory usage based on the calculations. We originally provisioned 2x the required memory and it appears that this will not be enough. Am I correct in that the top cmd's RES memory for the beam.smp command is the memory being used by riak for "key storage", if the server was to be rebooted the memory would eventually climb back to this level? Thanks for your help -Original Message- From: Luke Bakken [mailto:lbak...@basho.com] Sent: July-18-16 11:35 AM To: Travis Kirstine Cc: riak-users@lists.basho.com; ac...@jdbarnes.com Subject: Re: riak bitcask calculation Hi Travis - The calculation provided for bitcask memory consumption is only a rough guideline. Using more memory than the calculation suggests is normal and expected with Riak. As you increase load on this cluster memory use may go up further as the operating system manges disk operations and buffers. Is there a reason you're concerned about this usage? -- Luke Bakken Engineer lbak...@basho.com On Mon, Jul 18, 2016 at 8:28 AM, Travis Kirstine wrote: > Yes from the free command > > [root@riak1 ~]# free -g > totalusedfree shared buff/cache > available > Mem: 45 9 0 0 36 > 35 > Swap:23 0 22 > > Or from top > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND > 24421 riak 20 0 16.391g 8.492g 41956 S 82.7 18.5 10542:10 beam.smp > > > I don't think that we are IO bound > dstat > > total-cpu-usage -dsk/total- -net/total- ---paging-- ---system-- > usr sys idl wai hiq siq| read writ| recv send| in out | int csw > 0 0 99 0 0 0| 150k 633k| 0 0 | 1B 25B| 702 2030 > 0 5 95 0 0 0| 0 0 | 10k 2172B| 0 0 |1125 765 > 2 6 92 0 0 0| 0 0 | 213k 135k| 0 0 |2817 7502 > 2 5 92 0 0 0| 0 0 | 159k 88k| 0 0 |2758 9834 > 2 5 93 0 0 0| 0 4884k| 278k 70k| 0 0 |2923 7453 > 0 5 95 0 0 0|4096B 10M| 21k 1066B| 0 0 |3121 781 > 4 7 89 0 0 0| 010M| 258k 160k| 0 0 | 13k 16k > 0 5 95 0 0 0| 0 4096B| 200k 65k| 0 0 |1413 1589 > 1 5 92 1 0 0| 026k| 287k 206k| 0 0 |2124 4990 > 1 4 95 0 0 0| 0 2048B| 67k 78k| 0 0 |1667 4504 > 1 4 95 0 0 0| 0 1560k| 102k 105k| 0 0 |1639 4146 > 3 8 88 1 0 0| 086M| 453k 335k| 0 0 |609716k > 4 14 81 0 0 0| 015k| 635k 564k| 0 0 |538314k > 0 4 96 0 0 0| 0 0 | 29k 1697B| 0 0 |1121 769 > 4 7 89 0 0 0| 0 0 | 339k 376k| 0 0 |801715k > 5 16 79 0 0 0| 011M| 847k 824k| 0 0 | 13k 30k > 2 12 86 1 0 0|4096B 10M| 301k 272k| 0 0 |463911k > 3 10 87 0 0 0| 010M| 508k 610k| 0 0 |826017k > 2 9 87 2 0 0| 013k| 523k 354k| 0 0 |343210k > 0 4 96 0 0 0| 0 0 |3434B 1468B| 0 0 |1063 774 ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Riak Python Client 2.5.5 released
Hello everyone, I released version 2.5.5 of the Python client today. This fixes a long-standing issue with multi-get and multi-put operations where the thread pool did not shut down cleanly when the interpreter shuts down. Remember to "close()" your RiakClient instances to ensure cleanup. https://pypi.python.org/pypi/riak/2.5.5 https://github.com/basho/riak-python-client/blob/master/RELNOTES.md API docs: http://basho.github.io/riak-python-client/ https://github.com/basho/riak-python-client/releases/tag/2.5.5 Milestone in GH: https://github.com/basho/riak-python-client/issues?q=milestone%3Ariak-python-client-2.5.5 -- Luke Bakken Engineer lbak...@basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
