Calling erlang shell from the command line
Hi RIAK users I would like to attach to the riak shell and compile an .erl program and quit. The reason is I want to be absolutely sure I am building the erl program with the version of erlang that my riak installation has Something like riak attach c(myprog.erl). Ctr-C a Any ideas ? Thanks Arun ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
List all keys on a small bucket
Hello Riak Users I have a use case where I would really like to list all keys of a bucket despite all the warnings about performance. The number of keys is relatively small - in the few thousands at the very most, Usually its no more than 100 I also have other buckets in the same cluster that have millions of keys and tens of tera bytes of data Question: Will listing all keys on the small bucket adversely impact performance of the other larger buckets? Thanks Arun ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reaping Tombstones
Hello Riakers I wonder if there is anyway to delete all tombstones that were left behind when delete_mode was set to 'keep' Also how do I estimate how much space the tombstones take ? We have a ton of large scale and frequent delete operations and I suspect some space is used by Tombstones. I would like to find out if it is significant enough to warrant some cleanup. If it does, how do I remove them? Thanks and wish you all Happy, Peaceful and Prosperous 2017 ! ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Reaping Tombstones
Thanks Matthew & Luca Re: global expiry - will that option retroactively remove objects? That is remove objects that became "unneeded" before the option was set ? Same question w.r.t delete_mode Re: Map / Reduce - I am not sure the delete would remove the tombstone unless I set the delete_mode to immediate AND there are no copies on non-primary nodes. Or am I mistaken ? On Fri, Dec 30, 2016 at 10:11 AM, Luca Favatella < luca.favate...@erlang-solutions.com> wrote: > On 30 December 2016 at 15:06, Matthew Von-Maszewski > wrote: > > Greetings, > > > > I am not able to answer your tombstone questions. That question needs a > > better expert. > > > > Just wanted to point out that Riak now has global expiry in both the > leveldb > > and bitcask backends. That might be a quicker solution for your frequent > > delete operations: > > > > http://basho.com/products/riak-kv/global-object-expiration/ > > > > Technical details for the leveldb expiry are found here: > > https://github.com/basho/leveldb/wiki/mv-expiry > > > > Matthew > > > > On Dec 30, 2016, at 9:55 AM, Arun Rajagopalan < > arun.v.rajagopa...@gmail.com> > > wrote: > > > > Hello Riakers > > > > I wonder if there is anyway to delete all tombstones that were left > behind > > when delete_mode was set to 'keep' > > Hi Arun, > > Brute-force rate-limited map-reduce custom application code looking > for `X-Riak-Deleted` in object metadata and then deleting? Refs: > * http://docs.basho.com/riak/kv/2.2.0/developing/app-guide/ > advanced-mapreduce/#map-phase > * https://github.com/basho/basho_docs/blob/6b2f42a7243bfded737bd62eed12c4 > 90376c67e2/content/riak/kv/2.2.0/developing/app-guide/ > advanced-mapreduce.md#map-phase > > Interested to hear if you come up with a better way though. > > > > Also how do I estimate how much space the tombstones take ? We have a > ton of > > large scale and frequent delete operations and I suspect some space is > used > > by Tombstones. I would like to find out if it is significant enough to > > warrant some cleanup. > > I understand this is a separate question - I am not sure about the answer. > > > > If it does, how do I remove them? > > I believe I proposed an answer above. > > > Regards > Luca > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
CS compatibility with KV 2.2.0
Folks Could somebody mention which versions of CS are compatible with KV 2.2.0 please ? Maybe someone can update this page ? http://docs.basho.com/riak/cs/2.1.1/cookbooks/version-compatibility/#working-version-combinations Thanks Arun ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Active Anti Entropy Directory when AAE is disabled
Hello Riak Users Lets say I stop Active Anti-Entropy by disabling it. Will the node continue to populate the anti_entropy ? This is part of a thinking exercise in case you wonder why I would want to do that :) Thanks Arun ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Active Anti Entropy Directory when AAE is disabled
Thanks Luke. Sorry it took me some time to experiment ... I am not sure what happens in a couple of scenarios. Maybe you can explain Lets say I lose a node completely and want to replace it. Will the keys yet to be "anti-entropied" by that node be distributed correctly when I restore that node ? Secondly restore multiple nodes from a backup, should I replace the anti-entropy directory also ? Arun Rajagopalan 215-901-5989 On Wed, Jan 18, 2017 at 3:22 PM, Luke Bakken wrote: > Hi Arun - > > I don't know the answer off the top of my head, but I suspect that > disabling AAE will leave that directory and the files in it untouched > afterward. > > One way to find out would be to disable AAE and monitor the access > time of the files in the anti_entropy directory. > > -- > Luke Bakken > Engineer > lbak...@basho.com > > On Wed, Jan 18, 2017 at 11:49 AM, Arun Rajagopalan > wrote: > > Hello Riak Users > > > > Lets say I stop Active Anti-Entropy by disabling it. Will the node > continue > > to populate the anti_entropy ? > > > > This is part of a thinking exercise in case you wonder why I would want > to > > do that :) > > > > Thanks > > Arun > > > > ___ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Truncated bit-cask files
Hello Riak Users We have situations where we dont or cant gracefully stop riak. When that happens we occasionally get a truncated last-record in bitcask files If I delete those bitcask dir and the anti_entropy directory, Riak rebuilds those bitcask files correctly Is there a way to rectify those borken bitcask files ? Thanks Arun ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Truncated bit-cask files
Hi Magnus
RIAK crashes on startup when I have trucated bitcask file
It also crashes when the AAE files are bad too I think. Example below
2017-02-13 21:18:30 =CRASH REPORT
crasher:
initial call: riak_kv_index_hashtree:init/1
pid: <0.6037.0>
registered_name: []
exception exit: {{{badmatch,{error,{db_open,"Corruption: truncated
record at end of file"}}},[{hashtree,new_segment_
store,2,[{file,"src/hashtree.erl"},{line,675}]},{hashtree,new,2,[{file,"src/hashtree.erl"},{line,246}]},{riak_kv_index_h
ashtree,do_new_tree,3,[{file,"src/riak_kv_index_hashtree.erl"},{line,610}]},{lists,foldl,3,[{file,"lists.erl"},{line,124
8}]},{riak_kv_index_hashtree,init_trees,3,[{file,"src/riak_kv_index_hashtree.erl"},{line,474}]},{riak_kv_index_hashtree,
init,1,[{file,"src/riak_kv_index_hashtree.erl"},{line,268}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}
,{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line
,328}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}
ancestors: [<0.715.0>,riak_core_vnode_sup,riak_core_sup,<0.160.0>]
messages: []
links: []
dictionary: []
trap_exit: false
status: running
heap_size: 1598
stack_size: 27
reductions: 889
neighbours:
Regards
Arun
On Tue, Feb 14, 2017 at 4:48 AM, Magnus Kessler wrote:
> On 13 February 2017 at 19:56, Arun Rajagopalan <
> arun.v.rajagopa...@gmail.com> wrote:
>
>> Hello Riak Users
>>
>> We have situations where we dont or cant gracefully stop riak. When that
>> happens we occasionally get a truncated last-record in bitcask files
>>
>> If I delete those bitcask dir and the anti_entropy directory, Riak
>> rebuilds those bitcask files correctly
>>
>> Is there a way to rectify those borken bitcask files ?
>>
>> Thanks
>> Arun
>>
>>
>>
> Hi Arun,
>
> There should be no need to remove truncated bitcask files. Any objects up
> to the point of truncation should still be available to Riak. However, it
> may take longer for the affected partition to start up, as the
> corresponding hint file will not match the data file, and Riak will scan
> the latter to populate the key set it keeps in memory.
>
> By removing anti_entropy files you are forcing the AAE trees to be
> rebuilt. Once this has completed, AAE will fill back any objects missing
> due to the truncated bitcask files. You can also force a full partition
> repair by following the instructions from the documentation [0].
>
> Can you let me know why you cannot shut down the node gracefully? Unclean
> shutdowns should be a last resort and not part of normal operating
> procedures.
>
> Kind Regards,
>
> Magnus
>
>
> [0]: http://docs.basho.com/riak/kv/2.2.0/using/repair-recovery/
> repairs/#repairing-partitions.
>
> --
> Magnus Kessler
> Client Services Engineer
> Basho Technologies Limited
>
> Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Truncated bit-cask files
Thanks Matthew. I will try one of those solutions
On Tue, Feb 14, 2017 at 3:51 PM, Matthew Von-Maszewski
wrote:
> Arun,
>
> You are running out of RAM for the leveldb AAE. There are several ways to
> fix that:
>
> - reduce memory allocated to bitcask
> - more memory per server
> - more servers of same memory
> - reduce the ring size from 64 to 8, and rebuild data within the cluster
> from scratch
> - lie to leveldb and give it a big than real memory setting in riak.conf:
> leveldb.maximum_memory=8G
>
>
> The key LOG lines are:
>
> Options.total_leveldb_mem: 2,901,766,963<-- this is the total memory
> assigned to ALL of leveldb, but
> only 20% of it goes to AAE vnodes
>
> File cache size: 5833527 <-- the first vnode says, cool enough memory
> for me
> Block cache size: 7930679 <-- ditto
>
> ... but as more vnodes start:
>
> File cache size: 0<-- things are just not going to work
> well
> Block cache size: 0
>
> There are no actual file system error messages in your LOG files. That
> supports that the real problem is memory unhappiness.
>
> Matthew
>
>
> On Feb 14, 2017, at 3:34 PM, Arun Rajagopalan <
> arun.v.rajagopa...@gmail.com> wrote:
>
> Hi Matthew, Magnus
>
> I have attached the log files for your review
>
> Thanks
> Arun
>
>
> On Tue, Feb 14, 2017 at 11:55 AM, Matthew Von-Maszewski <
> matth...@basho.com> wrote:
>
>> Arun,
>>
>> The AAE code uses leveldb for its storage of anti-entropy data, no matter
>> which backend holds the user data. Therefore the error below suggests
>> corruption within leveldb files (which is not impossible, but becoming
>> really rare except with bad hardware or full disks).
>>
>> Before wiping out the AAE directory, you should copy the LOG file within
>> it. There are likely more useful error messages within that file ... maybe
>> put the file in drop box or zip attach to a reply for us to review.
>>
>> Matthew
>>
>> On Feb 14, 2017, at 10:42 AM, Magnus Kessler wrote:
>>
>> On 14 February 2017 at 14:46, Arun Rajagopalan <
>> arun.v.rajagopa...@gmail.com> wrote:
>>
>>> Hi Magnus
>>>
>>> RIAK crashes on startup when I have trucated bitcask file
>>>
>>> It also crashes when the AAE files are bad too I think. Example below
>>>
>>> 2017-02-13 21:18:30 =CRASH REPORT
>>> crasher:
>>> initial call: riak_kv_index_hashtree:init/1
>>> pid: <0.6037.0>
>>> registered_name: []
>>> exception exit: {{{badmatch,{error,{db_open,"Corruption: truncated
>>> record at end of file"}}},[{hashtree,new_segment_
>>> store,2,[{file,"src/hashtree.erl"},{line,675}]},{hashtree,ne
>>> w,2,[{file,"src/hashtree.erl"},{line,246}]},{riak_kv_index_h
>>> ashtree,do_new_tree,3,[{file,"src/riak_kv_index_hashtree.erl
>>> "},{line,610}]},{lists,foldl,3,[{file,"lists.erl"},{line,124
>>> 8}]},{riak_kv_index_hashtree,init_trees,3,[{file,"src/riak_k
>>> v_index_hashtree.erl"},{line,474}]},{riak_kv_index_hashtree,
>>> init,1,[{file,"src/riak_kv_index_hashtree.erl"},{line,268}]}
>>> ,{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]}
>>> ,{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,23
>>> 9}]}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line
>>> ,328}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{
>>> line,239}]}]}
>>> ancestors: [<0.715.0>,riak_core_vnode_sup,riak_core_sup,<0.160.0>]
>>> messages: []
>>> links: []
>>> dictionary: []
>>> trap_exit: false
>>> status: running
>>> heap_size: 1598
>>> stack_size: 27
>>> reductions: 889
>>> neighbours:
>>>
>>>
>>> Regards
>>> Arun
>>>
>>>
>> Hi Arun,
>>
>> The crash log you provided shows that there is a corrupted file in the
>> AAE (anti_entropy) backend. Entries in console.log should have more
>> information about which partition is affected. Please post output from the
>> affected node at around 2017-02-13T21:18:30. As this is AAE data, it is
>> safe to remove the directory named after the affected partition from the
>> active_entropy directory before restarting the node. You may find that
>> there is more than one affected partition, the next of which will be
>> encountered after the attempted restart only. If this is the case, simply
>> identify the next partition in the same way and remove it, too, until the
>> node starts up successfully again.
>>
>> Is there a reason why the nodes aren't shut down in the regular way?
>>
>> Kind Regards,
>>
>> Magnus
>>
>>
>>
>> --
>> Magnus Kessler
>> Client Services Engineer
>> Basho Technologies Limited
>>
>> Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>
>
>
>
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
