CorrugatedIron v1.4.1 released

2013-08-18 Thread Jeremiah Peschka 
CorrugatedIron v1.4.1 has been released. This is a minor release and is
mainly an enhancement to bucket options, plus the ability to delete based
on a RiakObject.

Download from nuget [1] or build from source [2]

## Release notes for v1.4.1

* Support added to bucket properties for Replication Mode
(#132
)
* Support added to bucket properties for `big_vclock` and `little_vclock` (
#131 )
* `IRiakClient.Delete` accepts a RiakObject
(#158)
- it's now possible to pass in a full RiakObject and have CorrugatedIron
generate the correct RiakObjectId for deletion

[1]: http://www.nuget.org/packages/CorrugatedIron/
[2]: http://github.com/DistributedNonsense/CorrugatedIron/
---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Riak Memory Bloat issues with RiakCS/BitCask

2013-08-18 Thread Idan Shinberg 
Hi all

We have a ~300GB Riak Single Node Cluster
This seems to have worked fine ( merging worked good ) until an
OpenFile/OpenPorts limit was reached ( since then , we've tweaked both to
64K )
The above error caused a crash that left corrupted hint files .We've
deleted the hint ( and their corrosponding the data files ) to allow a
clean start to riak ( no errors upon start) .

However , merges have not been really  working  ( taking forever to
complete  ) since then  , therefor causing :

   - Huge Bloat on disk ( Data is around 150K objects of roughly 8MB each ,
   but has already more then quadrupled in size the riak storage used ( around
   1.2 TB )
   - Huge Bloat in memory , which eventually kills riak itself ( OOM killer
   )

We're not doing anything complex , just using riak and riak-cs to emulate
S3 access ( and only it ) for roughly 15 client writes  per minute.

Our merge settings ( uber-low , but have worked correctly in the up till a
few days ago ) :

* {riak_kv, [*
*%% Storage_backend specifies the Erlang module defining the
storage*
*%% mechanism that will be used on this node.*
*{add_paths, ["/usr/lib64/riak-cs/lib/riak_cs-1.3.1/ebin"]},
*
*{storage_backend, riak_cs_kv_multi_backend},*
*{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]},*
*{multi_backend_default, be_default},*
*{multi_backend, [*
*{be_default, riak_kv_eleveldb_backend, [*
*{max_open_files, 50},*
*{data_root, "/var/lib/riak/leveldb"}*
*]},*
*{be_blocks, riak_kv_bitcask_backend, [*
*
*
*{max_file_size, 16#200}, %% 32MB*
*
*
*%% Trigger a merge if any of the following are
true:*
*{frag_merge_trigger, 10}, %% fragmentation >= 10%*
*{dead_bytes_merge_trigger, 8388608}, %% dead bytes
> 8 MB*
*
*
*%% Conditions that determine if a file will be
examined during a merge:*
*{frag_threshold, 5}, %% fragmentation >= 5%*
*{dead_bytes_threshold, 2097152}, %% dead bytes > 2
MB*
*{small_file_threshold, 16#8000}, %% file is <
2GB*
*
*
*{data_root, "/var/lib/riak/bitcask"},*
*{log_needs_merge, true}*
*
*
*
*
*]}*
*]},*

As you've noticed , log_needs_merge is set to true and we do get our logs
filled with needs_merge messages such as this one :

*,{"/var/l...",...},...]*
*2013-08-19 00:09:49.043 [info] <0.17972.0>
"/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728"
needs_merge:
[{"/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/1153.bitcask.data",[{small_file,20506434}]},{"/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/1152.bitcask.data",[{small_file,33393237}]},{"/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/1151.bitcask.data",[{small_file,33123254}]},{"/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/1150.bitcask.data",[{small_file,32505520}]},{"/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/1149.
*
...
...
...

Yet a merge Only a single merge happened  ( and only after around 20
minutes since we started putting pressure on the riak) :

*2013-08-19 00:17:29.456 [info] <0.18964.14> Merged
{["/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/712.bitcask.data","/var/lib/riak/
*
*
bitcask/388211372416021087647853783690262677096107081728/711.bitcask.data","/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/710.bitcask
*
*
.data","/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/709.bitcask.data","/var/lib/riak/bitcask/38821137241602108764785378369026267709
*
*
6107081728/708.bitcask.data","/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/707.bitcask.data","/var/lib/riak/bitcask/388211
*
*...*
*...*
*...*
*var/lib/riak/bitc*
*
ask/388211372416021087647853783690262677096107081728/697.bitcask.data","/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/696.bitcask.dat
*
*
a","/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/695.bitcask.data","/var/lib/riak/bitcask/388211372416021087647853783690262677096107
*
*
081728/694.bitcask.data","/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/693.bitcask.data","/var/lib/riak/bitcask/38821137241602108764
*
*
7853783690262677096107081728/692.bitcask.data","/var/lib/riak/bitcask/388211372416021087647853783690262677096107081728/691.bitcask.data","/var/lib/riak/bitcas
*
*k/38821137241602108...",...],...} in 1325.611982 seconds.*

Is it reasonable for a merge to take more then 20 minutes ?
Especially assuming riak's memory usage is bloating muc