that's a hex number or 2147483648 decimal
Chad DePue
inakanetworks.com - development consulting | skype cdepue | @chaddepue
+1 206.866.5707
On Mon, Jun 13, 2011 at 7:08 PM, Steve Webb wrote:
> Dan -
>
> Q: What does the syntax: 16#8000 represent in the max_file_size
> parameter? It's supposed to be 2GB, but I can't see where that means 2GB
> anywhere.
>
> Even if that meant 16 files of 80MB each, that only comes out to slightly
> over 1GB.
>
>
> - Steve
>
> --
> Steve Webb - Senior System Administrator for gnip.com
> http://twitter.com/GnipWebb
>
> On Mon, 13 Jun 2011, Dan Reverri wrote:
>
> Hi Steve,
>>
>> The article points out that the active data file is not considered during
>> merge checks. Your 250-ish MB data file is the active file and not
>> considered during the merge check. The file will eventually role over to a
>> non-active file when it hits 2 GB in size. Once the file is not active it
>> will be considered during the merge check and merging will take place.
>>
>> The 2 GB file size is configurable via the max_file_size parameter:
>> https://github.com/basho/bitcask/blob/master/ebin/bitcask.app#L22
>>
>> Thanks,
>> Dan
>>
>> Daniel Reverri
>> Developer Advocate
>> Basho Technologies, Inc.
>> d...@basho.com
>>
>>
>> On Mon, Jun 13, 2011 at 2:38 PM, Steve Webb wrote:
>>
>> Dan -
>>>
>>> I've got dead_bytes_threshold=5242880 (5M) and
>>> dead_bytes_merge_trigger=10242880. My bitcask *.data files are 250-ish
>>> MB
>>> in size:
>>>
>>> root@ha2
>>> :/data/riaksearch/bitcask/1027618338748291114361965898003636498195577569280#
>>> ls -lah
>>> total 771M
>>> drwxr-xr-x 2 riak riak 4.0K 2011-06-12 01:08 .
>>> drwxr-xr-x 34 riak riak 4.0K 2011-06-12 01:10 ..
>>> -rw--- 1 riak riak 229M 2011-06-08 13:11 1307415077.bitcask.data
>>> -rw-r--r-- 1 riak riak 4.3M 2011-06-08 13:11 1307415077.bitcask.hint
>>> -rw--- 1 riak riak 276M 2011-06-10 13:30 1307562153.bitcask.data
>>> -rw-r--r-- 1 riak riak 5.1M 2011-06-10 13:30 1307562153.bitcask.hint
>>> -rw--- 1 riak riak 1.4M 2011-06-08 13:45 1307562333.bitcask.data
>>> -rw-r--r-- 1 riak riak 27K 2011-06-08 13:45 1307562333.bitcask.hint
>>> -rw--- 1 riak riak 246M 2011-06-13 15:34 1307862506.bitcask.data
>>> -rw-r--r-- 1 riak riak 9.4M 2011-06-13 15:34 1307862506.bitcask.hint
>>> -rw--- 1 riak riak 107 2011-06-12 01:08 bitcask.write.lock
>>>
>>> I'm pretty sure that 50% or more of the data in these files should've
>>> aged-off by now and the merge trigger should've happened. The article
>>> shows
>>> why merges happen when a restart is done, but it doesn't really explain
>>> why
>>> merges don't happen at normal runtime.
>>>
>>> I really don't want to restart riak every day to merge files.
>>>
>>> Q: What are some good trigger settings for my use case?
>>>
>>> I want to collect and store 1 day worth of tweets from the twitter
>>> spritzer
>>> feed and have the data files auto-merge once in a while (once a day or
>>> more
>>> frequently) when they've gotten 10% of 'dead' data in them (aka, the
>>> tweets
>>> expire after 1 day).
>>>
>>>
>>> - Steve
>>>
>>> --
>>> Steve Webb - Senior System Administrator for gnip.com
>>> http://twitter.com/GnipWebb
>>>
>>> On Mon, 13 Jun 2011, Dan Reverri wrote:
>>>
>>> Hi Steve,
>>>
>>>>
>>>> This Knowledge Base article may be related:
>>>>
>>>>
>>>> https://help.basho.com/entries/20141178-why-does-it-seem-that-bitcask-merging-is-only-triggered-when-a-riak-node-is-restarted
>>>>
>>>> Thanks,
>>>> Dan
>>>>
>>>> Daniel Reverri
>>>> Developer Advocate
>>>> Basho Technologies, Inc.
>>>> d...@basho.com
>>>>
>>>>
>>>> On Mon, Jun 13, 2011 at 10:25 AM, Steve Webb wrote:
>>>>
>>>> Justin -
>>>>
>>>>>
>>>>> My current bitcask settings are:
>>>>>
>>>>> %% Bitcask Config
>>>>> {bitcask, [
>>>>> {data_root, "/var/lib/riaksearch/bitcask" },
>>>>> {dead_bytes_merge_trigger, 10242880 },
>>>>> {dead_b
