Thanks -- ulimit -n 1024 fixed my problem. It's wonderfully stable now -- even
under high load.
On Jul 19, 2010, at 2:30 AM, Daniel Reverri wrote:
> Hi Shane,
>
> Can you try increasing the number of allowed open file handles on your system?
> ulimit -n 1024
>
> You can check this setting from a riak console by typing:
> os:cmd("ulimit -a").
>
> Thanks,
> Dan
>
> Sent from my iPhone
>
> On Jul 18, 2010, at 10:32 PM, Shane Holloway wrote:
>
>> I'm having a bit of a problem with Riak. When I keep my new documents to
>> about 10 per second, Riak works fine. But as I increase my speed to about
>> 100 docs per second, Riak gets dramatically slow. After a while, the beam
>> process takes 100% of my CPU and no longer responds to ping/stop or any
>> other command from the bin folder. I end up sending kill TERM to stop the
>> process.
>>
>> If I restart Riak, the bitcask backend seems to be limited to about 1-2
>> document stores per second, but read seems fairly responsive still. I'd
>> appreciate any thoughts or troubleshooting tips you may have. It's an
>> in-development database with demo data, so I can run any experiments you'd
>> like to see.
>>
>> Thanks,
>> -Shane
>>
>>
>>
>> Riak/riakOne% ./bin/riak-admin status
>> 1-minute stats for 'r...@172.20.0.49'
>> ---
>> vnode gets : 104
>> vnode_puts : 103
>> vnode_gets_total : 136
>> vnode_puts_total : 127
>> node_gets : 36
>> node_gets_total : 47
>> node_get_fsm_time_mean : 2635.3
>> node_get_fsm_time_median : 667
>> node_get_fsm_time_95 : 9156
>> node_get_fsm_time_99 : 40652
>> node_get_fsm_time_100 : 40652
>> node_puts : 36
>> node_puts_total : 44
>> node_put_fsm_time_mean : 1670575.6
>> node_put_fsm_time_median : 1157
>> node_put_fsm_time_95 : 23900
>> node_put_fsm_time_99 : 6511
>> node_put_fsm_time_100 : 6511
>> cpu_nprocs : 0
>> cpu_avg1 : 0
>> cpu_avg5 : 0
>> cpu_avg15 : 0
>> mem_total : 0
>> mem_allocated : 0
>> disk : [{"none",0,0}]
>> nodename : 'r...@172.20.0.49'
>> connected_nodes : ['riak_maint_61...@172.20.0.49']
>> sys_driver_version : <<"1.5">>
>> sys_global_heaps_size : 0
>> sys_heap_type : private
>> sys_logical_processors : 2
>> sys_otp_release : <<"R13B04">>
>> sys_process_count : 167
>> sys_smp_support : true
>> sys_system_version : <<"Erlang R13B04 (erts-5.7.5) [source] [64-bit]
>> [smp:2:2] [rq:2] [async-threads:5] [hipe] [kernel-poll:true]\n">>
>> sys_system_architecture : <<"i386-apple-darwin9.4.0">>
>> sys_threads_enabled : true
>> sys_thread_pool_size : 5
>> sys_wordsize : 8
>> ring_members : ['r...@172.20.0.49']
>> ring_num_partitions : 64
>> ring_ownership : <<"[{'r...@172.20.0.49',64}]">>
>> ring_creation_size : 64
>> storage_backend : riak_kv_bitcask_backend
>> pbc_connects_total : 0
>> pbc_connects : 0
>> pbc_active : 0
>>
>> ___
>> riak-users mailing list
>> riak-users@lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
