Hello. Do you have any update about this issue? I am affected by the same memory leak. Sadly, this problem does the Riak memory backend useless in production.
You can reproduce it with this dirty script (be careful, it is resource intensive): https://gist.github.com/indpnday/76d99c5852e0100c5a04 - 1 Riak node 2.0.1. - riak.conf: buckets.default.n_val = 1 buckets.default.last_write_wins = true buckets.default.r = 1 buckets.default.w = 1 multi_backend.memory.storage_backend = memory multi_backend.memory.memory_backend.ttl = 90s multi_backend.memory.memory_backend.max_memory_per_vnode = 250MB # curl http://127.0.0.1:8098/buckets/memory/props {"props":{"allow_mult":false,"backend”:"memory","basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dvv_enabled":false,"dw":"quorum","last_write_wins":true,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":1,"name”:"memory","notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":"quorum","small_vclock":50,"w":1,"young_vclock":20}} Is there any bug report opened in github where to follow this issue? Best regards, Juan Luis Francés 2014-10-20 15:43 GMT+02:00 Luke Bakken: >Lucas, > >Thanks for all the detailed information. This is not expected >behavior. What MIME type are you using for storing the long integer >data (64 binary bits, I assume)? > >I'd like to try and reproduce this. There have been issues with TTL >and max_memory but they should have been fixed for Riak 2.0. >-- >Luke Bakken >Engineer / CSE >lbak...@basho.com > > >On Mon, Oct 20, 2014 at 1:56 AM, Lucas Grijander > Hi Luke, > > Indeed, when removed the thousands of requests, the memory is stabilized. > However the memory consumption is still very high: > > riak-admin status |grep memory > memory_total : 18494760128 > memory_processes : 145363184 > memory_processes_used : 142886424 > memory_system : 18349396944 > memory_atom : 561761 > memory_atom_used : 554496 > memory_binary : 7108243240 > memory_code : 13917820 > memory_ets : 11200328880 > > I have test also with Riak 1.4.10 and the performance is the same. > > Is it normal that the "memory_ets" has more than 10GB when we have a > "ring_size" of 16 and a max_memory_per_vnode = 250MB? > > 2014-10-15 20:50 GMT+02:00 Lucas Grijander: >> >> Hi Luke. >> >> About the first issue: >> >> - From the beginning, the servers are all running ntpd. They are Ubuntu >> 14.04 and the ntpd service is installed and running by default. >> - Anti-entropy was also disabled from the beginning: >> >> {anti_entropy,{off,[]}}, >> >> >> About the second issue, I am perplex because, after 2 restarts of the Riak >> server, just now there is a big memory consumption but is not growing like >> the previous days. The only change was to remove this code (it was used >> thousands of times/s). It was a possible workaround about the previous >> problem with the TTL but this code now is useless because the TTL is working >> fine with this node alone: >> >> self.db.delete((key) >> self.db.get(key, r=1) >> >> >> # riak-admin status|grep memory >> memory_total : 18617871264 >> memory_processes : 224480232 >> memory_processes_used : 222700176 >> memory_system : 18393391032 >> memory_atom : 561761 >> memory_atom_used : 552862 >> memory_binary : 7135206080 >> memory_code : 13779729 >> memory_ets : 11209256232 >> >> The problem is that I don't remember if the code change was after or >> before the second restart. I am going to restart the riak server again and I >> will report you about if the "possible memory leak" is reproduced. >> >> This is the props of the bucket: >> >> {"props":{"allow_mult":false,"backend":"ttl_stg","basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dvv_enabled":false,"dw":"quorum","last_write_wins":true,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":1,"name":"ttl_stg","notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":1,"rw":"quorum","small_vclock":50,"w":1,"young_vclock":20}} >> >> About the data that we put into the bucket are all with this schema: >> >> KEY: Alphanumeric with a length of 47 >> DATA: Long integer. >> >> # riak-admin status|grep puts >> vnode_puts : 84708 >> vnode_puts_total : 123127430 >> node_puts : 83169 >> node_puts_total : 123128062 >> >> # riak-admin status|grep gets >> vnode_gets : 162314 >> vnode_gets_total : 240433213 >> node_gets : 162317 >> node_gets_total : 240433216 _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
