Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

On May 24, 2012, at 11:43 AM, Steve Warren wrote: > Thanks Reid, that's a very clear explanation. Are there any logs created > under the cited circumstances? I'm not seeing any errors or logs that > indicate any of the below conditions are in fact happening and would like to > confirm the exac

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Thanks Reid, that's a very clear explanation. Are there any logs created under the cited circumstances? I'm not seeing any errors or logs that indicate any of the below conditions are in fact happening and would like to confirm the exact condition. I'm also not clear why doing this "PR = PW = R = W

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

I have a pretty good idea what is causing this problem. Riak uses "tombstone" values to denote that an object has been deleted. Under normal conditions, this tombstone value (really, the key/value pair) will be deleted three (3) seconds after the delete. The delete_mode config lets you change the

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Steve, Kyle, In order to track it down, I'd like to be able to reproduce this issue. Can you provide some minimal Java code to reproduce, from an empty Riak cluster? Please include any changes to the Riak app.config as well. Some things to think about in writing code to reproduce: * Are you conn

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Whew! I was afraid I was the only one! On Wed, May 23, 2012 at 1:37 PM, Kyle Kingsbury wrote: > On 05/23/2012 01:08 PM, Steve Warren wrote: > >> I'm seeing this pretty consistently and have no explanation for it. I >> delete a large number of keys (20k to 100k), but when I then search on >> the

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

I have a 5 node cluster and given a successful delete call, I expect to get the latest data back given the bucket properties (as shown below)... Bucket properties: {"props":{"name":"mybucket","allow_mult":false,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"cha

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

On 05/23/2012 01:08 PM, Steve Warren wrote: I'm seeing this pretty consistently and have no explanation for it. I delete a large number of keys (20k to 100k), but when I then search on the keys ($key/0/g) anywhere from 0-200 or so of the deleted keys show up in the results. It doesn't matter how

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Riak is eventually consistent. Deleting it doesn't show up immediately. There is an option like delete_immediate Shuhao On May 23, 2012 4:08 PM, "Steve Warren" wrote: > I'm seeing this pretty consistently and have no explanation for it. I > delete a large number of keys (20k to 100k), but when I

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

I'm seeing this pretty consistently and have no explanation for it. I delete a large number of keys (20k to 100k), but when I then search on the keys ($key/0/g) anywhere from 0-200 or so of the deleted keys show up in the results. It doesn't matter how long I wait after completing the deletion step

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Thank you for the reply. My observation does not quite match up with this though so I'm still a bit confused. The deleted keys appeared to stay long past the 3 seconds described in the post you referenced. In fact, I don't know if they ever "went away". I'll run some more tests to see if I can narr

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Hi Steve. There is no caching of key lists in riak. What you are seeing is likely the fact that listing of keys or index queries can pick up deleted keys due to the fact that riak keeps tombstone markers around for deleted objects for some period. For a really good explanation of riak's delete b

Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

The last message I saw on this (from a year ago) says the caching of key lists will be removed. I just ran into it while running a $key index range search. I then ran a ?key=stream search on the bucket and the same stale key list appeared (I had created a bunch of data and then deleted it as a test

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

I think that configuration key should be a bit more verbose :-)* Jonathan Langevin Systems Administrator Loom Inc. Wilmington, NC: (910) 241-0433 - jlange...@loomlearning.com - www.loomlearning.com - Skype: intel352 * On Thu, May 26, 2011 at 3:12 PM, Sean Cribbs w

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

> > 1) I would rather be hit by a large cost that I can see and feel instead of > trying to run down hidden keys from a stale cache (reflect on chasing memory > corruptions...) I think that's fair; it emphasizes the fact that if you shouldn't use it once, you shouldn't be using it twice! (or

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

On Thu, May 26, 2011 at 11:21 AM, Sean Cribbs wrote: > With all of this discussion it has been pointed out to me there are two > issues at hand, possibly conflated as one: > > * Which is the least surprise, caching the key list or the incurring the > large cost of the operation? Or is it that it

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Kyle, you bring up a good point that I feel strongly about -- most people use list-keys as a substitute for better solutions (also known as an anti-pattern). In most cases what they really need is one of: * Better key/schema design, so keys are at least guessable if not knowable. * Secondary in

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

In software products that have containment metaphors, how often do we see a function return a cached value rather than the up-to-date value, especially for products that manage shared data? Pretty frequently, actually. Every Ruby ORM I've used caches associations by default. Even when listing is

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Additionally, perhaps the automatically updating cache (regarding inserts/deletes) could be an optionally enabled behavior? As there are cases where it could be needlessly expensive (i.e. - high write/delete scenarios), especially when someone does not use the key listing feature. *

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

A cache seems legitimate for performance, but perhaps the cache could additionally be maintained for inserts/deletes? At least then the cache is still being used, but is also accurate. I don't know how expensive that would be though, but hopefully less expensive than a key list reload, correct? *

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

On May 26, 2011, at 12:40 PM, Sean Cribbs wrote: > With recent commits ( > https://github.com/seancribbs/ripple/compare/35d7323fb0e179c8c971...da3ab71a19d194c65a7b > ), it is cached until you either refresh it manually by passing :reload => > true or a block (for streaming key lists). This was

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Agreed. In fact, jrecursive pointed out to me last week that vnode operations are synchronous. That means that when you call list-keys, not only is it going to take a long time (right now upwards of 5 minutes) to complete, but while each vnode is returning its list of keys *it blocks any other

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

With recent commits ( https://github.com/seancribbs/ripple/compare/35d7323fb0e179c8c971...da3ab71a19d194c65a7b ), it is cached until you either refresh it manually by passing :reload => true or a block (for streaming key lists). This was the compromise reached in that pull-request. All of this

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Sean - Thanks for responding so quickly. I posted a response on github (https://github.com/seancribbs/ripple/pull/168). Regards, Keith On May 26, 2011, at 10:35 AM, Sean Cribbs wrote: > Keith, > > There was a pull-request issue out for this on the Github project > (https://github.com/seanc

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

How long is the key list cached like that, naturally?* Jonathan Langevin Systems Administrator Loom Inc. Wilmington, NC: (910) 241-0433 - jlange...@loomlearning.com - www.loomlearning.com - Skype: intel352 * On Thu, May 26, 2011 at 10:35 AM, Sean Cribbs wrote: >

Re: Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

Keith, There was a pull-request issue out for this on the Github project (https://github.com/seancribbs/ripple/pull/168). For various reasons, the list of keys is memoized in the Riak::Bucket instance. Passing :reload => true to the #keys method will cause it to refresh. I like to discourage

Riak Client Resources, Deleting a Key Doesn't Remove it from bucket.keys

All - I just started working with Riak, and am using the riak-client Ruby gem. When I delete a key from a bucket, and try to fetch the value associated with that key, I get a 404 error (which is reasonable). However, it remains in the bucket's list of keys (i.e. the value returned by bucket.ke