Object not found after successful PUT on S3 API

2017-01-25 Thread Daniel Miller
I have a 9-node Riak CS cluster that has been working flawlessly for about 3 months. The cluster configuration, including backend and bucket parameters such as N-value are using default settings. I'm using the S3 API to communicate with the cluster. Within the past week I had an issue where two ob

Re: Object not found after successful PUT on S3 API

2017-01-25 Thread Luke Bakken
Hi Daniel - This is a strange scenario. I recommend looking at all of the log files for "[error]" or other entries at about the same time as these PUTs or 404 responses. Is there anything unusual about the key being used? -- Luke Bakken Engineer lbak...@basho.com On Wed, Jan 25, 2017 at 6:40 AM

Re: Active Anti Entropy Directory when AAE is disabled

2017-01-25 Thread Arun Rajagopalan
Thanks Luke. Sorry it took me some time to experiment ... I am not sure what happens in a couple of scenarios. Maybe you can explain Lets say I lose a node completely and want to replace it. Will the keys yet to be "anti-entropied" by that node be distributed correctly when I restore that node ?

Re: Object not found after successful PUT on S3 API

2017-01-25 Thread Daniel Miller
Thanks for the quick response, Luke. There is nothing unusual about the keys. The format is a name + UUID + some other random URL-encoded charaters, like most other keys in our cluster. There are no errors near the time of the incident in any of the logs (the last [error] is from over a month bef