Thanks for the clarification! Best, -Minglong
On 11/29/13, 4:26 PM, "Samuel Merritt" <s...@swiftstack.com<mailto:s...@swiftstack.com>> wrote: On 11/27/13 9:24 PM, Shao, Minglong wrote: Thanks for your reply! I understand the three PUTs by the proxy server and how the replicator works. What I don’t understand is the update of the container DB. The update is sent by individual object servers which don’t know whether a PUT (from the client’s perspective) succeeds. Consider the following scenario: 1. Proxy server sends three requests to three object servers. 2. One object server writes the object successfully, sends an update to the container DB and an “OK” reply to the proxy server. But the other two fail, so they send “failed” to the proxy server. 3. The proxy server sends back “failed” to the client because it doesn’t meet the quorum. But the container DB still gets the update to insert an entry of this object. The word "failed" is a bit nebulous here, and I think that's the source of the confusion. The proxy server doesn't send "failed" to the client; it sends back an HTTP response with status code 503. This sounds like nitpicking, but it's not. A 503 certainly doesn't indicate success, but neither does it indicate that the requested operation failed. It really doesn't tell the client anything except that something went wrong somewhere; whether that constitutes a failure or not depends on the details. It's possible that the object servers finished the upload successfully, but the final stage of the upload took too long, and the proxy server timed out the requests and sent a 503 to the client. In this case, one could have all 3 (or N, really) replicas of the object successfully written to disk, but the client still got a 503. Is that a failure? It's possible that one object server finished the upload successfully, but the others did not. In this case, the client will get a 503 due to lack of quorum. However, let's say that the one, lonely copy of the object is replicated before its disk malfunctions, and so the cluster ends up with the full 3 (or N) replicas. Is that a failure? It's possible that (again) one object server finished the upload successfully, but the others did not. In this case, the client will get a 503 due to lack of quorum. However, let's say that the one, lonely copy of the object dies a horrible death due to disk malfunction, and so the container listing contains an object that will always 404. Is that a failure? It's possible that the client is uploading an object and sends the last bytes of the request body to the proxy, and before those critical last bytes are sent to the object servers, the proxy's internal NIC fails, the object-server requests all time out, and the client replies with a 503. Is that a failure? (Yes, this one definitely is a failure.) I hope this helps clarify that a 503 indicates neither upload success nor upload failure, but simply that something went wrong somewhere, and that the object may exist and have full durability, may exist with poor durability, or may not exist at all. _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org<mailto:openstack@lists.openstack.org> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
