Public bug reported: [0] shows a sample image that was left in an inconsistent state when a call to registry failed during image deletion.
Glance v1 API makes two registry calls when deleting an image. The first call [1] is made to to set the status of an image to deleted/pending_delete. And, the other [2], to delete the rest of the metadata, which sets 'deleted_at' and 'deleted' fields in the db. If the first call fails, the image deletion request fails and the image is left intact in it's previous status. However, if the first call succeeds and the second one fails, the image is left in an inconsistent status where it's status is set to pending_delete/deleted but it's 'deleted_at' and 'deleted' fields are not set. If delayed delete is turned on, these images are never collected by the scrubber as they won't appear as deleted images because their deleted field is not set. So, these images will continue to occupy storage in the backend. Also, further attempts at deleting these images will fail with a 404 because the status is already set to pending_delete/deleted. [0] http://paste.openstack.org/show/477577/ [1]: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L1115-L1116 [2]: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L1132 ** Affects: glance Importance: Critical Assignee: Hemanth Makkapati (hemanth-makkapati) Status: Triaged ** Affects: glance/juno Importance: Undecided Status: New ** Affects: glance/kilo Importance: Undecided Status: New ** Affects: glance/liberty Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1511061 Title: Images in inconsistent state when calls to registry fail during image deletion Status in Glance: Triaged Status in Glance juno series: New Status in Glance kilo series: New Status in Glance liberty series: New Bug description: [0] shows a sample image that was left in an inconsistent state when a call to registry failed during image deletion. Glance v1 API makes two registry calls when deleting an image. The first call [1] is made to to set the status of an image to deleted/pending_delete. And, the other [2], to delete the rest of the metadata, which sets 'deleted_at' and 'deleted' fields in the db. If the first call fails, the image deletion request fails and the image is left intact in it's previous status. However, if the first call succeeds and the second one fails, the image is left in an inconsistent status where it's status is set to pending_delete/deleted but it's 'deleted_at' and 'deleted' fields are not set. If delayed delete is turned on, these images are never collected by the scrubber as they won't appear as deleted images because their deleted field is not set. So, these images will continue to occupy storage in the backend. Also, further attempts at deleting these images will fail with a 404 because the status is already set to pending_delete/deleted. [0] http://paste.openstack.org/show/477577/ [1]: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L1115-L1116 [2]: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L1132 To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1511061/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
