Pearl1594 opened a new pull request #4748: URL: https://github.com/apache/cloudstack/pull/4748
### Description Addresses: https://github.com/apache/cloudstack/issues/4470 This PR fixes the issue noticed while deleting a template in a zone and also adds cleanup, in order to remove such templates left behind. The following is currently observed currently: In a Zone with only 1 store: - when a template (private/public) is deleted, the template is marked as Inactive, but not removed In a Zone with > 1 store: - when a private template is deleted, the template is marked as Inactive, but not removed. For eg, after deletion, the following is observed: ``` MariaDB [cloud]> select id,type,name,uuid,format,public,size,state,created,removed from vm_template where id =204\G *************************** 1. row *************************** id: 204 type: USER name: template-private-2storesZone uuid: d899e386-fd23-4c8b-9834-9cac9cd6febe format: QCOW2 public: 0 size: 52428800 state: Inactive created: 2021-03-04 06:09:22 removed: NULL 1 row in set (0.00 sec) MariaDB [cloud]> select id,template_id,store_id,created,state,size,store_role,install_path from template_store_ref where template_id =204\G *************************** 1. row *************************** id: 18 template_id: 204 store_id: 1 created: 2021-03-04 06:09:22 state: Destroyed size: 52428800 store_role: Image install_path: template/tmpl/2/204/1ba40914-9eba-33de-93b0-8c05ea6fc753.qcow2 1 row in set (0.01 sec) ``` - When a public template is deleted, the templates are set as destroyed on the secondary store (template_store_ref), but it continues to be marked as Active in the vm_template table. For example, On deletion: ``` MariaDB [cloud]> select id,type,name,uuid,format,public,size,state,created,removed from vm_template where id =205\G *************************** 1. row *************************** id: 205 type: USER name: template-public-2storesZone uuid: 25cbaa26-d114-46d7-8285-cb1180fb5cee format: QCOW2 public: 1 size: 52428800 state: Active created: 2021-03-04 06:11:52 removed: NULL 1 row in set (0.00 sec) MariaDB [cloud]> select id,template_id,store_id,created,state,size,store_role,install_path from template_store_ref where template_id =205\G *************************** 1. row *************************** id: 19 template_id: 205 store_id: 2 created: 2021-03-04 06:11:52 state: Destroyed size: 52428800 store_role: Image install_path: template/tmpl/2/205/848ef57e-80a3-3b79-a8c9-a7ef7316165a.qcow2 *************************** 2. row *************************** id: 20 template_id: 205 store_id: 1 created: 2021-03-04 06:11:52 state: Destroyed size: 52428800 store_role: Image install_path: template/tmpl/2/205/40db7f18-3d06-3527-b7f3-6db5811a3121.qcow2 2 rows in set (0.01 sec) ``` **Notice template still appears Active when destroyed on secondary store** Post fix: On deletion of a public template registered in a zone with > 1 image store (here, 2) ``` MariaDB [cloud]> select id,type,name,uuid,format,public,size,state,created,removed from vm_template where id =208\G *************************** 1. row *************************** id: 208 type: USER name: template-public-2storesPostFix uuid: 2e60aba8-4daf-4dee-9626-d78ccaeca158 format: QCOW2 public: 1 size: 52428800 state: Inactive created: 2021-03-04 07:14:37 removed: 2021-03-04 07:16:01 1 row in set (0.00 sec) MariaDB [cloud]> select id,template_id,store_id,created,state,size,store_role,install_path from template_store_ref where template_id =208\G *************************** 1. row *************************** id: 24 template_id: 208 store_id: 1 created: 2021-03-04 07:14:37 state: Destroyed size: 52428800 store_role: Image install_path: template/tmpl/2/208/38b47a1b-7e92-3851-952b-99b9e898b268.qcow2 *************************** 2. row *************************** id: 25 template_id: 208 store_id: 2 created: 2021-03-04 07:14:37 state: Destroyed size: 52428800 store_role: Image install_path: template/tmpl/2/208/8f30693b-c3a7-3f68-be23-dc3f0ced082e.qcow2 2 rows in set (0.00 sec) ``` With this patch, when the cleanup job is triggered, it also cleans up all templates that are marked as Inactive but not removed (reduced the cleanup interval by modifying the following global settings: `storage.cleanup.interval` and `storage.cleanup.delay`) ``` MariaDB [cloud]> select id,type,name,uuid,format,public,size,state,created,removed from vm_template where id in (202, 203, 204, 205)\G *************************** 1. row *************************** id: 202 type: USER name: template-private uuid: 195264b5-65e9-41ba-9c65-c992c5f34046 format: QCOW2 public: 0 size: 52428800 state: Inactive created: 2021-03-04 05:47:30 removed: 2021-03-04 07:07:54 *************************** 2. row *************************** id: 203 type: USER name: template-public uuid: e8039473-25c2-4254-b4fb-25b74ea6ef44 format: QCOW2 public: 1 size: 52428800 state: Inactive created: 2021-03-04 05:58:26 removed: 2021-03-04 07:07:54 *************************** 3. row *************************** id: 204 type: USER name: template-private-2storesZone uuid: d899e386-fd23-4c8b-9834-9cac9cd6febe format: QCOW2 public: 0 size: 52428800 state: Inactive created: 2021-03-04 06:09:22 removed: 2021-03-04 07:07:54 *************************** 4. row *************************** id: 205 type: USER name: template-public-2storesZone uuid: 25cbaa26-d114-46d7-8285-cb1180fb5cee format: QCOW2 public: 1 size: 52428800 state: Inactive created: 2021-03-04 06:11:52 removed: 2021-03-04 07:07:54 ``` Also tested a scenario where in a template can be present in other zones as well. In such a case, when a public template is destroyed in a specific zone, the template is left as Active - if it present in other zones: ``` MariaDB [cloud]> select * from template_zone_ref where template_id = 212\G *************************** 1. row *************************** id: 24 zone_id: 1 template_id: 212 created: 2021-03-04 07:55:08 last_updated: 2021-03-04 07:55:08 removed: 2021-03-04 07:57:11 *************************** 2. row *************************** id: 25 zone_id: 2 template_id: 212 created: 2021-03-04 07:56:23 last_updated: 2021-03-04 07:56:23 removed: NULL 2 rows in set (0.00 sec) MariaDB [cloud]> select id,type,name,uuid,format,public,size,state,created,removed from vm_template where id =212\G *************************** 1. row *************************** id: 212 type: USER name: test-multizone uuid: 0ef1505a-8863-4728-87fd-cbb0c8884cc2 format: QCOW2 public: 1 size: 52428800 state: Active created: 2021-03-04 07:55:08 removed: NULL 1 row in set (0.00 sec) MariaDB [cloud]> select id,template_id,store_id,created,state,size,store_role,install_path from template_store_ref where template_id =212\G *************************** 1. row *************************** id: 37 template_id: 212 store_id: 3 created: 2021-03-04 07:55:08 state: Destroyed size: 52428800 store_role: Image install_path: template/tmpl/2/212/18fe8ef9-565c-3cf0-b418-885bdd6ade55.qcow2 *************************** 2. row *************************** id: 38 template_id: 212 store_id: 1 created: 2021-03-04 07:55:08 state: Destroyed size: 52428800 store_role: Image install_path: template/tmpl/2/212/c4fada96-bf51-356d-b47a-3efb95e71f84.qcow2 2 rows in set (0.01 sec) ``` <!-- For new features, provide link to FS, dev ML discussion etc. --> <!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. --> <!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged --> <!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" --> <!-- Fixes: # --> <!--- ********************************************************************************* --> <!--- NOTE: AUTOMATATION USES THE DESCRIPTIONS TO SET LABELS AND PRODUCE DOCUMENTATION. --> <!--- PLEASE PUT AN 'X' in only **ONE** box --> <!--- ********************************************************************************* --> ### Types of changes - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] New feature (non-breaking change which adds functionality) - [X] Bug fix (non-breaking change which fixes an issue) - [ ] Enhancement (improves an existing feature and functionality) - [ ] Cleanup (Code refactoring and cleanup, that may add test cases) #### Bug Severity - [ ] BLOCKER - [ ] Critical - [X] Major - [] Minor - [ ] Trivial ### How Has This Been Tested? Observations and tests mentioned above ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
