shwstppr opened a new pull request, #8999: URL: https://github.com/apache/cloudstack/pull/8999
### Description Feature spec: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Purge+Expunged+Resources This PR introduces the functionality of purging removed DB entries for CloudStack entities (currently only for VirtualMachine). There would be three mechanisms for purging removed resources: - Background task - CloudStack will run a background task which runs at a defined interval. Other parameters for this task can be controlled with new global settings. - API - New admin-only API `purgeExpungedResources`. It will allow passing the following parameters - `resourcetype`, `batchsize`, `startdate`, `enddate`. Currently, API is not supported in the UI. - Config for service offering. Service offerings can be created with `purgeresources` parameter which would allow purging resources immediately on expunge. Following new global settings have been added: - `expunged.resources.purge.enabled`: Default: false. Whether to run a background task to purge the DB records of the expunged resources. - `expunged.resources.purge.resources`: Default: (empty). A comma-separated list of resource types that will be considered by the background task to purge the DB records of the expunged resources. Currently only VirtualMachine is supported. An empty value will result in considering all resource types for purging. - `expunged.resources.purge.interval`: Default: 86400. Interval (in seconds) for the background task to purge the DB records of the expunged resources. - `expunged.resources.purge.delay`: Default: 300. Initial delay (in seconds) to start the background task to purge the DB records of the expunged resources task. - `expunged.resources.purge.batch.size`: Default: 50. Batch size to be used during purging of the DB records of the expunged resources. - `expunged.resources.purge.start.time`: Default: (empty). Start time to be used by the background task to purge the DB records of the expunged resources. Use format `yyyy-MM-dd` or `yyyy-MM-dd HH:mm:ss`. - `expunged.resources.purge.keep.past.days`: Default: 30. The number of days in the past from the execution time of the background task to purge the DB records of the expunged resources for which the expunged resources must not be purged. To enable purging DB records of the expunged resource till the execution of the background task, set the value to zero. - `expunged.resource.purge.job.delay`: Default: 180. Delay (in seconds) to execute the purging of the DB records of an expunged resource initiated by the configuration in the offering. Minimum value should be 180 seconds and if a lower value is set then the minimum value will be used. Documentation PR: https://github.com/apache/cloudstack-documentation/pull/397 ### Types of changes - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] New feature (non-breaking change which adds functionality) - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Enhancement (improves an existing feature and functionality) - [ ] Cleanup (Code refactoring and cleanup, that may add test cases) - [ ] build/CI ### Feature/Enhancement Scale or Bug Severity #### Feature/Enhancement Scale - [ ] Major - [ ] Minor #### Bug Severity - [ ] BLOCKER - [ ] Critical - [ ] Major - [ ] Minor - [ ] Trivial ### Screenshots (if appropriate): ### How Has This Been Tested? <!-- Please describe in detail how you tested your changes. --> <!-- Include details of your testing environment, and the tests you ran to --> #### How did you try to break this feature and the system with this change? <!-- see how your change affects other areas of the code, etc. --> API usage: ``` > purge expungedresources startdate=2024-04-15 enddate=2024-04-20 resourcetype=VirtualMachine { "purgeexpungedresourcesresponse": { "resourcecount": 6 } } ``` Verified new smoke test, ``` [root@pr8999-t6925-kvm-centos7-marvin marvin]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=./pr8999-t6925-kvm-centos7-advanced-cfg -s -a tags=advanced --hypervisor=KVM tests/smoke/test_purge_expunged_vms.py /usr/local/lib/python3.6/site-packages/paramiko/transport.py:32: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6. from cryptography.hazmat.backends import default_backend ==== Marvin Init Started ==== === Marvin Parse Config Successful === === Marvin Setting TestData Successful=== ==== Log Folder Path: /marvin/MarvinLogs/May_01_2024_08_09_34_TNMKB9 All logs will be available here ==== === Marvin Init Logging Successful=== ==== Marvin Init Successful ==== === TestName: test_01_purge_expunged_api_vm_start_date | Status : SUCCESS === === TestName: test_02_purge_expunged_api_vm_end_date | Status : SUCCESS === === TestName: test_03_purge_expunged_api_vm_start_end_date | Status : SUCCESS === === TestName: test_04_purge_expunged_api_vm_no_date | Status : SUCCESS === === TestName: test_05_purge_expunged_vm_service_offering | Status : SUCCESS === {'mgtSvrIp': '10.1.33.150', 'port': 8080, 'user': 'root', 'passwd': 'P@ssword123', 'hypervisor': 'kvm', 'useHttps': 'False', 'certCAPath': 'NA', 'certPath': 'NA', 'apiKey': 'YZDWI356FRnLh8vEHROd_ZUVD5U8JeGG-mObb7wPhyL5OQqi6df3KCmF5AlVWxPsZvsvNY2-z9BbyqSZGPaBCg', 'securityKey': 'U4LP9-ZdTuRl4_hJl3x2KN6nTNZCYAWhcMXmFKBSBYTNoUghjj3Pb5-0s2hN_tVG0l8oK6jDrmp953NOP59GAg'} ====Trying SSH Connection: Host:10.1.33.150 User:root Port:22 RetryCnt:60=== ===SSH to Host 10.1.33.150 port : 22 SUCCESSFUL=== {Cmd: service cloudstack-management stop via Host: 10.1.33.150} {returns: ['Redirecting to /bin/systemctl stop cloudstack-management.service']} {Cmd: service cloudstack-management start via Host: 10.1.33.150} {returns: ['Redirecting to /bin/systemctl start cloudstack-management.service']} ====Trying SSH Connection: Host:10.1.35.104 User:root Port:22 RetryCnt:60=== ====Trying SSH Connection: Host:10.1.35.104 User:root Port:22 RetryCnt:60=== ===SSH to Host 10.1.35.104 port : 22 SUCCESSFUL=== ===SSH to Host 10.1.35.104 port : 22 SUCCESSFUL=== {Cmd: service cloudstack-management stop via Host: 10.1.35.104} {returns: ['Redirecting to /bin/systemctl stop cloudstack-management.service']} {Cmd: service cloudstack-management start via Host: 10.1.35.104} {returns: ['Redirecting to /bin/systemctl start cloudstack-management.service']} {'mgtSvrIp': '10.1.33.150', 'port': 8080, 'user': 'root', 'passwd': 'P@ssword123', 'hypervisor': 'kvm', 'useHttps': 'False', 'certCAPath': 'NA', 'certPath': 'NA', 'apiKey': 'YZDWI356FRnLh8vEHROd_ZUVD5U8JeGG-mObb7wPhyL5OQqi6df3KCmF5AlVWxPsZvsvNY2-z9BbyqSZGPaBCg', 'securityKey': 'U4LP9-ZdTuRl4_hJl3x2KN6nTNZCYAWhcMXmFKBSBYTNoUghjj3Pb5-0s2hN_tVG0l8oK6jDrmp953NOP59GAg'} ====Trying SSH Connection: Host:10.1.33.150 User:root Port:22 RetryCnt:60=== ===SSH to Host 10.1.33.150 port : 22 SUCCESSFUL=== {Cmd: service cloudstack-management stop via Host: 10.1.33.150} {returns: ['Redirecting to /bin/systemctl stop cloudstack-management.service']} {Cmd: service cloudstack-management start via Host: 10.1.33.150} {returns: ['Redirecting to /bin/systemctl start cloudstack-management.service']} ====Trying SSH Connection: Host:10.1.35.104 User:root Port:22 RetryCnt:60=== ====Trying SSH Connection: Host:10.1.35.104 User:root Port:22 RetryCnt:60=== ===SSH to Host 10.1.35.104 port : 22 SUCCESSFUL=== ===SSH to Host 10.1.35.104 port : 22 SUCCESSFUL=== {Cmd: service cloudstack-management stop via Host: 10.1.35.104} {returns: ['Redirecting to /bin/systemctl stop cloudstack-management.service']} {Cmd: service cloudstack-management start via Host: 10.1.35.104} {returns: ['Redirecting to /bin/systemctl start cloudstack-management.service']} === TestName: test_06_purge_expunged_vm_background_task | Status : SUCCESS === ``` <!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
