Here's a follow up on the HA rules and especially the HA resource
affinity rules.
The first three patches haven't changed as they were lower priority for
me than the last part about loosening restrictions on mixed resource
references.
Patches #1 - #8 are rather independent patches but still have some
ordering / dependencies in between (e.g. patch #1 - #3 depend on each
other). The larger part here are patches #9 - #12, which allow mixed
resource references between node and resource affinity rules.
Also did a
git rebase HEAD~12 --exec 'make clean && make deb'
on the series.
Changelog to v1
---------------
- add missing return schema to read_rule api
- ignore disable parameter as otherwise rule feasibility tests will be
skipped
- do not check ignored resources as otherwise migrations are not
possible or co-migrations are done even if the resource is not
ha-managed (could be an use case in the future, but not for now)
- change behavior for single resource fails for resources in positive
affinity rules: try to recover resource by migrating all resources to
another available node
- loosen restrictions on mixed resource references, i.e., if a resource
is used in a node affinity rule and a resource affinity rule at the
same time (see more information in the patch)
TODO (more important)
---------------------
- more test cases and dev/user testing
- do not include ignored resources in feasibility check
- updates to pve-docs about the new behavior and missing docs:
- new: mixing node and resource affinity rules are allowed now
- new: interactions between node and resource affinity rules
- new: ha rule conflicts for mixed usage
- missing: for positive resource affinity rules, if resources are
separated across nodes, the most populated one will be chosen as the
one where all are migrated to
- maybe missing: strictness behavior of negative resource affinity
rules, i.e., that one resource cannot be migrated to either where
another negative affinity resource is located nor migrated to
- using node affinity rules and negative resource affinity rules should
still be improved as there are quite a few cases where manual
intervention is needed to follow the resources node affinity rules
(for now, resource affinity rules often times overwrite node affinity
rules; the former should get more information about the latter in the
scheduler; but might not be a blocker as its rather an edge case)
- some of these cases are also represented in the added test cases, so
that changes to them are documented in the future if those cases are
accomodated for as well
TODO (less important)
---------------------
- adding node affinity rule blockers as well (but not really important)
- maybe log not-migrated groups because they had no group members?
- other suggestions to the original patch series'
- cleaning up
Daniel Kral (12):
manager: fix ~revision version check for ha groups migration
test: ha tester: add ha groups migration tests with runtime upgrades
tree-wide: pass optional parameters as hash values for for_each_rule
helper
api: rules: add missing return schema for the read_rule api endpoint
api: rules: ignore disable parameter if it is set to a falsy value
rules: resource affinity: make message in inter-consistency check
clearer
config, manager: do not check ignored resources with affinity when
migrating
rules: make positive affinity resources migrate on single resource
fail
rules: allow same resources in node and resource affinity rules
rules: restrict inter-plugin resource references to simple cases
test: rules: add test cases for inter-plugin checks allowing simple
use cases
test: ha tester: add resource affinity test cases mixed with node
affinity rules
src/PVE/API2/HA/Rules.pm | 12 +-
src/PVE/HA/Config.pm | 2 +
src/PVE/HA/Manager.pm | 11 +-
src/PVE/HA/Rules.pm | 289 +++++++++++++--
src/PVE/HA/Rules/NodeAffinity.pm | 14 +-
src/PVE/HA/Rules/ResourceAffinity.pm | 30 +-
src/PVE/HA/Sim/Hardware.pm | 8 +
...onsistent-node-resource-affinity-rules.cfg | 54 +++
...nt-node-resource-affinity-rules.cfg.expect | 73 ++++
...sistent-resource-affinity-rules.cfg.expect | 8 +-
...egative-resource-affinity-rules.cfg.expect | 4 +-
...y-for-positive-resource-affinity-rules.cfg | 37 ++
...ositive-resource-affinity-rules.cfg.expect | 111 ++++++
...-affinity-with-resource-affinity-rules.cfg | 35 ++
...ty-with-resource-affinity-rules.cfg.expect | 48 +++
.../multiple-resource-refs-in-rules.cfg | 52 ---
...multiple-resource-refs-in-rules.cfg.expect | 111 ------
src/test/test-group-migrate3/README | 7 +
src/test/test-group-migrate3/cmdlist | 17 +
src/test/test-group-migrate3/groups | 7 +
src/test/test-group-migrate3/hardware_status | 5 +
src/test/test-group-migrate3/log.expect | 344 ++++++++++++++++++
src/test/test-group-migrate3/manager_status | 1 +
src/test/test-group-migrate3/service_config | 5 +
src/test/test-group-migrate4/README | 8 +
src/test/test-group-migrate4/cmdlist | 15 +
src/test/test-group-migrate4/groups | 7 +
src/test/test-group-migrate4/hardware_status | 5 +
src/test/test-group-migrate4/log.expect | 277 ++++++++++++++
src/test/test-group-migrate4/manager_status | 1 +
src/test/test-group-migrate4/service_config | 5 +
.../README | 9 +-
.../log.expect | 26 +-
.../README | 14 +
.../cmdlist | 3 +
.../hardware_status | 5 +
.../log.expect | 41 +++
.../manager_status | 1 +
.../rules_config | 7 +
.../service_config | 5 +
.../README | 13 +
.../cmdlist | 3 +
.../hardware_status | 7 +
.../log.expect | 63 ++++
.../manager_status | 1 +
.../rules_config | 7 +
.../service_config | 5 +
.../README | 17 +
.../cmdlist | 6 +
.../hardware_status | 6 +
.../log.expect | 67 ++++
.../manager_status | 1 +
.../rules_config | 15 +
.../service_config | 5 +
.../README | 15 +
.../cmdlist | 3 +
.../hardware_status | 5 +
.../log.expect | 49 +++
.../manager_status | 1 +
.../rules_config | 7 +
.../service_config | 5 +
.../README | 15 +
.../cmdlist | 3 +
.../hardware_status | 5 +
.../log.expect | 49 +++
.../manager_status | 1 +
.../rules_config | 7 +
.../service_config | 5 +
68 files changed, 1852 insertions(+), 248 deletions(-)
create mode 100644
src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg
create mode 100644
src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect
create mode 100644
src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg
create mode 100644
src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect
create mode 100644
src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg
create mode 100644
src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect
delete mode 100644 src/test/rules_cfgs/multiple-resource-refs-in-rules.cfg
delete mode 100644
src/test/rules_cfgs/multiple-resource-refs-in-rules.cfg.expect
create mode 100644 src/test/test-group-migrate3/README
create mode 100644 src/test/test-group-migrate3/cmdlist
create mode 100644 src/test/test-group-migrate3/groups
create mode 100644 src/test/test-group-migrate3/hardware_status
create mode 100644 src/test/test-group-migrate3/log.expect
create mode 100644 src/test/test-group-migrate3/manager_status
create mode 100644 src/test/test-group-migrate3/service_config
create mode 100644 src/test/test-group-migrate4/README
create mode 100644 src/test/test-group-migrate4/cmdlist
create mode 100644 src/test/test-group-migrate4/groups
create mode 100644 src/test/test-group-migrate4/hardware_status
create mode 100644 src/test/test-group-migrate4/log.expect
create mode 100644 src/test/test-group-migrate4/manager_status
create mode 100644 src/test/test-group-migrate4/service_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative1/README
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative1/cmdlist
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative1/hardware_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative1/log.expect
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative1/manager_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative1/rules_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative1/service_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative2/README
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative2/cmdlist
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative2/hardware_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative2/log.expect
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative2/manager_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative2/rules_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative2/service_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative3/README
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative3/cmdlist
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative3/hardware_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative3/log.expect
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative3/manager_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative3/rules_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-negative3/service_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive1/README
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive1/cmdlist
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive1/hardware_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive1/log.expect
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive1/manager_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive1/rules_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive1/service_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive2/README
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive2/cmdlist
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive2/hardware_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive2/log.expect
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive2/manager_status
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive2/rules_config
create mode 100644
src/test/test-resource-affinity-with-node-affinity-strict-positive2/service_config
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
