On 2015-01-15 11:43 AM, Jesse Keating wrote:
We have a need to better manage the various openstack capacities across
our numerous clouds. We want to be able to detect when capacity of one
system or another is approaching the point where it would be a good idea
to arrange to increase that capacity. Be it volume space, VCPU
capability, object storage space, etc...
What systems are you folks using to monitor and react to such things?
Thanks for bringing up the subject Jesse.
I believe you are not the only one facing this challenge because I am too.
I added the subject to the midcycle ops meetup (Capacity
planning/monitoring) which I hope to be able to attend:
https://etherpad.openstack.org/p/PHL-ops-meetup
We are using host aggregates and have a complex combination of them.
(imaging a venn diagram)
What we do is retrieving all:
- hypervisor stats
- host aggregates
From there, we compute resource usage (vcpus, ram, disk) in any given
host aggregate.
This part is very challenging as we have to partially reimplement
nova-scheduler logic to determine if a given hypervisor has different
resource allocation ratios based on host aggregate attributes.
The result in a table with resource usage percentage (and absolute
numbers) for each host aggregates (and combinations).
Unfortunately, I can't share yet this first tool as my coworker very
tightly integrated it to our internal monitoring tool and wouldn't work
outside it. No promise but I'll try to find time to extract it and share
it with you guys.
We also coded a very primitive tool which takes a flavor name and
compute available "slots" on each hypervisors (regardless of host
aggregate memberships):
https://gist.github.com/mgagne/bc54c3434a119246a88d
This tool is not actively used in our monitoring due to mentioned
limitation as we would again have to partially reimplement
nova-scheduler logic to determine if a given flavor can (or not) be
spawn on a given hypervisor and filter it out from the output if it
can't accept the flavor. Furthermore, it does not take into account
resource allocation ratios based on host aggregates.
Hopefully, other people will join in and share their tools so we can all
improve our OpenStack operations experience.
--
Mathieu
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
