Public bug reported: Description: The accumulation of instance_group_member records in the nova-api database causes the server list to slow down.
Details: Information about instance group members is currently stored in the instance_group_member table of the nova-api database. Even if an instance belonging to a particular instance group is deleted, it is not removed from the instance_group_member table. There is a mechanism to delete records of deleted instances from instance_group_member using the nova-manage db archive_deleted_rows command. [1] In os-compute-api-version 2.96, it was updated to obtain and display the AZ information from request_specs when listing (showing) servers. [2] At this time, instance_group information is also retrieved. [3] If the records in instance_group_member have grown significantly, the response time for the server list becomes extremely slow. In our environment, there were nearly 20,000 instance_group_member records linked to a single instance group (the cause of which was a bug in the Kubernetes managed service on our OpenStack). References: [1] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/cmd/manage.py#L493-L495 [2] https://review.opendev.org/c/openstack/nova/+/904568/17 [3] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/objects/request_spec.py#L747-L763 Environment: By os-compute-api-version 2.96 Our Reproduction Environment: Dalmatial release (2024.2) ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2106380 Title: Slow server list response with API version 2.96 Status in OpenStack Compute (nova): New Bug description: Description: The accumulation of instance_group_member records in the nova-api database causes the server list to slow down. Details: Information about instance group members is currently stored in the instance_group_member table of the nova-api database. Even if an instance belonging to a particular instance group is deleted, it is not removed from the instance_group_member table. There is a mechanism to delete records of deleted instances from instance_group_member using the nova-manage db archive_deleted_rows command. [1] In os-compute-api-version 2.96, it was updated to obtain and display the AZ information from request_specs when listing (showing) servers. [2] At this time, instance_group information is also retrieved. [3] If the records in instance_group_member have grown significantly, the response time for the server list becomes extremely slow. In our environment, there were nearly 20,000 instance_group_member records linked to a single instance group (the cause of which was a bug in the Kubernetes managed service on our OpenStack). References: [1] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/cmd/manage.py#L493-L495 [2] https://review.opendev.org/c/openstack/nova/+/904568/17 [3] https://github.com/openstack/nova/blob/9d910ec4bf2a12baf3b5f0ec3bc41686413538fb/nova/objects/request_spec.py#L747-L763 Environment: By os-compute-api-version 2.96 Our Reproduction Environment: Dalmatial release (2024.2) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2106380/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
