[Yahoo-eng-team] [Bug 2084562] [NEW] Default charsets/collates have changed in recent versions of MySQL and MariaDB
Public bug reported: Description === Since MySQL 8.0 and MariaDB 10.6, the utf8 charset has been removed and replaced by utf8mb3. A 'utf8' alias is still there to prepare the transition, but next move will be to remove it completely in a future major release. I have also noticed that in MariaDB 11.5, the default collations for a few character sets have changed, including the one for utf8mb3. You can see the default collation set in 11.5 on this page: https://mariadb.com/kb/en/server-system-variables/#character_set_collations The same for MySQL>5.7, the default collation of utf8mb4 has changed (https://dev.mysql.com/doc/refman/9.1/en/server-system-variables.html#sysvar_default_collation_for_utf8mb4). For now, the migrations scripts to upgrade/create OpenStack tables use the utf8 charset without specifying any collation, so it uses the default collation one: https://github.com/openstack/nova/blob/master/nova/db/main/migrations/versions/8f2f1571d55b_initial_version.py#L274-L275. Problems occur when we try to create new tables that have foreign keys referencing previously existing tables. For example, the nova caracal update fails because there is a table create request (share_mapping) that has a foreign key on the existing `instances` table. Since the collations of the 2 columns linked by the foreign key constraints are different, the request fails. Steps to reproduce == We need to have a working OpenStack Nova (2023.2 for example) with a MySQL (<8) or MariaDB (<10.6). Then upgrade your database cluster to a higher version, 8.4 for MySQL or 11.5 for MariaDB for example. Finally try to upgrade Nova from 2023.2 to 2024.1 and run the 'nova-manage db sync' command. Expected result === No error with the 'nova-manage db sync' command. Actual result = $ nova-manage db sync Error: (pymysql.err.OperationalError) (1005, 'Can\'t create table `nova_cell0`.`share_mapping` (errno: 150 "Foreign key constraint is incorrectly formed")') [SQL: CREATE TABLE share_mapping ( created_at DATETIME, updated_at DATETIME, id BIGINT NOT NULL AUTO_INCREMENT, uuid VARCHAR(36), instance_uuid VARCHAR(36), share_id VARCHAR(36), status VARCHAR(32),
[Yahoo-eng-team] [Bug 2085135] [NEW] nova fails to map host which has a provisioned node identity after its nova-compute service was deleted
Public bug reported: Hi, When we delete a nova-compute service, the 'mapped' field in nova_cell1.compute_nodes table stays at 1 instead of 0 while nova_api.host_mappings related entry is completely removed. By letting that 'mapped' field to 1, it prevents nova-manage cell_v2 discover_hosts command to create the missing mapping. And that way, we can't anymore spawn on that compute because it's not linked to any cell. It may be linked to that bug: https://bugs.launchpad.net/nova/+bug/2077070 To fix, we can use the --by-service option: nova-manage cell_v2 discover_hosts --verbose --by-service like mentioned here: https://bugs.launchpad.net/nova/+bug/2077070/comments/7 A more global fix would be to set the 'mapped' field to 0 at nova- compute service deletion. Environment === - fresh Antelope deployment on Debian12 - self provisioned compute id ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2085135 Title: nova fails to map host which has a provisioned node identity after its nova-compute service was deleted Status in OpenStack Compute (nova): New Bug description: Hi, When we delete a nova-compute service, the 'mapped' field in nova_cell1.compute_nodes table stays at 1 instead of 0 while nova_api.host_mappings related entry is completely removed. By letting that 'mapped' field to 1, it prevents nova-manage cell_v2 discover_hosts command to create the missing mapping. And that way, we can't anymore spawn on that compute because it's not linked to any cell. It may be linked to that bug: https://bugs.launchpad.net/nova/+bug/2077070 To fix, we can use the --by-service option: nova-manage cell_v2 discover_hosts --verbose --by-service like mentioned here: https://bugs.launchpad.net/nova/+bug/2077070/comments/7 A more global fix would be to set the 'mapped' field to 0 at nova- compute service deletion. Environment === - fresh Antelope deployment on Debian12 - self provisioned compute id To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2085135/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2073862] Re: test_vmdk_bad_descriptor_mem_limit and test_vmdk_bad_descriptor_mem_limit_stream_optimized fail if qemu-img binary is missing
** Changed in: nova/2024.1
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2073862
Title:
test_vmdk_bad_descriptor_mem_limit and
test_vmdk_bad_descriptor_mem_limit_stream_optimized fail if qemu-img
binary is missing
Status in OpenStack Compute (nova):
Fix Released
Status in OpenStack Compute (nova) 2024.1 series:
Fix Released
Status in OpenStack Compute (nova) antelope series:
Triaged
Status in OpenStack Compute (nova) bobcat series:
Triaged
Bug description:
When qemu-img binary is not present on the system, these tests fail
like we can see on these logs:
==
ERROR:
nova.tests.unit.image.test_format_inspector.TestFormatInspectors.test_vmdk_bad_descriptor_mem_limit
--
pythonlogging:'': {{{
2024-07-23 11:44:54,011 WARNING [oslo_policy.policy] JSON formatted
policy_file support is deprecated since Victoria release. You need to use YAML
format which will be default in future. You can use
``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted
policy file to YAML-formatted in backward compatible way:
https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
2024-07-23 11:44:54,012 WARNING [oslo_policy.policy] JSON formatted
policy_file support is deprecated since Victoria release. You need to use YAML
format which will be default in future. You can use
``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted
policy file to YAML-formatted in backward compatible way:
https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
2024-07-23 11:44:54,015 WARNING [oslo_policy.policy] Policy Rules
['os_compute_api:extensions', 'os_compute_api:os-floating-ip-pools',
'os_compute_api:os-quota-sets:defaults',
'os_compute_api:os-availability-zone:list', 'os_compute_api:limits',
'project_member_api', 'project_reader_api', 'project_member_or_admin',
'project_reader_or_admin', 'os_compute_api:limits:other_project',
'os_compute_api:os-lock-server:unlock:unlock_override',
'os_compute_api:servers:create:zero_disk_flavor',
'compute:servers:resize:cross_cell',
'os_compute_api:os-shelve:unshelve_to_host'] specified in policy files are the
same as the defaults provided by the service. You can remove these rules from
policy files which will make maintenance easier. You can detect these redundant
rules by ``oslopolicy-list-redundant`` tool also.
}}}
Traceback (most recent call last):
File
"/home/jlejeune/dev/pci_repos/stash/nova/nova/tests/unit/image/test_format_inspector.py",
line 408, in test_vmdk_bad_descriptor_mem_limit
self._test_vmdk_bad_descriptor_mem_limit()
File
"/home/jlejeune/dev/pci_repos/stash/nova/nova/tests/unit/image/test_format_inspector.py",
line 382, in _test_vmdk_bad_descriptor_mem_limit
img = self._create_allocated_vmdk(image_size // units.Mi,
File
"/home/jlejeune/dev/pci_repos/stash/nova/nova/tests/unit/image/test_format_inspector.py",
line 183, in _create_allocated_vmdk
subprocess.check_output(
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'qemu-img convert -f raw -O vmdk -o
subformat=monolithicSparse -S 0
/tmp/tmpw0q0ibvj/nova-unittest-formatinspector--monolithicSparse-wz0i4kj1.raw
/tmp/tmpw0q0ibvj/nova-unittest-formatinspector--monolithicSparse-qpo78jee.vmdk'
returned non-zero exit status 127.
==
ERROR:
nova.tests.unit.image.test_format_inspector.TestFormatInspectors.test_vmdk_bad_descriptor_mem_limit_stream_optimized
--
pythonlogging:'': {{{
2024-07-23 11:43:31,443 WARNING [oslo_policy.policy] JSON formatted
policy_file support is deprecated since Victoria release. You need to use YAML
format which will be default in future. You can use
``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted
policy file to YAML-formatted in backward compatible way:
https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
2024-07-23 11:43:31,443 WARNING [oslo_policy.policy] JSON formatted
policy_file support is deprecated since Victoria release. You need to use YAML
format which will be default in future. You can use
``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted
policy file to YAML-formatted in backward compatible way:
https://docs.openstack
[Yahoo-eng-team] [Bug 2075178] [NEW] test_snapshot_running test fails if qemu-img binary is missing
Public bug reported:
When qemu-img binary is not present on the system, this test fails like
we can see on that log:
==
ERROR:
nova.tests.unit.virt.test_virt_drivers.LibvirtConnTestCase.test_snapshot_running
--
pythonlogging:'': {{{
2024-07-30 15:47:15,058 INFO [nova.db.migration] Applying migration(s)
2024-07-30 15:47:15,170 INFO [nova.db.migration] Migration(s) applied
2024-07-30 15:47:15,245 INFO [nova.db.migration] Applying migration(s)
2024-07-30 15:47:15,901 INFO [nova.db.migration] Migration(s) applied
2024-07-30 15:47:15,997 WARNING [oslo_policy.policy] JSON formatted policy_file
support is deprecated since Victoria release. You need to use YAML format which
will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool
to convert existing JSON-formatted policy file to YAML-formatted in backward
compatible way:
https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
2024-07-30 15:47:15,998 WARNING [oslo_policy.policy] JSON formatted policy_file
support is deprecated since Victoria release. You need to use YAML format which
will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool
to convert existing JSON-formatted policy file to YAML-formatted in backward
compatible way:
https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
2024-07-30 15:47:16,000 WARNING [oslo_policy.policy] Policy Rules
['os_compute_api:extensions', 'os_compute_api:os-floating-ip-pools',
'os_compute_api:os-quota-sets:defaults',
'os_compute_api:os-availability-zone:list', 'os_compute_api:limits',
'project_member_api', 'project_reader_api', 'project_member_or_admin',
'project_reader_or_admin', 'os_compute_api:limits:other_project',
'os_compute_api:os-lock-server:unlock:unlock_override',
'os_compute_api:servers:create:zero_disk_flavor',
'compute:servers:resize:cross_cell',
'os_compute_api:os-shelve:unshelve_to_host'] specified in policy files are the
same as the defaults provided by the service. You can remove these rules from
policy files which will make maintenance easier. You can detect these redundant
rules by ``oslopolicy-list-redundant`` tool also.
2024-07-30 15:47:17,245 INFO [os_vif] Loaded VIF plugins: linux_bridge, noop,
ovs
2024-07-30 15:47:17,426 INFO [nova.virt.libvirt.driver] Creating image(s)
2024-07-30 15:47:17,560 INFO [nova.virt.libvirt.host] kernel doesn't support
AMD SEV
2024-07-30 15:47:17,642 INFO [nova.virt.libvirt.driver] Instance spawned
successfully.
2024-07-30 15:47:17,711 INFO [nova.virt.libvirt.driver] Beginning live snapshot
process
}}}
Traceback (most recent call last):
File "/home/jlejeune/dev/pci_repos/stash/nova/nova/virt/libvirt/driver.py",
line 3110, in snapshot
metadata['location'] = root_disk.direct_snapshot(
File "/usr/lib/python3.10/unittest/mock.py", line 1114, in __call__
return self._mock_call(*args, **kwargs)
File "/usr/lib/python3.10/unittest/mock.py", line 1118, in _mock_call
return self._execute_mock_call(*args, **kwargs)
File "/usr/lib/python3.10/unittest/mock.py", line 1173, in _execute_mock_call
raise effect
NotImplementedError: direct_snapshot() is not implemented
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/jlejeune/dev/pci_repos/stash/nova/nova/tests/unit/virt/test_virt_drivers.py",
line 60, in wrapped_func
return f(self, *args, **kwargs)
File
"/home/jlejeune/dev/pci_repos/stash/nova/nova/tests/unit/virt/test_virt_drivers.py",
line 242, in test_snapshot_running
self.connection.snapshot(self.ctxt, instance_ref, img_ref['id'],
File "/home/jlejeune/dev/pci_repos/stash/nova/nova/virt/libvirt/driver.py",
line 3149, in snapshot
self._live_snapshot(context, instance, guest,
File "/home/jlejeune/dev/pci_repos/stash/nova/nova/virt/libvirt/driver.py",
line 3402, in _live_snapshot
libvirt_utils.create_image(
File "/home/jlejeune/dev/pci_repos/stash/nova/nova/virt/libvirt/utils.py",
line 249, in create_image
processutils.execute(*cmd)
File
"/home/jlejeune/dev/pci_repos/stash/nova/.tox/py310/lib/python3.10/site-packages/oslo_concurrency/processutils.py",
line 438, in execute
raise ProcessExecutionError(exit_code=_returncode,
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while
running command.
Command: env LC_ALL=C LANG=C qemu-img create -f qcow2
./tmp_bnmjhqw/e560a316990743598911504ed0cbbf41.delta 123456
Exit code: 127
Stdout: ''
Stderr: 'env: ‘qemu-img’: No such file or directory\n'
** Affects: nova
Importance: Undecided
Status: In Progress
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2075178
Title:
test_snapshot_runn
