[Yahoo-eng-team] [Bug 2077879] Re: [OVN] Router gateway does not have a chassis defined
** Changed in: neutron Status: New => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2077879 Title: [OVN] Router gateway does not have a chassis defined Status in neutron: Fix Released Bug description: # Problem Description I have: - 2 VMs connected to inner-net - inner-router(with default gateway in outer-net and a port in inner-net) - outer router(with default gateway in public network and a port in outer-net) NOTE: I don't have any static routes defined for these routers. Graphical visualization can be found here: https://i.ibb.co/gzjd604/Screenshot-from-2024-08-20-13-26-55.png This scenario works perfectly fine for OVS ML2 driver(VMs have Internet connectivity), but not for OVN. I noticed that gateway port for inner-router is DOWN(you can see this on the above screenshot) which looks quite suspicious. I applied https://review.opendev.org/c/openstack/neutron/+/907504 but it didn't solve the problem. # Further Investigation I noticed that inner-router's gateway interface does not have a chassis assigned: ``` router 7a5baad4-657d-42fc-bf35-1b8e4115050e (neutron-028eb3f7-af0b-4080-87d6-e84b24675b6d) (aka inner-router) port lrp-a221d264-8fa3-4430-99f7-f453887b96aa mac: "fa:16:3e:af:b0:ae" networks: ["10.10.0.60/24"] port lrp-9ac7815d-75dc-4198-aa94-bfe5ad5431e2 mac: "fa:16:3e:05:30:57" networks: ["10.0.0.1/24"] nat 8fca2dfd-2284-4e18-98be-137606f0f0b9 external ip: "10.10.0.60" logical ip: "10.0.0.0/24" type: "snat" ``` I fixed it with `ovn-nbctl lrp-set-gateway-chassis lrp-a221d264-8fa3-4430-99f7-f453887b96aa efcb326f-f18c-4b65-9da9-260dd0e2e603`. Now everything looks good. Internet connectivity is working and neutron gateway port(10.10.0.60) is ACTIVE instead of DOWN. # How to reproduce the issue (assuming that you have a test environment with 'public' network already defined) ``` openstack network create outer-net --external --disable-port-security openstack subnet create --network outer-net --subnet-range 10.10.0.0/24 outer-subnet openstack router create outer-router --external-gateway public openstack router add subnet outer-router outer-subnet openstack network create inner-net --disable-port-security openstack subnet create --network inner-net --subnet-range 10.0.0.0/24 inner-subnet openstack router create --external-gateway outer-net inner-router openstack router add subnet inner-router inner-subnet openstack server create \ --network inner-net \ --image 'cirros' \ --flavor 'tempest1' \ vm-inner-1 ``` Then, log in to vm-inner-1 and try to ping 8.8.8.8. For OVS it works, for OVN it doesn't. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2077879/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2082344] Re: [unmaintained] ``NetworkWritableMtuTest`` test don't work in ML2/OVN
Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/930594 Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/68e1139b8c721d760d2d1dfc1d67a359478400ea Submitter: "Zuul (22348)" Branch:master commit 68e1139b8c721d760d2d1dfc1d67a359478400ea Author: Rodolfo Alonso Hernandez Date: Thu Sep 26 15:01:32 2024 + Exclude ``NetworkWritableMtuTest`` test class in ML2/OVN E/W packet fragmentation is not supported in ML2/OVN [1]. [1]https://docs.openstack.org/neutron/latest/ovn/gaps.html Closes-Bug: #2082344 Change-Id: Ieeed73b2a5fc8319b3c199cdd2888e0090139077 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2082344 Title: [unmaintained] ``NetworkWritableMtuTest`` test don't work in ML2/OVN Status in neutron: Fix Released Bug description: This bug is related to https://review.opendev.org/c/openstack/neutron- tempest-plugin/+/929633. The test case in ``NetworkWritableMtuTest`` cannot be executed in ML2/OVN. E/W packet fragmentation is not supported [1]. This test class is failing in several unmaintained CI: * Yoga: https://zuul.opendev.org/t/openstack/build/e9534f8759c24bb5bfe8ac452d5cb931 * Zed: https://zuul.opendev.org/t/openstack/build/739a64647ad146138f37652c1af8d47e [1]https://docs.openstack.org/neutron/latest/ovn/gaps.html To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2082344/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2082066] Re: Logging not supported in unmaintained branches
Reviewed: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/930546 Committed: https://opendev.org/openstack/neutron-tempest-plugin/commit/a107a258a6f389c6e1b240aeb85027701660f594 Submitter: "Zuul (22348)" Branch:master commit a107a258a6f389c6e1b240aeb85027701660f594 Author: Rodolfo Alonso Hernandez Date: Thu Sep 26 09:20:45 2024 + Disable the execution of ``LoggingTestJSON`` in older branches The OVN logging feature needed for Neutron was implemented in [1]. This patch is provided in v20.12.0. The unmatained branches running with ML2/OVN in Focal (Ubuntu 20.04), use the provided OVN package, v20.03, that doesn't have this patch. [1]https://github.com/ovn-org/ovn/commit/880dca99eaf73db7e783999c29386d03c82093bf Closes-Bug: #2082066 Change-Id: Id3ecf975516d459358bb5fcd01085ec6a3bdbd26 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2082066 Title: Logging not supported in unmaintained branches Status in neutron: Fix Released Bug description: Network logging is not supported in unmaintained branches, when running with ML2/OVN. The neutron-tempest-plugin class ``LoggingTestJSON`` tests cannot be executed in unmaintained branches: * yoga: https://zuul.opendev.org/t/openstack/build/18e52813deff45d7a4e0d705e6b22edc * xena: https://zuul.opendev.org/t/openstack/build/917f7eae275a46d4a8e371fecf7498e5 These jobs are running in Focal. The OVN version is 20.03. The support was provided in [1] (v20.12.0). [1]https://github.com/ovn- org/ovn/commit/880dca99eaf73db7e783999c29386d03c82093bf To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2082066/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2083004] [NEW] Group/Role Assignment Changes in Keycloak Not Reflected Immediately in Keystone Authentication
Public bug reported: ## Problem Currently, changes to group assignments in Keycloak are not immediately reflected during the user login process in Keystone. For example: - When a user is assigned to Group A in Keycloak, they are unable to access the projects where Group A has role assignments until after a 10-minute delay. - Similarly, if a user is removed from Group A, they can still access projects where Group A has role assignments for up to 10 minutes after the removal. ## Root Cause After troubleshooting, it was discovered that this issue stems from Keystone's core assignment implementation. Specifically, in the following commit: https://opendev.org/openstack/keystone/commit/63124f703a81074793360c1b91711b6ee5a76196 The `list_projects_for_user` function in Keystone leverages `oslo.cache`. This cache is invalidated when any role assignment is added or removed within Keystone itself. In keycloak federation, since group assignment changes occur within Keycloak and not Keystone, these operations fall outside Keystone's direct control, and thus the cache is not invalidated when group assignments are changed in Keycloak. As a result, users must wait for the cache to expire (configured as `cache.expiration_time` https://docs.openstack.org/keystone/latest/configuration/config- options.html#cache.expiration_time in Keystone’s configuration, typically 10 minutes) before changes in group assignments are reflected in OpenStack. ## Scope This issue exists in not only Keycloak backend but also other authentication backends such as LDAP. ## Proposed solution This delay in reflecting group assignment changes is due to Keystone caching the list of projects and domains for users, and the fact that group assignment operations in Keycloak are not within Keystone's scope. To address this issue, either the caching mechanism needs adjustment, or a mechanism for invalidating the cache when group assignments change in Keycloak needs to be implemented. ** Affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/2083004 Title: Group/Role Assignment Changes in Keycloak Not Reflected Immediately in Keystone Authentication Status in OpenStack Identity (keystone): New Bug description: ## Problem Currently, changes to group assignments in Keycloak are not immediately reflected during the user login process in Keystone. For example: - When a user is assigned to Group A in Keycloak, they are unable to access the projects where Group A has role assignments until after a 10-minute delay. - Similarly, if a user is removed from Group A, they can still access projects where Group A has role assignments for up to 10 minutes after the removal. ## Root Cause After troubleshooting, it was discovered that this issue stems from Keystone's core assignment implementation. Specifically, in the following commit: https://opendev.org/openstack/keystone/commit/63124f703a81074793360c1b91711b6ee5a76196 The `list_projects_for_user` function in Keystone leverages `oslo.cache`. This cache is invalidated when any role assignment is added or removed within Keystone itself. In keycloak federation, since group assignment changes occur within Keycloak and not Keystone, these operations fall outside Keystone's direct control, and thus the cache is not invalidated when group assignments are changed in Keycloak. As a result, users must wait for the cache to expire (configured as `cache.expiration_time` https://docs.openstack.org/keystone/latest/configuration/config- options.html#cache.expiration_time in Keystone’s configuration, typically 10 minutes) before changes in group assignments are reflected in OpenStack. ## Scope This issue exists in not only Keycloak backend but also other authentication backends such as LDAP. ## Proposed solution This delay in reflecting group assignment changes is due to Keystone caching the list of projects and domains for users, and the fact that group assignment operations in Keycloak are not within Keystone's scope. To address this issue, either the caching mechanism needs adjustment, or a mechanism for invalidating the cache when group assignments change in Keycloak needs to be implemented. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/2083004/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2083033] [NEW] The file permissions injected into the instance through Config Drive do not meet security regulations.
Public bug reported: Description === I created an instance of the configuration driver in iso9660 format, and then logged in to the instance. I found that the permissions of the files and directories injected through the configuration driver were all 555. For example, ec2/2009-04-04/meta-data.json ec2/2009-04-04/user-data ec2/latest/meta-data.json ec2/latest/user-data openstack/2012-08-10/meta_data.json openstack/2012-08-10/user_data openstack/content openstack/content/ openstack/content/0001 openstack/latest/meta_data.json openstack/latest/user_data If the data injected by the user contains sensitive information such as passwords and secret keys, and the file and directory permissions are not set appropriately, sensitive information may be leaked and security attacks may occur. Steps to reproduce == step1: I set config_drive_format=iso9660 step2: I created a flavor and image step3: nova boot --image my_image --flavor my --nic net-name=config_net my_vm --config-drive true step4: I logged in to the instance to view file and directory permissions Expected result === I would like to be able to set different file permissions based on different file types to meet security regulations. For example, the configuration directory is 750, the configuration file is 640, and the program files and directories are 550, etc. Actual result = The permissions of the files and directories injected through the configuration driver were all 555. Environment === 1. version: nova 20.1.1 2. hypervisor:Libvirt + KVM 2. storage type: LVM 3. networking: Neutron with OpenVSwitch ** Affects: nova Importance: Undecided Status: New ** Summary changed: - The file permissions injected into the virtual machine through Config Drive do not meet security requirements. + The file permissions injected into the virtual machine through Config Drive do not meet security regulations. ** Summary changed: - The file permissions injected into the virtual machine through Config Drive do not meet security regulations. + The file permissions injected into the instance through Config Drive do not meet security regulations. -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2083033 Title: The file permissions injected into the instance through Config Drive do not meet security regulations. Status in OpenStack Compute (nova): New Bug description: Description === I created an instance of the configuration driver in iso9660 format, and then logged in to the instance. I found that the permissions of the files and directories injected through the configuration driver were all 555. For example, ec2/2009-04-04/meta-data.json ec2/2009-04-04/user-data ec2/latest/meta-data.json ec2/latest/user-data openstack/2012-08-10/meta_data.json openstack/2012-08-10/user_data openstack/content openstack/content/ openstack/content/0001 openstack/latest/meta_data.json openstack/latest/user_data If the data injected by the user contains sensitive information such as passwords and secret keys, and the file and directory permissions are not set appropriately, sensitive information may be leaked and security attacks may occur. Steps to reproduce == step1: I set config_drive_format=iso9660 step2: I created a flavor and image step3: nova boot --image my_image --flavor my --nic net-name=config_net my_vm --config-drive true step4: I logged in to the instance to view file and directory permissions Expected result === I would like to be able to set different file permissions based on different file types to meet security regulations. For example, the configuration directory is 750, the configuration file is 640, and the program files and directories are 550, etc. Actual result = The permissions of the files and directories injected through the configuration driver were all 555. Environment === 1. version: nova 20.1.1 2. hypervisor:Libvirt + KVM 2. storage type: LVM 3. networking: Neutron with OpenVSwitch To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2083033/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2083092] [NEW] sqlalchemy.exc.InvalidRequestError im image update
Public bug reported:
A strange failure was caught in glance-multistore-cinder-import job.
According to the tempest test case failure, image deleted failed because
of 500 error.
https://zuul.opendev.org/t/openstack/build/bcb2045e696c473e9ab8a1f376bcc714
And the following traceback is logged in g-api.log. I'm reporting this
here in case this could be related to recent migration to SQLAlchemy 2.0
.
```
Sep 27 15:44:58.566377 np0038636614 devstack@g-api.service[88666]: DEBUG
glance.api.middleware.version_negotiation [None
req-cb60319b-481e-4127-96ff-8e6dfdfafeca tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] Determining version of
request: POST /v2/images Accept: application/json {{(pid=88666) process_request
/opt/stack/glance/glance/api/middleware/version_negotiation.py:44}}
Sep 27 15:44:58.566594 np0038636614 devstack@g-api.service[88666]: DEBUG
glance.api.middleware.version_negotiation [None
req-cb60319b-481e-4127-96ff-8e6dfdfafeca tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] Using url versioning
{{(pid=88666) process_request
/opt/stack/glance/glance/api/middleware/version_negotiation.py:57}}
Sep 27 15:44:58.566806 np0038636614 devstack@g-api.service[88666]: DEBUG
glance.api.middleware.version_negotiation [None
req-cb60319b-481e-4127-96ff-8e6dfdfafeca tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] Matched version: v2
{{(pid=88666) process_request
/opt/stack/glance/glance/api/middleware/version_negotiation.py:69}}
Sep 27 15:44:58.567040 np0038636614 devstack@g-api.service[88666]: DEBUG
glance.api.middleware.version_negotiation [None
req-cb60319b-481e-4127-96ff-8e6dfdfafeca tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] new path /v2/images
{{(pid=88666) process_request
/opt/stack/glance/glance/api/middleware/version_negotiation.py:70}}
...
'cinder://lvmdriver-1/09bc6160-d566-45c9-9cde-4e35497dd7a9', 'metadata':
{'store': 'lvmdriver-1'}, 'status': 'active'}, {'id': 38, 'url':
'cinder://lvmdriver-2/3cd47cd3-ad7f-437e-bfa4-10101e019d15', 'metadata':
{'store': 'lvmdriver-2'}, 'status': 'active'}] {{(pid=88666)
sort_image_locations /opt/stack/glance/glance/common/utils.py:735}}
Sep 27 15:44:58.033280 np0038636614 devstack@g-api.service[88666]: DEBUG
glance_store._drivers.cinder.store [None
req-cb60319b-481e-4127-96ff-8e6dfdfafeca tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] Cinderclient connection
created for user glance using URL: https://158.69.74.140/identity/v3.
{{(pid=88666) get_cinderclient
/opt/stack/data/venv/lib/python3.10/site-packages/glance_store/_drivers/cinder/store.py:648}}
Sep 27 15:44:58.041288 np0038636614 devstack@g-api.service[88665]: DEBUG
glance.common.utils [-] Sorted locations: [{'id': 37, 'url':
'cinder://lvmdriver-1/09bc6160-d566-45c9-9cde-4e35497dd7a9', 'metadata':
{'store': 'lvmdriver-1'}, 'status': 'active'}, {'id': 38, 'url':
'cinder://lvmdriver-2/3cd47cd3-ad7f-437e-bfa4-10101e019d15', 'metadata':
{'store': 'lvmdriver-2'}, 'status': 'active'}] {{(pid=88665)
sort_image_locations /opt/stack/glance/glance/common/utils.py:735}}
Sep 27 15:44:58.062043 np0038636614 devstack@g-api.service[88665]: DEBUG
glance.common.utils [-] Sorted locations: [{'id': 37, 'url':
'cinder://lvmdriver-1/09bc6160-d566-45c9-9cde-4e35497dd7a9', 'metadata':
{'store': 'lvmdriver-1'}, 'status': 'active'}, {'id': 38, 'url':
'cinder://lvmdriver-2/3cd47cd3-ad7f-437e-bfa4-10101e019d15', 'metadata':
{'store': 'lvmdriver-2'}, 'status': 'active'}] {{(pid=88665)
sort_image_locations /opt/stack/glance/glance/common/utils.py:735}}
Sep 27 15:44:58.139389 np0038636614 devstack@g-api.service[88666]: DEBUG
glance_store._drivers.cinder.store [None
req-cb60319b-481e-4127-96ff-8e6dfdfafeca tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] Cinderclient connection
created for user glance using URL: https://158.69.74.140/identity/v3.
{{(pid=88666) get_cinderclient
/opt/stack/data/venv/lib/python3.10/site-packages/glance_store/_drivers/cinder/store.py:648}}
...
Sep 27 15:44:58.507784 np0038636614 devstack@g-api.service[88666]: DEBUG
glance.db.sqlalchemy.api [None req-cb60319b-481e-4127-96ff-8e6dfdfafeca
tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] StaleDataError while
deleting property 'os_glance_import_task' from image
'f16c174b-46fd-4241-aeb6-ae85583564d1' likely means we raced during delete:
UPDATE statement on table 'image_properties' expected to update 1 row(s); 0
were matched. {{(pid=88666) _image_property_delete
/opt/stack/glance/glance/db/sqlalchemy/api.py:1328}}
Sep 27 15:44:58.532097 np0038636614 devstack@g-api.service[88666]: ERROR
glance.common.wsgi [None req-cb60319b-481e-4127-96ff-8e6dfdfafeca
tempest-ImagesFormatTest-522606120
tempest-ImagesFormatTest-522606120-project-member] Caught error:
