[Yahoo-eng-team] [Bug 2083287] Re: test_add_router_interfaces_on_overlapping_subnets_returns_400 failed when retrying router interface removal after API timeout; (pymysql.err.OperationalError) (1205,

2024-10-28 Thread yatin
Some new occurances after fixes merged, so re opening to check further https://8f5091142d0d93e54bb3-66f76d6fb4c84b410723fddf17d0dbe7.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-mariadb-full/933107e/testr_results.html https://32e6a1c8c02fba10617a-55d9f2ceb0

[Yahoo-eng-team] [Bug 2087824] [NEW] random router related test failures since wsgi switch

2024-11-10 Thread yatin
Public bug reported: With couple of recent fixes since wsgi switch we still seeing some random failures, this to track fixes of these unfixed issues. Example failures:- 1) test_router_interface_port_update_with_fixed_ip https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zu

[Yahoo-eng-team] [Bug 2081174] [NEW] Handle EndpointNotFound in nova notifier

2024-09-19 Thread yatin
Public bug reported: When nova endpoint for endpoint_type(public/internal/admin) is not exist, following traceback is raised:- 2024-09-18 13:19:38.182 15 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'name': 'network-changed', 'server_uuid': '3c634df2-eb78-4f49-bb01-ae1c54

[Yahoo-eng-team] [Bug 2087824] Re: random router related test failures since wsgi switch

2024-11-25 Thread yatin
Still seeing it, latest failure:- https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ae5/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-mariadb-full/ae590b4/testr_results.html 2024-11-25 03:28:51,737 73074 WARNING [urllib3.connectionpoo

[Yahoo-eng-team] [Bug 2083482] Re: "neutron-fullstack-with-uwsgi-fips" failing in stable releases (Zed, 2023.1)

2024-11-18 Thread yatin
** Changed in: neutron Status: New => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2083482 Title: "neutron-fullstack-with-uwsgi-fips" failing in stable releases (Ze

[Yahoo-eng-team] [Bug 2088423] [NEW] [functional test] test_ovn_nb_sync* tests fails randomly with Internal Server Error

2024-11-17 Thread yatin
Public bug reported: Fails like:- ft1.3: neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.ovsdb.test_ovn_db_sync.TestOvnNbSyncOverSsl.test_ovn_nb_sync_offtesttools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutro

[Yahoo-eng-team] [Bug 1939137] [NEW] ovn + log service plugin reports AttributeError: 'NoneType' object has no attribute 'tenant_id'

2021-08-06 Thread yatin
Public bug reported: Originally noticed in a TripleO job[1], and after enabling log service plugin in devstack seeing the similar error in neutron service log. Following Traceback is seen:- ERROR neutron_lib.callbacks.manager [None req-131d215c-1d03-48ce-a16e-28175c0f58ba tempest-DefaultSnatToE

[Yahoo-eng-team] [Bug 1939137] Re: ovn + log service plugin reports AttributeError: 'NoneType' object has no attribute 'tenant_id'

2021-09-27 Thread yatin
Seems it's fixed with https://github.com/openstack/neutron/commit/7f063223553a18345891bf42e88989edb67038e7, no longer seeing issue in TripleO job:- https://d0e3fd572414115f797b-39524de8c5a1fb89d206195b6f692473.ssl.cf5.rackcdn.com/808056/1/check/tripleo- ci- centos-8-standalone/6cb888b/logs/underclo

[Yahoo-eng-team] [Bug 1952393] [NEW] [master] neutron-tempest-plugin-scenario-ovn broken with "ovn-northd did not start"

2021-11-25 Thread yatin
Public bug reported: Job failing consistently with below error since https://review.opendev.org/c/openstack/devstack/+/806858:- 2021-11-26 05:58:40.377912 | controller | + functions-common:test_with_retry:2339: timeout 60 sh -c 'while ! test -e /usr/local/var/run/openvswitch/ovn-northd.pi

[Yahoo-eng-team] [Bug 1952393] Re: [OVN] neutron-tempest-plugin-scenario-ovn broken with "ovn-northd did not start"

2021-12-03 Thread yatin
** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1952393 Title: [OVN] neutron-tempest-plugin-scenario-ovn broken with "ovn

[Yahoo-eng-team] [Bug 1957021] Re: openstack xena documenation has wrong neutron service name to start

2022-01-17 Thread yatin
Seems you following wrong documentation for CentOS Stream. You need to refer https://docs.openstack.org/neutron/xena/install/install-rdo.html instead of https://docs.openstack.org/neutron/xena/install/install- obs.html, The later one is for OpenSUSE. Marking the bug as invalid. https://docs.opensta

[Yahoo-eng-team] [Bug 1958961] [NEW] [ovn-octavia-provider] lb create failing with with ValueError: invalid literal for int() with base 10: '24 2001:db8::131/64'

2022-01-25 Thread yatin
Public bug reported: When deployed with octavia-ovn-provider with below local.conf, loadbalancer create(openstack loadbalancer create --vip-network-id public --provider ovn) goes into ERROR state. >From o-api logs:- ERROR ovn_octavia_provider.helper Traceback (most recent call last): ERROR ovn_oc

[Yahoo-eng-team] [Bug 1959176] [NEW] neutron fullstack/functional jobs TIMED_OUT randomly

2022-01-26 Thread yatin
Public bug reported: Neutron fullstack and functional jobs are timing out randomly across all branches, only few such failures though, 1 or 2 failures per week. Builds references:- - https://zuul.opendev.org/t/openstack/builds?job_name=neutron-functional-with-uwsgi&result=TIMED_OUT&skip=0 - htt

[Yahoo-eng-team] [Bug 1959564] [NEW] Random Tempest test failures(SSH failure) in openvswitch jobs

2022-01-31 Thread yatin
Public bug reported: Seen multiple similar occurences in Stable/wallaby patches, where tempest tests fails with ssh to VM Timeouts, some examples:- - https://cfaa2d1e4f6a936642aa-ae5561c9d080274a217713c4553af257.ssl.cf5.rackcdn.com/824022/2/check/neutron-tempest-plugin-scenario-openvswitch-wal

[Yahoo-eng-team] [Bug 1959573] [NEW] test_local_ip_connectivity test failing in stable/xena jobs

2022-01-31 Thread yatin
. It's happening as master version of openvswitch jobs is running in stable/xena branch and that feature is only available in master branch. This needs to be fixed by switch to xena-jobs in stable/xena branch. ** Affects: neutron Importance: Undecided Assignee: yatin (yatin

[Yahoo-eng-team] [Bug 1930402] Re: SSH timeouts happens very often in the ovn based CI jobs

2022-02-24 Thread yatin
Closing it as neutron-ovn-tempest-slow is moved to periodic in master branch and is no longer seeing ssh failures, if it happen again this can be reopened or new issue can be created. ** Changed in: neutron Status: Confirmed => Fix Released -- You received this bug notification because yo

[Yahoo-eng-team] [Bug 1962306] [NEW] [Prefix Delegation] router add segment fails with Subnet for router interface must have a gateway IP

2022-02-25 Thread yatin
without re introducing previous bug:- openstack subnet create --ip-version 6 --ipv6-ra-mode slaac --ipv6-address-mode slaac --use-default-subnet-pool --network ipv6-pd --gateway :: ipv6-pd-1 ** Affects: neutron Importance: Undecided Assignee: yatin (yatinkarel) Status: New

[Yahoo-eng-team] [Bug 1956344] Re: Functional test test_gateway_chassis_rebalance is failing intermittently

2022-02-25 Thread yatin
Closing it as the workaround is in place and recently we have not seen failures in this test and also the test code have a NOTE added for future cleanup. If we see again the failure bug can be re opened. ** Changed in: neutron Status: Confirmed => Fix Released -- You received this bug not

[Yahoo-eng-team] [Bug 1868110] Re: [OVN] neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.ovsdb.test_ovn_db_sync.TestOvnNbSyncOverTcp.test_ovn_nb_sync_log randomly fails

2022-02-25 Thread yatin
Closing at as not seeing this issue recently as timeout increased long back. ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bug

[Yahoo-eng-team] [Bug 1864833] Re: [OVN] Functional tests start with OVSDB binary 2.9 instead 2.12

2022-02-25 Thread yatin
It's no longer an issue now as i see ovs 2.16 in test and same is compiled in job, so closing the bug. OVS_BRANCH=v2.16.0 2022-02-25T11:12:50.733Z|3|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.16.0 https://91ac997443c049736e18-7ff9e89a9bc5ab4bd512192457a69ff2.ssl.cf1.rackcdn.com/828687/

[Yahoo-eng-team] [Bug 1960022] Re: neutron-tempest-with-uwsgi job failing tempest tests with: Floating ip failed to associate with server in time

2022-02-28 Thread yatin
Fixed in tempest, closing it. ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1960022 Title: neutron-tempest-with-uwsgi

[Yahoo-eng-team] [Bug 1960338] Re: [fullstack] Test "test_east_west_traffic" randomly failing

2022-02-28 Thread yatin
Job is much stable now[1], single failure since the patch[2] merge and that too different test, Closing for now, if we see the issue again, it can be reopened. [1] https://zuul.openstack.org/builds?job_name=neutron-fullstack&branch=stable%2Fwallaby&skip=0 [2] https://review.opendev.org/c/openstac

[Yahoo-eng-team] [Bug 1886807] Re: neutron-ovn-tempest-full-multinode-ovs-master job is failing 100% times

2022-03-03 Thread yatin
*** This bug is a duplicate of bug 1904117 *** https://bugs.launchpad.net/bugs/1904117 ** This bug is no longer a duplicate of bug 1885898 test connectivity through 2 routers fails in neutron-ovn-tempest-full-multinode-ovs-master job ** This bug has been marked a duplicate of bug 1904117

[Yahoo-eng-team] [Bug 1874447] Re: [OVN] Tempest test neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle fails randomly

2022-03-15 Thread yatin
** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1874447 Title: [OVN] Tempest test neutron_tempest_plugin.scenario.test

[Yahoo-eng-team] [Bug 1906500] [NEW] [ovn] Tempest tests failing while creating security group driver with KeyError: 'remote_address_group_id'

2020-12-02 Thread yatin
Public bug reported: This is failing post https://review.opendev.org/c/openstack/neutron/+/751110, detected in a packstack job, it fails with below Traceback:- 2020-12-01 13:05:53.257 85629 ERROR neutron.api.v2.resource Traceback (most recent call last): 2020-12-01 13:05:53.257 85629 ERROR neutr

[Yahoo-eng-team] [Bug 2017131] [NEW] neutron-loki job fails tests with ORM session: SQL execution without transaction in progress

2023-04-20 Thread yatin
Public bug reported: Job(neutron-ovn-tempest-with-uwsgi-loki) running neutron-loki service[1] fails tempest tests randomly with Tracebacks like below:- ORM session: SQL execution without transaction in progress, traceback: Apr 17 12:12:44.386611 np0033759257 neutron-server[57996]: File "/usr/

[Yahoo-eng-team] [Bug 2017478] [NEW] openstack-tox-py38 broken with tooz==4.0.0

2023-04-23 Thread yatin
ron&branch=master&skip=0 [2] https://review.opendev.org/c/openstack/requirements/+/880738 [3] https://review.opendev.org/c/openstack/tooz/+/879930 [4] https://review.opendev.org/c/openstack/governance/+/872232 ** Affects: neutron Importance: Critical Assignee: yatin (yatinkare

[Yahoo-eng-team] [Bug 2017992] [NEW] Jobs running on vexxhost provider failing with Mirror Issues

2023-04-28 Thread yatin
Public bug reported: Fails as below:- 2023-04-27 12:36:47.535483 | controller | ++ functions-common:apt_get_update:1155 : timeout 300 sh -c 'while ! sudo http_proxy= https_proxy= no_proxy= apt-get update; do sleep 30; done' 2023-04-27 12:36:50.877357 | controller | Err:1 https://mirror.c

[Yahoo-eng-team] [Bug 2018130] [NEW] [master][functional] test_cascading_del_in_txn fails with ovsdbapp=2.3.0

2023-04-29 Thread yatin
Public bug reported: With ovsdbapp==2.3.0 [1] releaase functional test neutron.tests.functional.agent.test_ovs_lib.OVSBridgeTestCase.test_cascading_del_in_txn fails consistently[2] as below:- ft1.7: neutron.tests.functional.agent.test_ovs_lib.OVSBridgeTestCase.test_cascading_del_in_txntesttools.

[Yahoo-eng-team] [Bug 2018967] [NEW] [fwaas] test_update_firewall_group fails randomly

2023-05-08 Thread yatin
Public bug reported: Seen twice till now recently:- - https://a78793e982809689fe25-25fa16d377ec97c08c4e6ce3af683bd9.ssl.cf5.rackcdn.com/881232/1/check/neutron-tempest-plugin-fwaas/b0730f9/testr_results.html - https://53a7c53d508ecea7485c-f8ccc2b7c32dd8ba5caab7dc1c36a741.ssl.cf5.rackcdn.com/88123

[Yahoo-eng-team] [Bug 2017992] Re: Jobs running on vexxhost provider failing with Mirror Issues

2023-05-15 Thread yatin
We got an update[1] from vexxhost Team and now the vexxhost node provider is re enabled[2]. Since it reenabled we are not seeing the issue. Noticed till now the jobs ran on 3 impacted nodes and is passing so can consider the issue resolved. Closing the bug. [1] We have performed some optimizati

[Yahoo-eng-team] [Bug 2019802] [NEW] [master] Not all fullstack tests running in CI

2023-05-16 Thread yatin
Public bug reported: >From last couple of days Only few tests(just 6 tests) are running in job neutron-fullstack-with-uwsgi job. Example:- https://d16311159baa9c9fc692-58e8a805a242f8a07eac2fd1c3f6b11b.ssl.cf1.rackcdn.com/880867/6/gate/neutron- fullstack-with-uwsgi/6b4c3ba/testr_results.html Buil

[Yahoo-eng-team] [Bug 2024674] [NEW] Unit tests fails with oslo_db.exception.DBNonExistentTable: (sqlite3.OperationalError) no such table: ml2_geneve_allocations when run with low concurrency or on lo

2023-06-22 Thread yatin
Public bug reported: The issue is noticed in RDO openstack-neutron package build[1], the package builds fails as unit tests fails randomly with below Traceback:- DEBUG: neutron.tests.unit.services.trunk.test_utils.UtilsTestCase.test_is_driver_compatible_multiple_drivers DEBUG: -

[Yahoo-eng-team] [Bug 1932043] Re: nova-ceph-multistore test_resize_server_revert fails with rbd.ReadOnlyImage: [errno 30] RBD read-only image

2023-06-28 Thread yatin
Seen it today in https://c83b20527acf2b0f8494-4a0455790e56cb733d68b35ced7c28e7.ssl.cf5.rackcdn.com/886250/2/check/nova- ceph-multistore/2231918/testr_results.html ** Changed in: nova Status: Expired => New -- You received this bug notification because you are a member of Yahoo! Engineerin

[Yahoo-eng-team] [Bug 2027817] [NEW] neutron slow jobs before xena broken with ERROR: unknown environment 'slow'

2023-07-14 Thread yatin
Public bug reported: Broken since https://review.opendev.org/c/openstack/tempest/+/887237 merged as neutron slow jobs inherit from tempest-slow-py3. These fails with ERROR: unknown environment 'slow' In releases before xena tempest is pinned and don't have the patch[1] which added slow toxenv. T

[Yahoo-eng-team] [Bug 1969592] Re: [OVN] Frequent DB leader changes causes 'VIF creation failed' on nova side

2023-07-18 Thread yatin
There were many fixes related to the reported issue in neutron and ovs since this bug report, some of these that i quickly catched are:- - https://patchwork.ozlabs.org/project/openvswitch/patch/20220819230810.2626573-1-i.maxim...@ovn.org/ - https://review.opendev.org/c/openstack/ovsdbapp/+/856200

[Yahoo-eng-team] [Bug 1996594] Re: OVN metadata randomly stops working

2023-07-18 Thread yatin
There were many fixes related to the reported issue in neutron and ovs since this bug report, some of these that i quickly catched are:- - https://patchwork.ozlabs.org/project/openvswitch/patch/20220819230810.2626573-1-i.maxim...@ovn.org/ - https://review.opendev.org/c/openstack/ovsdbapp/+/856200

[Yahoo-eng-team] [Bug 2028123] Re: Jobs setting up more than 1 image fails with ValueError: invalid literal for int() with base 10: ''

2023-07-19 Thread yatin
Fix proposed https://review.opendev.org/c/openstack/devstack/+/888906 ** Also affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/20

[Yahoo-eng-team] [Bug 2028285] [NEW] [unit test][xena+] test_port_deletion_prevention fails when runs in isolation

2023-07-20 Thread yatin
Public bug reported: Can be reproduced by Just running:- tox -epy3 -- test_port_deletion_prevention or run any of the below tests individually:- neutron.tests.unit.extensions.test_l3.L3NatDBSepTestCase.test_port_deletion_prevention_handles_missing_port neutron.tests.unit.extensions.test_extraroute

[Yahoo-eng-team] [Bug 2033752] [NEW] test_reboot_server_hard fails with AssertionError: time.struct_time() not greater than time.struct_time()

2023-09-01 Thread yatin
Public bug reported: Seen many occurrences recently, fails as below:- Traceback (most recent call last): File "/opt/stack/tempest/tempest/api/compute/servers/test_server_actions.py", line 259, in test_reboot_server_hard self._test_reboot_server('HARD') File "/opt/stack/tempest/tempest/ap

[Yahoo-eng-team] [Bug 2033752] Re: test_reboot_server_hard fails with AssertionError: time.struct_time() not greater than time.struct_time()

2023-09-03 Thread yatin
Closed for tempest and neutron, as regression is fixed in nova with https://review.opendev.org/c/openstack/nova/+/893502, jobs are back to green. ** Changed in: neutron Status: Confirmed => Invalid ** Changed in: tempest Status: New => Invalid -- You received this bug notification

[Yahoo-eng-team] [Bug 2034016] [NEW] openstack-tox-py310-with-sqlalchemy-master broken with latest alembic commit

2023-09-04 Thread yatin
Public bug reported: The job runs with latest alembic/sqlalchemy commits and is broken with recent alembic commit[1]. Test neutron.tests.unit.db.test_migration.TestCli.test_autogen_process_directives fails as below:- ft1.27: neutron.tests.unit.db.test_migration.TestCli.test_autogen_process_dir

[Yahoo-eng-team] [Bug 2034096] [NEW] OVN source(main branch) deploy jobs fails with make: *** No rule to make target '/opt/stack/ovs/python/ovs_build_helpers/soutil.py', needed by 'manpages.mk'. Stop

2023-09-04 Thread yatin
empest-ipv6-only-ovs-master&job_name=neutron-ovn-tempest-ovs-master- centos-9-stream&project=openstack%2Fneutron&skip=0 Broken with https://github.com/ovn- org/ovn/commit/558da0cd21ad0172405f7d93c5d0e7533edbf653 Need to update OVS_BRANCH in jobs to fix it. ** Affects: neutron Imp

[Yahoo-eng-team] [Bug 2033683] Re: openvswitch.agent.ovs_neutron_agent fails to Cmd: ['iptables-restore', '-n']

2023-09-06 Thread yatin
Hi Alex, << Can someone take a look why the above patch https://review.opendev.org/c/openstack/kolla/+/761182 mentioned here has been excluded from the neutron image? It would have been just missed, since train release Tripleo builds container images natively and not use kolla, You can propose a

[Yahoo-eng-team] [Bug 2012731] Re: [CI] "neutron-ovs-grenade-multinode-skip-level" and "neutron-ovn-grenade-multinode-skip-level" failing always

2023-09-12 Thread yatin
Required patch merged long back, jobs are green, Closing it:- * https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovs-grenade-multinode-skip-level&skip=0 * https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovn-grenade-multinode-skip-level&skip=0 ** Changed in: neutron

[Yahoo-eng-team] [Bug 1949606] Re: QEMU >= 5.0.0 with -accel tcg uses a tb-size of 1GB causing OOM issues in CI

2023-09-12 Thread yatin
Fixed with https://review.opendev.org/c/openstack/nova/+/868419 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bu

[Yahoo-eng-team] [Bug 2035578] [NEW] [stable branches] devstack-tobiko-neutron job Fails with InvocationError('could not find executable python', None)

2023-09-13 Thread yatin
Public bug reported: It started failing[1] since the job switched to ubuntu-jammy[2]. Fails as below:- 2023-09-13 16:46:18.124882 | TASK [tobiko-tox : run sanity test cases before creating resources] 2023-09-13 16:46:19.463567 | controller | neutron_sanity create: /home/zuul/src/opendev.org/x/t

[Yahoo-eng-team] [Bug 2038541] [NEW] LinuxBridgeARPSpoofTestCase functional tests fails with latest jammy kernel 5.15.0-86.96

2023-10-05 Thread yatin
Public bug reported: Tests fails while running ebtables(['-D', chain] + rule.split()) with:- 2023-10-05 12:09:19.307 41358 ERROR neutron.agent.linux.utils [None req-defd197a-c4e2-4761-a4cc-cc960a3ff71a - - - - - -] Exit code: 4; Cmd: ['ip', 'netns', 'exec', 'test-b58b5cf9-5018-4801-aacb-8b00fae3

[Yahoo-eng-team] [Bug 2039012] [NEW] [fwaas] Scenario job fails randomly

2023-10-11 Thread yatin
Public bug reported: Seen couple of occurrences across different releases:- Failures:- - https://zuul.opendev.org/t/openstack/build/30c625cd86aa40e6b6252689a7e88910 neutron-tempest-plugin-fwaas-2023-1 Traceback (most recent call last): File "/opt/stack/tempest/.tox/tempest/lib/python3.10/site

[Yahoo-eng-team] [Bug 2039066] [NEW] openstacksdk-functional-devstack-networking periodic job broken in xena/yoga/zed

2023-10-11 Thread yatin
%2Fyoga&branch=stable%2Fzed ** Affects: neutron Importance: Medium Assignee: yatin (yatinkarel) Status: Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2039066 Title:

[Yahoo-eng-team] [Bug 2039417] [NEW] [master][stable/2023.2][functional] test_maintenance.TestMaintenance tests fails randomly

2023-10-16 Thread yatin
Public bug reported: The functional test fails randomly as:- ft1.3: neutron.tests.functional.plugins.ml2.drivers.ovn.mech_driver.ovsdb.test_maintenance.TestMaintenance.test_porttesttools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/opens

[Yahoo-eng-team] [Bug 2039940] [NEW] test_resize_volume_backed_server_confirm which fails randomly with Kernel panic

2023-10-20 Thread yatin
Public bug reported: tempest.api.compute.servers.test_server_actions.ServerActionsTestOtherA.test_resize_volume_backed_server_confirm fails randomly with:- Traceback (most recent call last): File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 136, in _get_ssh_connection ssh.connect(s

[Yahoo-eng-team] [Bug 2040517] [NEW] host not removed from table ml2_vxlan_endpoints with the agent delete

2023-10-25 Thread yatin
Public bug reported: After deleting an agent, there is stale entry for the host in table 'ml2_vxlan_endpoints'. An use case is during node scale down, a agent is deleted, but the host entry is not removed from ml2_vxlan_endpoints; I have not checked other topologies but same should apply to other

[Yahoo-eng-team] [Bug 2042939] [NEW] sqlalchemy-master jobs broken with ValueError: invalid literal for int() with base 10: '0b1'

2023-11-07 Thread yatin
Public bug reported: Broken since https://github.com/sqlalchemy/sqlalchemy/commit/e93a5e89. Fails like:- 2023-11-07 10:51:53.328284 | ubuntu-jammy | Failed to import test module: neutron.tests.unit.agent.common.test_ovs_lib 2023-11-07 10:51:53.328310 | ubuntu-jammy | Traceback (most recent call

[Yahoo-eng-team] [Bug 2042941] [NEW] neutron-{ovn, ovs}-tempest-with-sqlalchemy-master jobs not installing sqlalchemy/alembic from source

2023-11-07 Thread yatin
Public bug reported: neutron-ovn-tempest-with-sqlalchemy-master and neutron-ovs-tempest-with-sqlalchemy-master jobs expected to install sqlalchemy and alembic from main branch as defined in required-projects, but these installs released versions instead:- required-projects: - name: gi

[Yahoo-eng-team] [Bug 2042947] [NEW] [stable branches] nftables job inherits from openvswitch-iptables_hybrid master job

2023-11-07 Thread yatin
Public bug reported: These jobs runs in periodic pipeline and are broken[1], these jobs inherit from ovs master jobs instead of stable variant. This needs to be fixed. [1] https://zuul.openstack.org/builds?job_name=neutron-linuxbridge-tempest-plugin-scenario-nftables ** Affects: neutron I

[Yahoo-eng-team] [Bug 2043959] [NEW] Unit test jobs with neutron-lib master(3.9.0) failing

2023-11-19 Thread yatin
Public bug reported: Since https://review.opendev.org/c/openstack/neutron-lib/+/895940 the job fails as:- ft1.2: neutron.tests.unit.plugins.ml2.drivers.agent.test_capabilities.CapabilitiesTest.test_registertesttools.testresult.real._StringException: Traceback (most recent call last): File "/h

[Yahoo-eng-team] [Bug 2045270] [NEW] [xena/yoga/zed] devstack-tobiko-neutron job broken

2023-11-30 Thread yatin
Public bug reported: These jobs are broken as these jobs running on focal and running tests which shouldn't on focal as per https://review.opendev.org/c/openstack/neutron/+/871982 (included in antelope+) Example failure https://d5a9a78dc7c742990ec8-242e23f01f4a3c50d38acf4a24e0c600.ssl.cf1.rackc

[Yahoo-eng-team] [Bug 2045383] [NEW] [xena] nftables periodic jobs fails with RETRY_LIMIT

2023-11-30 Thread yatin
Public bug reported: Since 18th Nov nftables jobs failing with RETRY_LIMIT and no logs are available. These jobs quickly fails and last task seen running in console is "Preparing job workspace". This looks a regression with zuul change https://review.opendev.org/c/zuul/zuul/+/900489. @fungi chec

[Yahoo-eng-team] [Bug 2045383] Re: [xena] nftables periodic jobs fails with RETRY_LIMIT

2023-12-03 Thread yatin
Thanks folks jobs are back to green now:- https://zuul.openstack.org/builds?job_name=neutron-linuxbridge-tempest- plugin-scenario-nftables&job_name=neutron-ovs-tempest-plugin-scenario- iptables_hybrid- nftables&project=openstack%2Fneutron&branch=stable%2Fxena&skip=0 ** Changed in: neutron S

[Yahoo-eng-team] [Bug 2045549] [NEW] OVS jobs randomly fails as Guest VMs not(or delayed) configured with DHCP

2023-12-04 Thread yatin
Public bug reported: Seen couple of hits recently, Tests fails as:- Traceback (most recent call last): File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 136, in _get_ssh_connection ssh.connect(self.host, port=self.port, username=self.username, File "/opt/stack/tempest/.tox/tempes

[Yahoo-eng-team] [Bug 2045725] [NEW] scenario jobs randomly fails downloading ubuntu image

2023-12-05 Thread yatin
Public bug reported: Jobs fails as:- 2023-11-30 10:17:01.182 | + /opt/stack/neutron-tempest-plugin/devstack/functions.sh:overridden_upload_image:36 : wget --progress=dot:giga -c https://cloud-images.ubuntu.com/minimal/releases/focal/release/ubuntu-20.04-minimal-cloudimg-amd64.img -O /opt/sta

[Yahoo-eng-team] [Bug 2045785] [NEW] test_unshelve_to_specific_host fails randomly at _shelve_offload_then_unshelve_to_host otherhost

2023-12-06 Thread yatin
Public bug reported: test_unshelve_to_specific_host fails randomly at _shelve_offload_then_unshelve_to_host otherhost like:- Traceback (most recent call last): File "/opt/stack/tempest/tempest/api/compute/admin/test_servers_on_multinodes.py", line 178, in test_unshelve_to_specific_host se

[Yahoo-eng-team] [Bug 1961740] Re: Network interface not found in namespace

2023-12-12 Thread yatin
Still seeing similar issue as below, reopening the bug - https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ed8/901717/1/gate/neutron- functional-with-uwsgi/ed8e900/testr_results.html Fails as:- neutron.privileged.agent.linux.ip_lib.NetworkInterfaceNotFou

[Yahoo-eng-team] [Bug 2045725] Re: scenario jobs randomly fails downloading ubuntu image

2023-12-12 Thread yatin
It's not an issue in neutron but was just a tracking bug for the issue in our jobs. https://review.opendev.org/q/I7163aea4d121cb27620e4f2a083a543abfc286bf handles the random issue. So closing this for neutron now. ** Changed in: neutron Status: New => Invalid -- You received this bug no

[Yahoo-eng-team] [Bug 2046196] Re: Unicast ICMPv6 Router Advertisement packets to VM's dropped by OVS firewall driver

2023-12-17 Thread yatin
*** This bug is a duplicate of bug 1958643 *** https://bugs.launchpad.net/bugs/1958643 Thanks @Stanislav for the confirmation, will close it as Duplicate. ** This bug has been marked a duplicate of bug 1958643 Unicast RA messages for a VM are filtered out by ovs rules -- You received thi

[Yahoo-eng-team] [Bug 2049488] [NEW] [ovn][source-main] job fails with error: too many arguments to function flow_compose

2024-01-16 Thread yatin
Public bug reported: Fails as:- 2024-01-16 03:02:53.651183 | controller | gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./ovn -I ./include -I ./lib -I ./lib -I /opt/stack/ovs/include -I /opt/stack/ovs/include -I /opt/stack/ovs/lib -I /opt/stack/ovs/lib -I /opt/stack/ovs -I /opt/stack/

[Yahoo-eng-team] [Bug 2051831] [NEW] [ovn] neutron-ovn-tempest-slow job fail tests relying on FIP

2024-01-31 Thread yatin
Public bug reported: Example failure:- https://2f4a32f753edcd6fd518-38c49964a79149719549049b602122d6.ssl.cf5.rackcdn.com/906628/1/experimental/neutron- ovn-tempest-slow/1b35fb8/testr_results.html Fails as:- Traceback (most recent call last): File "/opt/stack/tempest/tempest/scenario/test_securi

[Yahoo-eng-team] [Bug 2051845] [NEW] [ovn][source-main] job fails with error: too many arguments to function mcast_snooping_add_group4

2024-01-31 Thread yatin
/neutron/master/neutron- ovn-tempest-ipv6-only-ovs-master/3c5404e/job-output.txt Failing since https://github.com/ovn-org/ovn/commit/dc34b4d9f7f3efb4e7547f9850f6086a7e1a2338 need to update OVS_BRANCH ** Affects: neutron Importance: High Assignee: yatin (yatinkarel) Status: Tri

[Yahoo-eng-team] [Bug 2052508] [NEW] [doc][troubleshooting] how to enable ovs vswitchd debug logs

2024-02-06 Thread yatin
Public bug reported: It was raised while evaluating ovs-vswitchd debug logs in CI[1] that we should also document it in neutron docs on steps enabling/disabling debug logs for ovs-vswitchd. [1] https://review.opendev.org/c/openstack/neutron/+/907037 ** Affects: neutron Importance: Undecide

[Yahoo-eng-team] [Bug 2052509] [NEW] [CI][ovs/ovn sqlalchemy master] jobs broken with Exception: Not enough arguments given

2024-02-06 Thread yatin
Public bug reported: Fails like:- 2024-02-06 02:28:34.618579 | controller | ++ inc/python:_setup_package_with_constraints_edit:400 : cd /opt/stack/sqlalchemy 2024-02-06 02:28:34.621804 | controller | ++ inc/python:_setup_package_with_constraints_edit:400 : pwd 2024-02-06 02:28:34.624859 | co

[Yahoo-eng-team] [Bug 2056276] [NEW] unmaintained/yoga periodic jobs broken

2024-03-05 Thread yatin
Public bug reported: Couple of jobs failing in unmaintained/yoga periodic pipeline. https://zuul.openstack.org/buildsets?project=openstack%2Fneutron&branch=unmaintained%2Fyoga&branch=stable%2Fyoga&pipeline=periodic&skip=0 Before switching to unmaintained/yoga jobs were green:- https://zuul.opens

[Yahoo-eng-team] [Bug 2057492] [NEW] [devstack-tobiko-neutron] periodic broken running on ubuntu focal

2024-03-12 Thread yatin
Public bug reported: The job is broken in stable/zed and stable/2023.1 where it's running on ubuntu focal since patch https://review.opendev.org/c/x/tobiko/+/910589 Fails like:- 2024-03-12 02:33:20.806847 | controller | interpreter = self.creator.interpreter 2024-03-12 02:33:20.806858 | cont

[Yahoo-eng-team] [Bug 2058378] [NEW] [devstack-tobiko-neutron] stable/2023.2 periodic broken with ERROR: Cannot install tox==4.13.0, tox==4.14.0 and tox==4.14.1

2024-03-19 Thread yatin
Public bug reported: Fails like:- 2024-03-17 03:08:11.011222 | controller | ++ /opt/stack/devstack-plugin-tobiko/devstack/plugin.sh:install_tobiko_deps:14 : pip_install 'tox>=4.13' 2024-03-17 03:08:11.036778 | controller | Using python 3.10 to install tox>=4.13 2024-03-17 03:08:11.040275 | con

[Yahoo-eng-team] [Bug 2060029] [NEW] [devstack-tobiko-neutron] stable/2023.1 periodic failing stateless secgroup tests

2024-04-02 Thread yatin
Public bug reported: One of the test fails like:- testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/x/tobiko/tobiko/tests/scenario/neutron/test_security_groups.py", line 227, in test_security_group_stateful_to_stateless_switch s

[Yahoo-eng-team] [Bug 2075151] [NEW] ovs-master jobs failing with lib/actions.c:4549:9: error: struct ofpact_sample has no member named obs_domain_imm; did you mean obs_domain_id?

2024-07-30 Thread yatin
l.openstack.org/builds?job_name=neutron-ovn-tempest- ipv6-only-ovs-master&project=openstack/neutron need to use OVS_BRANCH=branch3.4 to be compatible with OVN_BRANCH=main ** Affects: neutron Importance: High Assignee: yatin (yatinkarel) Status: In Progress ** Changed in: ne

[Yahoo-eng-team] [Bug 2076430] [NEW] rally job broken with recent keystone change and fails with ValueError: Cannot convert datetime.date(2024, 8, 7) to primitive

2024-08-09 Thread yatin
Public bug reported: Test fails as:- 2024-08-07 17:06:05.650302 | controller | 2024-08-07 17:06:05.647 76504 ERROR rally_openstack.common.osclients [-] Unable to authenticate for user c_rally_927546a8_h6aDLbnK in project c_rally_927546a8_ahUUmlp9: keystoneauth1.exceptions.http.InternalServerErr

[Yahoo-eng-team] [Bug 2076430] Re: rally job broken with recent keystone change and fails with ValueError: Cannot convert datetime.date(2024, 8, 7) to primitive

2024-08-12 Thread yatin
<< Testing keystone revert in https://review.opendev.org/c/openstack/neutron/+/925968 The job passed with the revert. << Looks like the issue is with osprofiler being enabled for keystone in the neutron gate, but it's not being enabled in keystone gates, so that's why we didn't see the issue th

[Yahoo-eng-team] [Bug 2052915] Re: "neutron-ovs-grenade-multinode" and "neutron-ovn-grenade-multinode" failing in 2023.1 and Zed

2024-08-26 Thread yatin
Closing for neutron too as zed moved to unmaintained and we need not run grenade jobs there. ** Changed in: neutron Status: Triaged => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpa

[Yahoo-eng-team] [Bug 2011600] Re: functional test_get_datapath_id fails with neutron.common.utils.WaitTimeout: Timed out after 5 seconds

2024-08-26 Thread yatin
*** This bug is a duplicate of bug 2000163 *** https://bugs.launchpad.net/bugs/2000163 ** This bug has been marked a duplicate of bug 2000163 [FT] Error in "test_get_datapath_id" -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed

[Yahoo-eng-team] [Bug 2024160] Re: [OVN][Trunk] subport doesn't reach status ACTIVE

2024-08-26 Thread yatin
** Changed in: neutron Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2024160 Title: [OVN][Trunk] subport doesn't reach status ACTIVE Status in

[Yahoo-eng-team] [Bug 2007353] Re: Functional test neutron.tests.functional.services.ovn_l3.test_plugin.TestRouter.test_gateway_chassis_rebalance fails intermittently

2024-08-26 Thread yatin
** Changed in: neutron Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2007353 Title: Functional test neutron.tests.functional.services.ovn_l3.

[Yahoo-eng-team] [Bug 1971569] [NEW] [neutron][api] test_log_deleted_with_corresponding_security_group failing randomly

2022-05-04 Thread yatin
Public bug reported: test_log_deleted_with_corresponding_security_group api tests randomly failing with below Trace:- Traceback (most recent call last): File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/api/admin/test_logging.py", line 99, in test_log_de

[Yahoo-eng-team] [Bug 1971050] Re: Nested KVM Networking Issue

2022-05-09 Thread yatin
@Modyn, Ok Thanks for confirming, will close the bug as INVALID for neutron as i don't see anything to fix for this on neutron side. ** Changed in: neutron Status: New => Incomplete ** Changed in: neutron Status: Incomplete => Invalid -- You received this bug notification because

[Yahoo-eng-team] [Bug 1972764] [NEW] [Wallaby] OVNPortForwarding._handle_lb_on_ls fails with TypeError: _handle_lb_on_ls() got an unexpected keyword argument 'context'

2022-05-09 Thread yatin
ded in https://review.opendev.org/q/I0c4d492887216cad7a8155dceb738389f2886376 and backported till wallaby. Xena+ are ok, only wallaby impacted because before xena old notification format is used where arguments are passed as kwargs. ** Affects: neutron Importance: Undecided Assignee: yatin (yat

[Yahoo-eng-team] [Bug 1973221] [NEW] [master] pep8 jobs TIMING OUT randomly

2022-05-12 Thread yatin
Public bug reported: After https://review.opendev.org/c/openstack/neutron/+/806246 patch pep8 job is taking more than 30 minutes and TIMES_OUT. Specifically flake8 is taking much time now. Before the patch approx 2 minutes and now more than 12 minutes. builds:- https://zuul.opendev.org/t/openstac

[Yahoo-eng-team] [Bug 1973783] [NEW] [devstack] Segment plugin reports Traceback as placement client not configured

2022-05-17 Thread yatin
plugin is required to determine endpoint URL Ex. log:- https://8430494fc120d4e2add1-92777588630241a74fd2839fb5cc6a5d.ssl.cf5.rackcdn.com/841118/1/gate/neutron- tempest-plugin-scenario-openvswitch/92d6c29/controller/logs/screen-q- svc.txt This is happening as placement client is not configured with

[Yahoo-eng-team] [Bug 1976323] [NEW] [functional/fullstack][master] move fips periodic job to CentOS 9-Stream

2022-05-30 Thread yatin
Public bug reported: Currently these periodic jobs are running on CentOS 8-Stream(with python 3.6) and failing[1][2]. These are failing as master no longer supports py3.6. To unblock switching these jobs to run functional/fullstack tests with python3.8[3] and disabling dbcounter installation. Ide

[Yahoo-eng-team] [Bug 1976360] Re: [sqlalchemy-20] Missing DB context in networking-sfc flow classifier

2022-06-02 Thread yatin
This should have opened against networking-sfc, it's fixed with https://review.opendev.org/c/openstack/networking-sfc/+/844251. Will update project. ** Project changed: neutron => networking-sfc ** Changed in: networking-sfc Status: Confirmed => Fix Committed -- You received this bug not

[Yahoo-eng-team] [Bug 1978369] Re: [ovn] External Gateway Loop in NB SB DB

2022-06-12 Thread yatin
*** This bug is a duplicate of bug 1973347 *** https://bugs.launchpad.net/bugs/1973347 This looks duplicate of https://bugs.launchpad.net/neutron/+bug/1973347 and is fixed with https://review.opendev.org/c/openstack/neutron/+/842147. This should be backported to stable branches as well. @Amma

[Yahoo-eng-team] [Bug 1940243] Re: Neutron-tempest-plugin scenario tests - oom-killer is killing mysql process

2022-06-14 Thread yatin
Reopening the bz as seeing few failures(5 failures in last 15 days as per https://opensearch.logs.openstack.org) in linuxbridge and openvswitch scenario jobs:- https://1b33868f301e2201a22c-a64bb815b8796eabf8a53948331bd878.ssl.cf5.rackcdn.com/845366/2/check/neutron-tempest-plugin-openvswitch/aa310

[Yahoo-eng-team] [Bug 1978938] [NEW] [9-stream][master][yoga] ovs/ovn fips jobs fails randomly with DNS issue

2022-06-16 Thread yatin
release-fips&project=openstack/neutron - https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovs-tempest-fips&project=openstack/neutron ** Affects: neutron Importance: High Assignee: yatin (yatinkarel) Status: In Progress ** Changed in: neutron Status: New =>

[Yahoo-eng-team] [Bug 1979031] [NEW] [master] fullstack functional jobs broken with pyrouet2-0.6.12

2022-06-16 Thread yatin
Public bug reported: After pyroute2-0.6.12 update[1], multiple tests in fullstack/functional jobs failing[2][3]. Noting some failures here:- functional:- AssertionError: CIDR 2001:db8::/64 not found in the list of routes AssertionError: Route not found: {'table': 'main', 'cidr': '192.168.0.0/24',

[Yahoo-eng-team] [Bug 1981963] [NEW] Some jobs broken post pyroute2 update to 0.7.1

2022-07-18 Thread yatin
Public bug reported: pyroute2 updated to 0.7.1 with https://review.opendev.org/c/openstack/requirements/+/849790, since then couple of jobs are broken, like:- - https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovs-grenade-dvr-multinode&branch=master - https://zuul.opendev.org/t/open

[Yahoo-eng-team] [Bug 1989057] [NEW] [master] [functional] test__get_dvr_subnet_ids_on_host_query failing with oslo.db-12.1.0 release

2022-09-07 Thread yatin
Public bug reported: Fails as below:- ft1.12: neutron.tests.functional.services.l3_router.test_l3_dvr_ha_router_plugin.L3DvrHATestCase.test__get_dvr_subnet_ids_on_host_querytesttools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack

[Yahoo-eng-team] [Bug 1992352] [NEW] [OVN] POST requests stucks when rabbitmq is not available

2022-10-10 Thread yatin
Public bug reported: As some of the operations relies on Messaging Callbacks[1][2], these requests get's stuck when messaging driver like rabbitmq is not available, For OVN without any other agent running, there is no consumer for these messages so these operations should skip messaging callbacks.

[Yahoo-eng-team] [Bug 1970679] Re: neutron-tempest-plugin-designate-scenario cross project job is failing on OVN

2022-10-20 Thread yatin
Based on https://bugs.launchpad.net/neutron/+bug/1970679/comments/5 and https://review.opendev.org/c/openstack/devstack/+/848548 closing the issue. ** Changed in: neutron Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering

[Yahoo-eng-team] [Bug 1979047] Re: Interface attach fails with libvirt.libvirtError: internal error: unable to execute QEMU command 'netdev_add': File descriptor named '(null)' has not been found

2022-10-20 Thread yatin
** Changed in: neutron Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1979047 Title: Interface attach fails with libvirt.libvirtError: intern

[Yahoo-eng-team] [Bug 1973783] Re: [devstack] Segment plugin reports Traceback as placement client not configured

2022-10-20 Thread yatin
** Changed in: neutron Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1973783 Title: [devstack] Segment plugin reports Traceback as placement

[Yahoo-eng-team] [Bug 1972764] Re: [Wallaby] OVNPortForwarding._handle_lb_on_ls fails with TypeError: _handle_lb_on_ls() got an unexpected keyword argument 'context'

2022-10-20 Thread yatin
** Changed in: neutron Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1972764 Title: [Wallaby] OVNPortForwarding._handle_lb_on_ls fails with

  1   2   >