Hi Kevin, Thanks for your answer, that is what I was looking for! I'll check with you in irc to decide which workaround is better: 1. Mocking NeutronDbSubnet fetch_subnet for opencontrail tests. 2. Using session.query() directly in NeutronDbSubnet fetch_subnet.
- Pavel Bondar On 30.04.2015 22:46, Kevin Benton wrote: > The OpenContrail plugin itself doesn't even use the Neutron DB. I > believe what you are observing is a side effect of the fake server they > have for their tests, which does inherit the neutron DB. > > When you call a method on the core plugin in the contrail unit test > case, it will go through their request logic and will be piped into the > fake server. During this time, the db session that was associated with > the original context passed to the core plugin will be lost do to its > conversion to a dict.[1, 2] > > So I believe what you're seeing is this. > > 1. The FakeServer gets create_port called and starts its transactions. > 2. It now hits the ipam driver which calls out to the neutron manager to > get the core plugin handle, which is actually the contrail plugin and > not the FakeServer. > 3. IPAM calls _get_subnet on the contrail plugin, which serializes the > context[1] and sends it to the FakeServer. > 4. The FakeServer code receives the request and deserializes the > context[2], which no longer has the db session. > 5. The FakeServer then ends up starting a new session to read the > subnet, which will interfere with the transaction you created the port > under since they are from the same engine. > > This is why you can query the DB directly rather than calling the core > plugin. The good news is that you don't have to worry because the actual > contrail plugin won't be using any of this logic so you're not actually > breaking anything. > > I think what you'll want to do is add a mock.patch for the > NeutronDbSubnet fetch_subnet method to monkey patch in a reference to > their FakeServer's _get_subnet method. Ping me on IRC (kevinbenton) if > you need help. > > 1. > https://github.com/openstack/neutron/blob/master/neutron/plugins/opencontrail/contrail_plugin.py#L111 > 2. > https://github.com/openstack/neutron/blob/master/neutron/tests/unit/plugins/opencontrail/test_contrail_plugin.py#L121 > > On Thu, Apr 30, 2015 at 6:37 AM, Pavel Bondar <pbon...@infoblox.com > <mailto:pbon...@infoblox.com>> wrote: > > Hi, > > I am debugging issue observed in OpenContrail tests[1] and so far it > does not look obvious. > > Issue: > > In create_port[2] new transaction is started. > Port gets created, but disappears right after reading subnet from plugin > in reference ipam driver[3]: > > > plugin = manager.NeutronManager.get_plugin() > > return plugin._get_subnet(context, id) > > Port no longer seen in transaction, like it never existed before > (magic?). As a result inserting IPAllocation fails with foreing key > constraint error: > > DBReferenceError: (IntegrityError) FOREIGN KEY constraint failed > u'INSERT INTO ipallocations (port_id, ip_address, subnet_id, network_id) > VALUES (?, ?, ?, ?)' ('aba6eaa2-2b2f-4ab9-97b0-4d8a36659363', > u'10.0.0.2', u'be7bb05b-d501-4cf3-a29a-3861b3b54950', > u'169f6a61-b5d0-493a-b7fa-74fd5b445c84') > }}} > > Only OpenContrail tests fail with that error (116 failures[1]). Tests > for other plugin passes fine. As I see OpenContrail is different from > other plugins: each call to plugin is wrapped into http request, so > getting subnet happens in another transaction. In tests requests.post() > is mocked and http call gets translated into self.get_subnet(...). > Stack trace from plugin._get_subnet() to db_base get_subnet() in open > contrail tests looks next[4]. > > Also single test failure with full db debug was uploaded for > investigation[5]: > - Port is inserted at 362. > - Subnet is read by plugin at 384. > - IPAllocation was tried to be inserted at 407. > Between Port and IPAllocation insert no COMMIT/ROLLBACK or delete > statement were issued, so can't find explanation why port no longer > exists on IPAllocation insert step. > Am I missing something obvious? > > For now I have several workarounds, which are basically do not use > plugin._get_subnet(). Direct session.query() works without such side > effects. > But this issue bothers me much since I can't explain why it even happens > in OpenContrail tests. > Any ideas are welcome! > > My best theory for now: OpenContrail silently wipes currently running > transaction in tests (in this case it doesn't sound good). > > Anyone can checkout and debug patch set 50 (where issue is observed) > from review page[6]. > > Thank you in advance. > > - Pavel Bondar > > [1] > > http://logs.openstack.org/36/153236/50/check/gate-neutron-python27/dd83d43/testr_results.html.gz > [2] > https://review.openstack.org/#/c/153236/50/neutron/db/db_base_plugin_v2.py > line 1578 / line 1857 > [3] > > https://review.openstack.org/#/c/153236/50/neutron/ipam/drivers/neutrondb_ipam/driver.py > line 50 > [4] http://pastebin.com/n0AqhC5x > [5] http://pastebin.com/BDBAXFy9 > [6] http://logs.openstack.org/36/153236/ > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Kevin Benton > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev