On Tue, Jul 07, 2015, Salvatore Orlando <sorla...@nicira.com> wrote: > > > On 7 July 2015 at 14:00, Paul Michali <p...@michali.net > <mailto:p...@michali.net>> wrote: > > Thanks Salvatore for the responses. See @PCM in-line... > > > > On Tue, Jul 7, 2015 at 6:14 AM Salvatore Orlando <sorla...@nicira.com > <mailto:sorla...@nicira.com>> wrote: > > Some comments inline. > > Salvatore > > On 6 July 2015 at 20:00, Paul Michali <p...@michali.net > <mailto:p...@michali.net>> wrote: > > Hi, > > I have some urgent requests about migration that I'm hoping to > get some info on. I'm working on a bug where I need to add two > (related) fields to a table for VPNaaS. Here's the objectives > related to migration... > > 1) create local_v4_ip and lcoal_v6_ip fields in the vpnservice > table > 2) for each entry in the vpnservice table: > 2.1) Get the router.gw_port.fixed_ips list > 2.2) Determine the version of each fixed IP and store the > first of each version (if any) into the appropriate new field. > > I have created a migration file, and I changed the down_revision > to be the number of the revision that is the first in the > migration chain in the VPN repo. > > Here are the many questions I have... > > When I look in the VPN repo, the HEAD file has the version > 'kilo', which is not the current head. > > > Shouldn't it the version number of the first file in the > migration chain? > > > It should indeed. How are you generating the revision script? Using > neutron-db-manage it should be updated automatically [1] > > > @PCM I ran neutron-db-manage, when in the neutron repo, and it assigned > some version, but it was not the latest in the neutron-vpnaas repo. > > > when you create a revision Alembic automatically assigns it a unique id. > However, the neutron migration CLI (neutron-db-manage) then should take care > of updating the HEAD file automatically. If this is not happening, that's > where the problem should be. > > > I checked the VPN repo and there were a chain of versions, which I used > to determine what the head should be and have set the version > accordingly. However, in the current repo, head is set to "kilo", which > appears to be incorrect. The versions are: > > 56893333aa52 > kilo <<< HEAD > 3ea02b2a773e > start_neutron_vpnaas > None > > Should I do a separate commit that fixes the HEAD file, or just fix it as > part of the bug fix I'm working on. > > > In order to pass functional tests the HEAD file must point to the topmost > revision (56893333aa52) > > > BTW, at one point, after having correctly set the HEAD and versions in my > new migration file, I think I ran neutron-db-manage check_migration, and > I think it set the HEAD to my version, but it did that in the neutron > repo, and not the VPN repo. I might have been running from the wrong > repo? > > > Yes, probably. > neutron-db-manage by default works on the neutron repo. In order to work with > a service repo you should specify it on the command line > (http://git.openstack.org/cgit/openstack/neutron/tree/neutron/db/migration/cli.py#n41). > This might also explain why the HEAD is not getting updated in your repo. I have been playing around with this. Unfortunately, --service only works for the "upgrade" command. It does not work for the "revision" command, and even less for --autogenerate.
I am working on a fix. > > > > > > For my commit, I'm assuming I change the HEAD file to use my > migration file's version? > > > You can do that manually too, yes. > > > > I set HEAD to my migration file, and my file has a down revision > of the previous head's revision. If I run 'neutron-db-manage > --config-file ../neutron/etc/neutron.conf --config-file > ../neutron/etc/neutron/plugins/ml2/ml2_conf.ini check_migration' > there is no output so I guess that is OK. > > As I develop my new migration file, is there a way that I can > test it (running neutron-db-migration, maybe)? > > > When I test migrations I usually dump the database, run the migration > with neutron-db-manage upgrade HEAD (I think it's not necessary to > specify HEAD), and restore the db from the dump if the migration > fails. > > > Is there a way to run the migration file under the debugger, as > well (importing pdb, for example)? > > > The migration process is just like any python application, so I guess > you can debug it with pdb. > > > @PCM Ah, so use "neutron-db-manage upgrade HEAD". That was the piece that > was missing. I take it there are no specific unit tests of the migration > files? > > > ... and also specify --service vpnaas > > > > > > > In the migration, I can add the columns needed. What's the best > way to fill out those fields - using raw SQL queries or create a > Session object and access the VpnService object's router object? > > > If the default value for the column is not enough, and you need to > specify a value which depends on other values in the same row I would > prefer plain SQL statements, but if that become cumbersome I guess > it's ok to use sqlalchemy's session. > > > I see there is some op.bind() call and then engine.execute(), but > could use some help on the best way to extract the needed queries > (I need to access the vpnservice's router, and then access the > (Port) gw_port relationship, and from that access the > (IPAllocation) fixed_ips list). > > > Perhaps you can point us to the review pages on gerrit, and we can > provide detailed comments there. > > > @PCM Yeah, I haven't pushed it up yet. I have a few more changes to make, > and should be able to get it up in a few days. The LP bug is 1464387. > > Essentially, in the vpnservices table, I'm adding an IPv4 and/or IPv6 > addresses for the "local" end of VPN connections that will be > established. This is to allow alternative VPN implementation (appliances, > separate S/W, H/W, VM based VPN, etc) to specify addresses different than > what is available on the Neutron router. > > However, for the reference implementation, we'll use the Neutron router's > fixed_ips list (as is done today), and to handle the migration, I'm > thinking the following is needed: > > 1) create the new columns. > 2) Identify the router for that service and obtain it's GW fixed_ips list. > 3) Pick first IPv4 address (if any) and IPv6 address (if any), and store > in new columns. > > So I need to form a query and code to do this. > > > You can put all this logic in the DB migration and it should work, from what > I can tell. > An alternative could be to leave the value unset in the migration (therefore > doing a DDL only migration), and then handle missing IP addresses in > application code. > But if these are "default" values which can change in the future > independently of the VPN's router GW IP, then I'd suggest doing that in the > migration following your approach. > > > > > > > > > Appreciate any advise here on how to debug the migration stuff... > > Paul Michali (pc_m) > > > [1] > > http://git.openstack.org/cgit/openstack/neutron/tree/neutron/db/migration/cli.py#n124 > > > > > > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
