Resending with [Neutron] tag. Salvatore
On 14 April 2014 16:00, Salvatore Orlando <sorla...@nicira.com> wrote: > This is a rather long post. However the gist of it is that Neutron > migrations are failing to correctly perform database upgrades when service > plugins are involved, and this probably means the conditional migration > path we designed for the Grizzly release is proving not robust enough when > dealing with service plugins. > > Corrective actions are therefore required; otherwise there is a risk the > Grenade jobs for neutron can't be made voting again. > This issue is already being tracked with bug 1307344 which was > independently filed and amended by Sean to reflect the actual issue. > > Grenade is currently failing because of this issue. Indeed the 'metering > plugin' is enabled in devstack for Icehouse but now Havana. > A quick fix for grenade has already been rejected (backport metering > support to stable/havana) another quick fix instead would be running > grenade without metering at all. > > However, whether we put or not a patch to make grenade running, there is > still an issue that needs to be solved. > Now, if you're not yet bored you might try now reading the long version of > this post... > > Problem statement: > > Enabling or disabling a service plugin in neutron configuration might be > done at any point and can lead migrations to build a database which is out > of sync with expectations > > Assume: > > Current migration X (Service plugin SP enabled here) > Service plugin SP introduced at migration F1 < X > No auto-generation of DB schemas > The migration F1 will be skipped > > When SP is enabled, the relevant DB schema won't exist and the feature > will not work. > > Other failure scenarios: > > Current migration X > SP introduced at migration F1 < X (but not yet enabled) > SP enabled at migration Y > F1 > X > SP updated at migration F2 | X < F2 < Y. > > F2 will fail because schemas are not yet ready for feature, even with auto > schema generation > > SP introduced at migration F0, and enabled at migration F0 > SP disabled at migration X > SP updated at migration F1, which is skipped because it's disabled > SP enabled again at migration Y. Feature won't work as migration X < F1 < > Y was never executed. > > Requirements for a possible solution: > > Should solve problem not only for greenfield deployments but also for > existing ones > Must not make migration generation more complex or introduce error-prone > behaviours > > Possible solutions: > > 1) Specify that all migrations must run for every plugin (*) unless they > are really introducing schemas which are specific to a particular > technology (such as uuid mappings between neutron and backed) > > This approach will probably ensure all the important migrations are > executed. > However, it is not back portable, unless deployers decide to tear down and > rebuild their databases from scratch, which is unlikely to happen. > > To this aim "recovery" scripts might be provided to sync up the schema > state of specific service plugins with the current alembic migration > registered in the neutron database; or appropriate migrations can be added > in the path to fix database schemas. > > (*) Neutron does not address, and probably never will address, switching > from plugin "a" to "b" for a given service (e.g.: core features). > > 2) Have plugin-specific migration paths. > > This is not supported by alembic. In this case probably the best solution > would be to have completely different databases for the main plugin and > service plugins, which is however not possible because of foreign key > relationships among tables in different service plugins. > > Also, there are likely to be migrations which alter core and service > schemas at the same time. Adapting these migrations might be a > problem.Extra work will also be needed to make this solution work with > existing deployments. > > 3) Make migrations smarter > > When migration X is executed, it checks whether any previous migration for > the plugins it alters has been executed. If not it will execute the missing > migrations as well. > This *might* work; However, alembic currently offers no support for it, as > it's not possible to perform any form of "introspection" in the migration > themselves. > > The migration engine will also be augmented as the last executed migration > for each plugin should be stored in the database. > This might also prove to be error prone and lead to unexpected behaviours. > For example: > > Until migration X deployment uses plugin A which provides services S1 and > S2 > At migration Y user still uses plugin for service S1, but switches to > plugin B service S2 > Migration Z>Y applies to plugin B. System detects no migration for plugin > B have been executed so far. > However, the migrations for the service S2, offered by B were already > executed when running plugin A > System tries to execute migrations for plugin B which will fail as they > were actually already executed for plugin A > > The system could be made smarter by storing also a list of "known" > migrations, including whether they were executed or skipped. > > Summarising, in my opinion the approach #2 is not even worth being > considered. Between approach #1 and #3, I'd prefer the simplest one and > vote for approach #1. > > > Salvatore > > > > > > >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
