On 5/14/15 5:44 AM, John Garbutt wrote:
On 14 May 2015 at 07:18, Manickam, Kanagaraj <kanagaraj.manic...@hp.com> wrote:
Hi Nova team,
This mail is regarding an help required on the migration from
sqlalchemy migration tool to alembic tool.
Heat is currently using sqlalchemy-migration tool and In liberty release,
we are investigating on how to bring the alembic into heat. We found that
nova has already tried the same (https://review.openstack.org/#/c/15196/ )
almost 2 years back and in Kilo release, nova is still using sqlalchemy
migration tool.
(https://github.com/openstack/nova/tree/master/nova/db/sqlalchemy/migrate_repo/versions)
So we are assuming that, in nova, you might have faced blockers to bring in
alembic.
And we would like to seek your recommendation/suggestions based
on your experience on this. This will help us to take proper direction
on using alembic in heat. Could you kindly share it.
We are currently looking at going this direction (its not merged yet though):
http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/online-schema-changes.html
If we get that working, we wouldn't need to write or generate any
migrations, we end up doing a "diff" between the desired state and the
current database state, and split that into expand and contract
phases.
To support that approach, we are no longer doing any data migrations
in our migration scripts, they are happening in a different way,
"online":
http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/flavor-from-sysmeta-to-blob.html#other-deployer-impact
There are more details on how this is all planned to fit together here:
http://docs.openstack.org/developer/nova/devref/upgrade.html
Anyways, this is why we have paused the alembic vs sqlalchemy debate
for the moment.
The "online schema changes" patch has been abandoned. I regret that I
was not able to review the full nature of this spec in time to note some
concerns I have, namely that Alembic does not plan on ever acheiving
100% "automation" of migration generation; such a thing is not possible
and would require a vast amount of development resources in any case to
constantly keep up with the ever changing features and behaviors of all
target databases. The online migration spec, AFAICT, does not offer
any place for manual migrations to be added, and I think this will be a
major problem. The decisions made by "autogenerate" I have always stated
should always be manually reviewed and corrected, so I'd be very nervous
about a system that uses autogenerate on the fly and sends those changes
directly to a production database without any review.
FWIW in Nova I am planning to look into doing a simple one-shot of the
Nova migration scripts in nova/db/sqlalchemy/migrate_repo/versions to
Alembic. The only large migration script in this repo is
216_havana.py, which consists of all Table defs that do not need any
changes. There are 39 remaining scripts, and all of these are extremely
short and simple, and just a direct manual change to Alembic style is
most expedient for these.
The more intellectual part of the exercise is to replace the use of
sqlalchemy-migrate APIs in Nova's installation scripts and test
harnesses to use Alembic as well. At that point Nova will be on a
straight Alembic install that will at first behave similarly to the
Migrate one.
Assuming I can get that done in the next few months, the next step would
be that the migration streams can be broken into branches, e.g. juno,
kilo, liberty, etc. so that we can easily add new migration files that
are backportable in place to a target backport. Additionally, the
issue of end-user migrations that are divergent from what is part of
Nova itself should also be achieved using additional branches; if a
customer wants to make their own changes to a database, such as adding
extra indexes, this can be done in their own user-specific branch that
will integrate with the primary migration stream. That would
eliminate the need for a system that is tasked with diffing live
database schemas and then executing decisions directly without any
chance to review or version-control the changes that are being made.
Does that help?
Thanks,
John
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
