On Thu, Jan 22, 2015, Kekane, Abhishek <abhishek.kek...@nttdata.com> wrote: > With online schema changes/No downtime DB upgrades things would be > much lot easier for OpenStack deployments. > Big kudos to Johannes who initiated this feature. But as a service > provider, I'm curious to understand what is the development process > of adding new features to Kilo and future releases once the online > schema changes is in. > > > 1. Will the committer be responsible of adding new procedures of > upgrading db with minimal or zero downtime? or the online schema > changes framework itself will detect whatever db changes are required > on its own and the decision to apply db changes online or offline > will be left solely with the service provider?
The online schema change code will compare the running schema and the model and figure out what changes are needed to make the running schema match the model (it actually leverages alembic for most of this). (This automates much of the work currently done in sqlalchemy-migrate scripts) The scheduling of changes into the three phases is handled completely internally to the online schema change patch. It understands which changes are semantically safe (that can be safely applied when nova is running) and locking safe (so it doesn't block access to the table for a long time). Unless you are working on the code that implements the online schema changes, then a developer need not know how it operates. Developers just need to make changes to the model and write sqlalchemy-migrate scripts as we have always required. Eventually, developers will no longer need to write sqlalchemy-migrate scripts. This is likely to be 1 or 2 cycles away (certainly not in Kilo). There will be some minor restrictions on what kind of schema changes will be allowed. As an example column renames won't be allowed because they appear as a delete/add and we'll potentially lose data. However, this can be done as an explicit add/delete along with a data migration, if it's needed. Same thing with some column type changes. I'll clarify these in the patch when it's put up for review. > 2. Is it possible to predict how much time it would take to upgrade db > (expand/migrate/contract phases) for adding a new column, constraint. > For example, adding a new column with NULL constraint would take less > time than adding a default value. This is difficult to estimate. It varies on how fast the database server is (CPU, disk I/O, etc), the current load of the database, the number of rows in the table and the size of the data in the columns. However, I can develop some relative estimates. For instance, adding an index on MySQL 5.6 only acquires a lock very briefly and does the rest of the work in the background. It still produces load, but it doesn't block access to the table. This would be considered online safe and scheduled to the expand phase. The CREATE INDEX would appear to finish very quickly. However, other changes could potentially require a table rewrite which could be long if it's a large table (eg instance_system_metadata table), but very short if the table only has a handful of rows (eg instance_types table). I've written a test suite which takes a variety of database software (MySQL, PostgreSQL, etc), versions and storage engines (InnoDB, TokuDB, etc) and does tests to figure out which changes are online safe (as far as locking goes). I will be using this data to make better decisions on scheduling of operations to ensure the expand and contract phases don't cause any problems. I can also take that data and make it available somewhere as well and/or possibly annotate the output from the --dry-run option to explain why some operations are scheduled to migrate instead of the expand and contract phases. JE __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
