Yes, the large number of weird things people need custom SQL for is why I want to push for migrations much more overall - especially for cases like post_syncdb (should be replaced by data migrations) and the arbitrary initial SQL file support (should be replaced by custom migrations).
I don't plan to add in an equivalent to "syncdb --all" - the squashing should be enough to make things nice and fast for testing, and will have the added benefit where your test database matches your production one more closely. Squash (what was previously called rebase) is designed to let you throw away old migrations, especially in a case where you tightly control all installs. There might be some scope for a version of the migrate command that does a one-shot, in-memory squash-then-execute - essentially a very quick way of doing initial migrations - but that depends if it turns out to be useful and how fast the migration optimiser/compressor that runs on the squashes will be. Andrew On Fri, May 31, 2013 at 1:38 PM, Marc Tamlyn <[email protected]> wrote: > I have generally not kept my migrations so they always work from scratch > for similar reasons Luke said - Data migrations are often dependent on the > data in the site - this is particularly relevant to content-driven (CMSy) > sites where I may need to move a load of page about as a data migration at > some point in the history, but this data will not exist for the tests. > These "throwaway" data migrations are IMO pretty common. As a consequence > (and given the speed requirements) I've generally never run a site without > SOUTH_TESTS_MIGRATE set. > > Also, when using an older library which has a long migration history > included, when we start a new site using it we generally use `syncdb --all` > so as not to worry about the history. Will this command still be supported > by the new `migrate` command? > > I can see the argument for making sure the migrations work, and they may > be necessary for migrations containing custom SQL the site depends on, but > I would suggest that many developers effectively treat migrations as things > that can be thrown away once they've been run against all the relevant > databases. Perhaps the rebase command needs to consider this - how can we > throw away old migrations (especially data migrations). > > (PS: +1 to option 3 in principle) > > > On Friday, 31 May 2013 12:44:20 UTC+1, Anssi Kääriäinen wrote: >> >> On 05/31/2013 02:08 PM, Andrew Godwin wrote: >> >> One problem with this is that you have to be careful to write >>> migrations >>> that will always work from scratch. This is best practice, but I have on >>> occasion used data migrations that were for specific problems, and may >>> have depended on specific data in the database. I've also used them for >>> populating parts of the database at specific upgrade points. Usually I >>> wouldn't want this data to be installed for my tests - I want my tests >>> to explicitly do the setup they need, and not be complicated by data >>> from data migrations. >>> >>> For these reasons, and because of the slowness of running all the >>> migrations, there are some projects where I have a settings file for >>> running the tests that excludes South from INSTALLED_APPS, which solves >>> the problem nicely. >>> >>> The slowness problem can be fixed by squashing, but it seems a shame to >>> require that simply for the sake of running tests. And it doesn't solve >>> the other problems. I would like to be able to turn off migrations for >>> tests, and just skip to the final schema. >>> >> >> Yes, this has traditionally been a problem, and it's difficult to say >> what to do here. >> >> I personally have the opinion that stuff you add in data migrations >> _should_ be part of the tests - after all, it's usually crucial data that >> the site won't work without, and you're never going to test an installation >> without them. >> >> However, I understand people don't like doing this. Problem is, the >> alternative is to reserve the old syncdb method for running tests, and then >> to introduce a new setting (which in South is SOUTH_TESTS_MIGRATE) that >> lets users turn it on and off, which seems like a bad road to go down. >> >> I'd rather keep it so you must run migrations for tests - as I believe >> we should be making people do this as a general case - and if you want, >> having the data migration operations have an option you can pass where they >> no-op during tests, something like: >> >> operations = [ >> Python( >> """ >> for author in Author.objects.all(): >> author.fix() >> """, >> skip_during_tests=True, >> ), >> ] >> >> >> If test database is set up by syncdb and you have some SQL that isn't >> model migrations, then you will likely need two sets of migrations. One >> that applies to production database, and one that applies to fresh database >> from syncdb. You will need to constantly keep the "applies to syncdb >> database" migrations updated, and there is no guarantee that you will >> actually arrive to similar state than what you have in production database >> (in other words, migrations should be part of your tests like Andrew said). >> >> For example the SQL could be trigger or view creation SQL. I know, not >> the most typical setup. But there are enough users who want to use >> traditional SQL for schema alterations that this use case should be part of >> design goals. >> >> Dump + load of production schema is surprisingly good way to attack this >> problem - you are testing against exact copy of your production schema, and >> only "in development" migrations need to be applied. This is not a good >> default, but very useful if you happen to need it. My understanding is that >> I can actually do this somewhat easily with the planned design. >> >> - Anssi >> > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
