Ah ok thanks, well generally it's not a blocker, except that we first have to upgrade to 1.10.
We have done a lot of migrations in the last few months and given the size of the project it always takes a long time from one version to the other, so just not sure when I'll be able to test these changes on our project.. On Tuesday, September 6, 2016 at 5:02:24 PM UTC+1, Tim Graham wrote: > > Per our support versions policy, Django 1.9.x is only receiving fixes for > security and data loss issues. Migrations optimizations are unlikely to > qualify for a backport to Django 1.10.x either. Hopefully that doesn't > demotivate you from contributing an improvement. > > > https://docs.djangoproject.com/en/dev/internals/security/#supported-versions > > On Monday, September 5, 2016 at 9:55:30 AM UTC-4, andrea crotti wrote: >> >> Thanks Marcus and others, this issue is in general getting worse and >> worse. >> >> We did some more investigation and we should probably just help out the >> work you are doing. >> >> Do you think it would be possible to apply these changes also on Django >> 1.9.9 or it has to be from master/1.10 only? >> >> I also tried another approach: >> - use "./manage.py sqlmigrate <app> <name>" to generate the SQL >> - add in the generated SQL the INSERT to django_migrations >> - run the SQL manually in the DB >> >> This actually works and could be automated, but I'm quite sure it's not a >> good idea.. >> But what do you think could go wrong with this approach? >> >> I would say that as long as the dependencies are met and things are done >> in the right order it might be safe enough, but of course we get into muddy >> waters.. >> >> On Monday, August 8, 2016 at 2:36:23 PM UTC+1, Tim Graham wrote: >>> >>> I think that error suggests that you disabled migrations for an app >>> (something_else) that's a dependency of another app that doesn't have >>> migrations disabled (something). That isn't permitted. I guess there's no >>> good solution for your requirements as of now. >>> >>> On Monday, August 8, 2016 at 6:02:48 AM UTC-4, andrea crotti wrote: >>>> >>>> Since we also use --keepdb I thought we could simply avoid running all >>>> the migrations, and maybe dynamically even skip all the apps that were not >>>> changed at all. >>>> >>>> So I just tried something silly like this: >>>> >>>> >>>> NO_MIGRATIONS = os.environ.get('NO_MIGRATIONS', 'false').lower() in >>>> ('true', '1') >>>> if NO_MIGRATIONS: >>>> for app in INSTALLED_APPS: >>>> if app not in MIGRATION_MODULES: >>>> MIGRATION_MODULES[app] = None >>>> >>>> >>>> But I get this error: >>>> >>>> django.db.migrations.exceptions.NodeNotFoundError: Migration >>>> something.0001_initial dependencies reference nonexistent parent node >>>> ('something_else', u'0001_initial') >>>> >>>> So I guess it's not as simple (and maybe that ticket is what that's >>>> about, but sadly it's not even merged in master now from my understanding). >>>> >>>> PS. so to clarify our tests do rely on migrations (and there are some >>>> data migrations too) but since we normally use --keepdb anyway we >>>> definitively not need to run only a few migrations at a time max (and even >>>> then sometimes it takes ages and uses all the RAM). >>>> >>>> On Saturday, August 6, 2016 at 4:03:50 PM UTC+1, Tim Graham wrote: >>>>> >>>>> As long as your tests don't rely on any data migrations, you can set >>>>> MIGRATIONS_MODULES['app'] = None for all apps in a test settings file as >>>>> described in >>>>> https://docs.djangoproject.com/en/stable/ref/settings/#std:setting-MIGRATION_MODULES >>>>> >>>>> to disable migrations while testing. There's a ticket to make that a bit >>>>> easier: https://code.djangoproject.com/ticket/25388 >>>>> >>>>> On Saturday, August 6, 2016 at 6:07:04 AM UTC-4, andrea crotti wrote: >>>>>> >>>>>> >>>>>> Ok great thanks for the answer Markus. >>>>>> And yes I can try to help, it's quite a big issue for us.. >>>>>> Anything else we can do to improve the situation in the meanwhile? >>>>>> >>>>>> In general I was wondering why do we need to handle all the "baggage" >>>>>> of supporting backward migrations and doing things 100% properly while >>>>>> running migrations just for tests. >>>>>> >>>>>> I think that in general if there was just a way to render all the SQL >>>>>> that needs to be run statically for tests it would be great. >>>>>> >>>>>> On Friday, August 5, 2016 at 11:07:56 AM UTC+1, andrea crotti wrote: >>>>>>> >>>>>>> We have a very big Django project with > 100 apps and > 300 models. >>>>>>> We had some massive issues with Django 1.8 and migrations, which >>>>>>> would take forever and just take all the computer memory after a while. >>>>>>> >>>>>>> Now with Django 1.9 things improved, however we are again back with >>>>>>> some extremely bad performances and massive memory usages (up to 8GB of >>>>>>> RAM >>>>>>> for example), sometimes just to run ONE single migration. >>>>>>> >>>>>>> It's not even entirely deterministic though sometimes killing the >>>>>>> process and doing it again just works. >>>>>>> >>>>>>> I'm attaching the profile graph (done wtith gprof2dot) from running >>>>>>> "./manage migrate" and one thing that clearly looks bad is that >>>>>>> *render* >>>>>>> for example is called 44355 times, which is definitively not normal. >>>>>>> >>>>>>> Any idea about about what we can do about this and what could be the >>>>>>> problem? >>>>>>> I have the impression that it's related with the amount of models >>>>>>> and how they are interconnected, and mabye some caching would avoid all >>>>>>> this extra computation. >>>>>>> >>>>>>> Noone else has similar issues? >>>>>>> Thanks >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/7ce1cc3c-325f-496e-a76d-f7b30fe9f977%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
