Thanks Andrea, this is a known issue. Let me give you a bit of a history on this:
In 1.7 Django didn't do any caching. That was awfully slow for huge projects like yours. In 1.8 Django introduced heavy caching. Things improved speed wise but failed horrifically when you were altering relations or related models In 1.8.1 (I believe it was the .1 patch release, not sure right now) Django reduced the caching and ensured that related models are "reloaded" [1, 2]. At least that was the plan. In 1.9 Django slightly changed the way it detected the migration plan and dropped an undocumented and unimplemented feature [3, 4]. This improved the forward migrations but remains slow for backwards migrations. My plan is to get rid of the "render()" method during schema alterations. Only when using RunPython the requested models would need to be rendered. I started implementing that in [5]. If you're looking into contributing there, I'm happy to talk to you in more detail, preferably on IRC in #django-dev. Cheers, /Markus [1] https://github.com/django/django/commit/a1ba4627931591b80afa46e38e261f354151d91a [2] https://code.djangoproject.com/ticket/24225 [3] https://github.com/django/django/commit/5aa55038ca9ac44b440b56d1fc4e79c876e51393 [4] https://code.djangoproject.com/ticket/24745 [5] https://github.com/MarkusH/django/commits/schemaeditor-modelstate On Fri, Aug 05, 2016 at 03:07:56AM -0700, andrea crotti wrote:
We have a very big Django project with > 100 apps and > 300 models. We had some massive issues with Django 1.8 and migrations, which would take forever and just take all the computer memory after a while. Now with Django 1.9 things improved, however we are again back with some extremely bad performances and massive memory usages (up to 8GB of RAM for example), sometimes just to run ONE single migration. It's not even entirely deterministic though sometimes killing the process and doing it again just works. I'm attaching the profile graph (done wtith gprof2dot) from running "./manage migrate" and one thing that clearly looks bad is that *render* for example is called 44355 times, which is definitively not normal. Any idea about about what we can do about this and what could be the problem? I have the impression that it's related with the amount of models and how they are interconnected, and mabye some caching would avoid all this extra computation. Noone else has similar issues? Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/5b67eccf-d48f-4926-86ec-4bd2a8427dcb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
-- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/20160805115406.GD30863%40inel.local. For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: PGP signature
