Yes, it's basically a fundamental design flaw of having the migrations represented this way - it makes autodetection and code generation very accurate, but at the expense of calculation time. There is some optimisation work that can be done to try and avoid building intermediate states, but it's a pretty complex code optimisation problem.
I think it's an acceptable flaw providing you take the time to squash every so often, and I prefer it to South's flaw of missing dependencies out and making tables incorrectly sometimes, but I can understand that it's kind of ridiculously slow sometimes. Andrew On Thu, Jan 7, 2016 at 2:03 PM, Aymeric Augustin < [email protected]> wrote: > As far as I understand, the CPU cost comes from generating a full set of > model classes for each step of the migration history. That’s consistent > with the profile sent by Florian. > > I usually end up throwing away the migration history and regenerating a > new set of migrations when I get to that point. This requires truncating > the django_migrations table manually and faking the new set of migrations. > > If the project doesn’t use data migrations, squashmigrations may achieve > the same effect. Sadly real-life projects tend to have data migrations > whose only purpose is to run once in production. They prevent full > squashing. > > -- > Aymeric. > > On 7 janv. 2016, at 22:36, Florian Apolloner <[email protected]> > wrote: > > Okay, my base time is currently 95 seconds, where we are spending >60 > seconds in ModelBase.__new__. I've attached a picture of the profile run, > there are not really any obvious spots, so shaving time off there will be > hard I fear. > > Cheers, > Florian > > On Thursday, January 7, 2016 at 9:49:11 PM UTC+1, Pete wrote: >> >> It looks like there are some other fixes in 1.9 that weren't covered >> by my monkeypatch. I upgraded the project and included a new cProfile >> in the ticket https://code.djangoproject.com/ticket/22608#comment:23. >> It shaved 100s off, but still takes 2.5 minutes to create an empty >> migration. >> >> Florian, I'll contact you off-list about the project. >> >> -- Pete >> >> On Thu, Jan 7, 2016 at 1:11 PM, Florian Apolloner <[email protected]> >> wrote: >> > Also, is there any chance that I can get access to this project to >> profile a >> > little bit more and identify some hotspots? >> > >> > On Thursday, January 7, 2016 at 9:10:26 PM UTC+1, Florian Apolloner >> wrote: >> >> >> >> Just looking at the topmost three entries: >> >> _expire_cache spends 18 second in delattr, can you try replacing that >> with >> >> del self.__dict__[cache_key]. Maybe also try moving self.__dict__ in a >> local >> >> variable since it will get referenced a lot >> >> subclass_exception seems awfully slow, maybe there are some creative >> ways >> >> to speed it up? >> >> It would also be interesting to know where the entries from >> collections.py >> >> come and replace it with more efficient structures. >> >> >> >> The rest is as Tim said. >> >> >> >> Cheers, >> >> Florian >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Django developers (Contributions to Django itself)" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at https://groups.google.com/group/django-developers. >> > To view this discussion on the web visit >> > >> https://groups.google.com/d/msgid/django-developers/7b687031-050f-4de3-854c-afd6cbd00ca5%40googlegroups.com. >> >> > >> > For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/b2af60ae-c071-4f7c-94e7-7e3c2adb8e63%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/b2af60ae-c071-4f7c-94e7-7e3c2adb8e63%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > <Screenshot from 2016-01-07 22-31-45.png> > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/1A91799C-A133-4F71-8FBE-2451F619CE5F%40polytechnique.org > <https://groups.google.com/d/msgid/django-developers/1A91799C-A133-4F71-8FBE-2451F619CE5F%40polytechnique.org?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1upWBT2ifKeF4mDAQmAgf6Oq_tHc74rDTSc4zrP2cneCFg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
