Hi Marcin, Some of these are problems, yes, but you have to understand they are tradeoffs and the alternative is, in my opinion, worse.
> I believe that there exist some solution, which drops app layer dependency ands allow using custom fields I would love to see what this would be. I spent a decade trying not to import custom fields from the code to allow this and never succeeded; the only way that I could make it work was to import them. Because Django allows fields to change their type based on the database being talked to, you can't bake the type into the migration at creation time. > Python code in migrations This has always been optional; Django's makemigrations will never put in RunPython for you, so if you choose to use it, then it's your choice and you get the problems along with it (more tied to application code). > Separate files for changesets This is needed if you are going to have migrations per-app and thus let apps ship migrations. If you want to not have per-app migrations, then you can have just the one file, but then you make third-party apps have to ship migration snippets you need to remember to include when you make the next migration. > Squashing is a just a workaround tool for migrations design issues Yes, specifically it is designed to help solve the custom-fields-app-dependency issue. It's also just nice to have anyway when you get to 100 migrations. > A discussion about removing app layer dependency and removing or limiting RunPython usage, mostly. OK, so what is your proposed alternative for custom fields? It is no good to propose to remove something without proposing what to do in its place; we can't drop custom field support in migrations. I think RunPython is solvable by our users; if you want to limit usage, then just have a linter rule in your project that rejects RunPython being committed by your developers. Andrew On Fri, Jun 23, 2017 at 2:25 AM, Marcin Nowak <[email protected]> wrote: > Hi. > > At the begining please forgive my engilsh - i'm not a native speaker. > > I wrote here and in other places my thoughts about db migrations few > times, and probably Tim remembers me so well. > My opinion was not changed, but I realized that I cannot leave Django > ecosystem for a long time. > In that case I'd like to talk about migrations, their advantages and > disadvantages, and about possible solutions. > > Advantages: > > - fast (automatic) creation > - database independent (model-centric) > - a standard for django itself and reusable apps > - possibilty to create migrations manually > > Disadvantages: > > - dependent on application layer (a python code - field, model > classes, etc) > - allowing python code within migrations > - separate files for changesets makes ugly conflicts when merging > branches > - squashing required > > I'd like to focus on disadvantages, because they're a casue of using > alternative solutions by me (Liquibase in that case). > > > *Application layer dependency* > > This is something whch causes fails of whole migration system. > References to the application layer are included within migration files, > because of saving a "model state". > Any significant code change will broke migrations. We must avoid such > situations by squashing migrations at "the right time". > > In my opinion migrations should be application independent. > Unfortunatelly whole system is based on models written in Python, which may > includue custom solutions (i.e. model fields). > > I understand that it is hard to cut-out this feature, but I believe that > there exist some solution, which drops app layer dependency ands allow > using custom fields. > > > *Python code in migrations* > > This is a generally bad idea. Any python code is strictly related to the > time. When code changes, a "pythonic" migration may fail. > And you will never know about failure until you setup CI properly. > > There are very rare cases, when something from app layer must be called > between releases. > In that cases I'm using management commands, but Liquibase allows me to > execute any binary. > It is not so straightforward, so as a developer you feel that you're doing > something strange/non-standard, and you should be careful. > > The general problem is that some migrations stops working properly in the > time. > This should never happen. Well... I'm used to this and I adhere to this > principle. > > I'm using Liquibase for years and my migrations broke few times mostly on > changesets based on binaries execution (a python code through manage > commands). > > > *Separate files for changesets* > > I like Liquibase because of possibility of using files containing more > changesets. > When I need to split migration files, I can do that and use "include" > directive. > There are advantages: easy conflict resolution, just one file per database > and per current major release, and full changes history in older files. > > Django produces spearate migraiton files. This causes conflicts and > reordering migrations by declarations ("depends"). > And because squashing is almost required for long project, you're loosing > a changesets history. > > > *Squashing required* > > For long-live projects this is a required operation. Squashing deletes > changes history and removes obsolete python code from migrations. > Squashing is a just a workaround tool for migrations design issues. > > > > *What I need?* > > Well, I'd like to simplify my work. That's obvious. Maintaining a > Liquibase changes outside Django is harder and requires more time, > especially when I'm adding new apps or upgrading existing ones. > But I don't want to switch back to the Django migrations, because of the > design issues described above (mostly beacuse of the app layer dependency > and squashing requirement). > > I started prototyping a tool which translates Django migrations into pure > SQL, which can be embedded in a Liquibase changesets. > But Python migrations can't be translated to SQL, of course. And the worst > thing is that Django provides Python migrations even for contrib apps. > > In that case I realised, that building such tool without changing a > concept of the Django migrations (by you - Django Developers), is a little > unreasonable, until Python code execution is accepted and used internally > (contenttypes, 0002). > > > *What I would like to achieve by this post?* > > A discussion about removing app layer dependency and removing or limiting > RunPython usage, mostly. > This should eliminate requirement of squashing and increase migrations > stability. > > Separate files vs big file is not important now. This is just unhandy, but > does not produce failures. > > > Kind Regards, > Marcin > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/django-developers. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/django-developers/bd46c905-4402-41b8-a56c- > 1783020467f5%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/bd46c905-4402-41b8-a56c-1783020467f5%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1ur8c-_cNyA2aHavRdsncx%2Bcz1tsZt7V5RqYgLmky8MwCw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
