I think that is a good compromise between "full" integration and "optional
integration". And as long as we document and explain it with some example
scenarios ("what to do if you upgrade the FAB provider separately") - this
should be fine.It will likely work fine if we do some "consistency check" when FAB is used as "external_db_manager" - in this case, possibly airflow should refuse to start (and show helpful message) if there is a consistency problem between what FAB expects and what is stored in alembic DB. This should handle all cases where the deployment manager **should** do migration but they did not (manually upgrading/downgrading FAB provider for example). Also I **hardly** expect changes to the DB - between FAB provider versions. If FAB 5 will be out before Airflow 3 (rather likely) - then probably the migration to FAB 5 will involve some DB changes, but this will be covered by Airflow 3 migration (and FAB >= 5 should likely be Airflow's expectation due to sqlalchemy 2 and other limits that FAB 4 has). Other than that - I don't think we should expect DB changes from FAB IMHO. So even if this problem will be "somewhat" painful when it happens, it's not something the users will have to deal with regularly J On Tue, Sep 3, 2024 at 4:27 PM Ephraim Anierobi <ephraimanier...@apache.org> wrote: > Hey everyone, > > I have started work on AIP-79 and would like to share what using the FAB > provider would be like for users after the changes are released. > > In Airflow 3, FAB will have DB reset, migrate, and downgrade commands > similar to Airflow DB commands. Contributors can add migrations to FAB, and > the DB commands can be used to migrate them. > > These commands, namely, "airflow fab-db migrate", "airflow fab-db > downgrade", and "airflow fab-db reset," have arguments similar to those of > the equivalent Airflow DB commands. > > Using FAB will be different from using every other provider, especially > when using FAB becomes optional. > > As part of the AIP, I introduced the configuration "[database] > external_db_managers" to include the DB manager of any provider or plugin > that wants to run database migration during the Airflow migration process. > To use the FAB provider and have airflow run the migration command for you, > you must ensure that the FABDBManager is added to "[database] > external_db_managers". This way, when "airflow db migrate "or "airflow db > reset" commands are run, airflow will run the migrate or reset command for > the FAB provider. This is optional. If the FABDBManager is not added to the > config, you can manually manage the FAB DB migration using the provided > commands. > > “airflow db migrate” command does not run the FAB migrate command if the > upgrade is half-way and not to the heads. > > "airflow db downgrade" does not run FAB/plugins DB downgrade. The user > will have to manually use the DB commands in FAB to run the downgrade. > Downgrading is different since we don't know the version of FAB the user > wants to downgrade to. > > These are what to expect, let me know what you think. > > Thanks > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
