And just to clarify: The voting will last till Tuesday 25 July 20023 11am CST.
On Fri, Jul 21, 2023 at 10:43 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > Clarification: Ash is right (I totally forgot about it). > > We do have dask in > https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html#core-airflow-extras > and it even explicitly mentions it: > > * "dask pip install 'apache-airflow[dask]' DaskExecutor" > > So technically speaking you **should** use the dask extra to install > distributed, dask and cloudpickle package - all needed to run the dask > executor. And we will keep it this way - it will just move from "core > extras" to "providers". Actually it will move to "deprecated" and we > will alias "dask" it with the new "daskexecutor", because the name > "apache-airflow-providers-dask" is not allowed by PyPI (I guess the > -dask suffix is considered potentially harmful for typosquatting) so I > reserved "apache-airflow-providers-daskexecutor". > > J. > > > > On Fri, Jul 21, 2023 at 10:34 AM Hussein Awala <huss...@awala.fr> wrote: > > > > > based on the premise that your had to install the `dask` extra in the > > first place to get dask module of the right version > > > > After reviewing the Dask Executor > > <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/dask.html> > > section in Airflow documentation, I didn't find any requirement or > > recommendation to add the `dask` extra when using it. So, including the > > dependencies explicitly in the requirements file should suffice to use the > > executor. This means that this change might potentially break existing > > setups for some users when they upgrade Airflow version, which contradicts > > the Airflow deprecation policy. > > > > If the premise holds true for Dask, it should also apply to Celery and > > Kubernetes. Nevertheless, we decided to pre-install their providers after > > the migration. > > > > However, this executor isn't as popular as the others, and as an Airflow > > user, I prefer not to have Dask dependencies in my environment when I > > install Airflow with the Celery executor. This helps avoid conflicts and > > enables me to upgrade libraries independently from their constraints. One > > possible solution could be allowing users to instruct Airflow not to > > pre-install executors providers by passing an argument via > > `--install-option` during pip install, and add install them explicitly by > > providing the extras. > > > > Since the flexibility of our deprecation policy isn't entirely clear to me, > > I'm neutral on this matter, with a vote of +0. > > > > On Fri, Jul 21, 2023 at 9:10 AM Elad Kalif <elad...@apache.org> wrote: > > > > > I agree with Ash > > > -1 as well. > > > > > > > > > On Fri, Jul 21, 2023 at 9:29 AM Ash Berlin-Taylor <a...@apache.org> wrote: > > > > > > > -1 - based on the premise that your had to install the `dask` extra in > > > the > > > > first place to get dask module of the right version, so if we make the > > > > existing extra depends on the new provider then it's good enough. > > > > > > > > On 21 July 2023 06:22:28 BST, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > >Q: Do we want to pre-install Dask provider in Airflow 2.7.0 (with Dask > > > > >executor) once separated ? > > > > > > > > > >Discussion was here: > > > > >https://lists.apache.org/thread/0d9x4kl7hc2qzvho2mbdf35ohn5w12l6 > > > > > > > > > >Please vote: > > > > > > > > > >* +1 -> yes, we want to have dask provider preinstalled > > > > >* -1 -> no, it's fine to make it optional > > > > >* 0 -> no opinion > > > > > > > > > >Consider it my -1: I think we should NOT preinstall Dask provider. > > > > > > > > > >Voting guidelines here. This is really a "procedural" matter rather > > > > >than code modification: > > > > > > > > > >> Votes on procedural issues follow the common format of majority rule > > > > unless otherwise stated. That is, if there are more favourable votes > > > > than > > > > unfavourable ones, the issue is considered to have passed -- regardless > > > of > > > > the number of votes in each category. (If the number of votes seems too > > > > small to be representative of a community consensus, the issue is > > > typically > > > > not pursued. > > > > > > > > > >In this case committers have binding votes but other community members > > > > >are encouraged to state their non-binding votes as well. > > > > > > > > > >https://www.apache.org/foundation/voting.html > > > > > > > > > >--------------------------------------------------------------------- > > > > >To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > > >For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
