> Provider package name: "edge", so gets to package "airflow.providers.edge"
I'm not so sure about this name. I have no problems with the"edge" part, I am mostly questioning if this should be a provider, or should it be another kind of module. For instance it won't have an Operator, Sensor or Hook, nor need anything else provided by the Provider Manager. Similarly now I think about it for Celery executor, it's not got anything for use in dags does it? My thinking here is: if a dag author uses it directly -> "provider" If it's only used by the deployment manager -> something else, possibly "plugin"? Things that I think should fall in this second category would be the FAB, Celery and Edge executors. cncf.kubernetes is mixed as it has the executor and an operator. What do others think? On 8 September 2024 14:05:51 BST, Jens Scheffler <j_scheff...@gmx.de.INVALID> wrote: >Hi Devs, > >as we had a naming discussion for AIP-69 in >https://lists.apache.org/thread/br1jfoc8p1wjzk74c09srjgr29spytfy and PRs >are ready, Elad proposed to have at least a lazy consensus to close the >naming before merge. > >Not having real counts leave "Distributed" and "Edge" close-by with most >positive feedback. As nobody objected and the term "Edge" is shorter... >That means: > >- Provider package name: "edge", so gets to package "airflow.providers.edge" > >- Executor class: "EdgeExecutor" > >- Remotely running worker name: "Edge Worker" - called via `airflow edge >worker [options]` (similar like celery) > > >If somebody wants to have a look to the implementation, several PRs are >split for better review: > >- "Mothership" PR with the complete implementation if you want to have a >test drive: https://github.com/apache/airflow/pull/40900 (=3800 LoC) > > - Split 1: (Full) Edge provider package: >https://github.com/apache/airflow/pull/41729 (3700 LoC) > > - Split 1.1: (Empty) Provider package (Empty boilerplate) >https://github.com/apache/airflow/pull/42046 (450 LoC) > > - Split 1.2: DB Models for Edge Provider >https://github.com/apache/airflow/pull/42047 (1.1 + 950 LoC) > > - Split 1.3: EdgeExecutor >https://github.com/apache/airflow/pull/42048 (1.1+1.2 + 650 LoC) > > - Split 1.4: Rest API Endpoint >https://github.com/apache/airflow/pull/42049 (1.1+1.2 + 1050 LoC) > > - Split 1.5: Worker CLI >https://github.com/apache/airflow/pull/42050 (1.1+1.2 + 650 LoC) > > - Split 1.6: Leftover glue to bring all together >https://github.com/apache/airflow/pull/42051 (1.1-1.5 + 25 LoC) > > - Split 2: (Small) Adjustments in Core: >https://github.com/apache/airflow/pull/41730 (<10 LoC) > > - Split 3: Breeze adjustments to develop/start via CLI >https://github.com/apache/airflow/pull/41731 (<200 LoC) > > >If anyone objects to the consensus here, let me/devlist know - otherwise this >will be effective by Thursday 12th of September 2024 18.00 am PST (which >coincidentally is when >Airflow Summit is completed in San Francisco) - Also looking forward for >approvals on the codebase / PRs. PRs are ready to have something working. > >Jens > > >--------------------------------------------------------------------- >To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >For additional commands, e-mail: dev-h...@airflow.apache.org >
