Hello here,
As discussed last week on slack, I prepared a comprehensive document
describing the current state of extras in Airflow (or at least my view on
it :) ).
https://docs.google.com/document/d/1e1T3u34bqo7yYSJVvq1NlEJJhA5_hUNXOXo7J6KFg_0/edit?usp=sharing
The document describes the current
"separate POM" -> Separate pyproject.toml :D,, I've been in too many
discussions in Apache Software Foundation about java dependencies and POM
files and Maven (ough!) it seems :D
On Thu, Oct 17, 2024 at 12:22 PM Jarek Potiuk wrote:
> Hello here,
>
> As discussed last week on slack, I prepared a
Also one more comment. I added this paragraph (and highlighted it):
> The approach to extras depends very much on the packaging decisions we
will make before releasing Airflow 3. Currently we only have one
“apache-airflow” package and all extras are defined there. In Airflow 3 we
will have more pa
Hi Jarek, Everyone,
Thanks for starting this discussion!
I agree with everyone so far that this will be more of an additional
executor rather than a replacement for
anything we currently have.
I had submitted a talk that was mainly trying to explain about how we can
leverage some features of Yuni
Also one more thing added - I realized I have forgotten one more thing -
pre-installed providers. Those are automatically added to 'requirements" in
"standard" installation, and in "editable" mode - their dependencies are
added instead (not the providers themselves). This is an important case for
b
Hey All,
Thanks to those who provided feedback! As a reminder we will be sending the
survey out next Monday, Oct. 21st, and would love for the folks here to
help promote it to the Airflow community!
On Tue, Oct 8, 2024 at 1:56 PM Briana Okyere
wrote:
> Hey All,
>
> It's that time of year again!
I love the idea. Generally it is quite easy now to add new executors and there
is no harm in having more options. I don't think we need to justify it as a
replacement of anything honestly.
The biggest decision is whether this is a community managed executor or if we
can find stakeholders to cre
> As Jens said "K8sExecutor++".
> Just to be precise, I don't believe that this can be a replacement for
Celery Executor (at least at first glance).
Yes. Fully agree. My bad framing from the initial message.
> I also believe that for this to be effective, this will need some
dedicated work includ
Oh. I don't think we want to "vote" on it (but I will let David to chime in
because I was mostly guessing what's his expectation and worries are).
On Thu, Oct 17, 2024 at 1:07 AM Vikram Koka
wrote:
> Hmm, I can think of a different solution to the problem here as well, but I
> could be misunder
