Cool :) On Sun, Oct 13, 2024 at 1:55 AM Vikram Koka <vik...@astronomer.io.invalid> wrote:
> Jarek, > > Absolutely intend to have you and Elad be co-authors on the proposed > solution here. > > Jens has done a great job of identifying some of the early parts of the > solution as part of the diagrams which we discussed in the last two dev > calls. There are a few core elements to this problem and clearly one > dimension i.e. Extra Links and Connection Extra Fields were the focus of > the discussion earlier this week. > > I have been thinking about this in the context of making it easier for end > users to upgrade Airflow Versions and this is one of the key benefits of > AIP-72 and the separation between the Task SDK and Airflow server > components. One of the other dimensions which I had been thinking about was > the handling of providers for Logging, Secrets, and Notifications. > > Because of the different dimensions here, it is best to have a write-up. I > started a draft proposal which includes the motivation (as I understand > it), the different dimensions to be solved, and some early thoughts around > the solutions. I will share that with you by early next week. > > Clearly, both you and Elad are critical in thinking this through and > looking forward to working together on this. > > Best regards, > Vikram > > > On Sat, Oct 12, 2024 at 12:26 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > Thanks Ash and Jens. I see that we have a doc to write (and Vikram agreed > > to do it) - Vikram, Jens - I am happy to join the efforts and have the > > first stab and maybe even lead on implementing that part if no-one else > > signed up already. I can lead preparation of such a doc or simply be > added > > as co-author :). > > > > Just a comment here - I strongly support the split and I think it's very > > much worth it to attempt the "extra work" for 3.0 (it does not > necessarily > > have to be done for 3.0, but if we can bundle it with everything else in > > 3.0 it will be easier to introduce as part of the "3.0 breaking changes". > > And a bit of consequence of that (which I will add in the doc) - is the > > need to publish more variants of reference images for our users to use > (but > > this is all totally doable and automatable). > > > > The main reason (and why it's good I might want to take a lead) - it very > > much falls into my main focus area. Such change significantly improves > the > > overall security properties of Airflow installation. This was one of the > > learnings so far from the "Airflow Beach Cleaning" exercise - that > > generally supply chain is the next "frontier" for Airflow (and generally > > any other project) - and if we could decrease the number of dependencies > > for some of the sensitive sub-components of Airflow (which > > internal-api/webserver certainly is) - the security risks connected with > > supply chain problems are generally somewhere between quadratically and > > exponentially depend on the number of dependencies of those components. > > > > J. > > > > On Sat, Oct 12, 2024 at 7:07 PM Ash Berlin-Taylor <a...@apache.org> > wrote: > > > > > Nothing much more was covered about splitting packages this time, I > think > > > it happened in the previous meeting, so that might be the best one to > > watch > > > back. All we covered on Thursday was talking about how not installing > > > providers in the web server would need to work (i.e. ExtraLinks and > > > connection field strcuture/metadata to display the Add Connection form > > > would have to be persisted in the DB most likely) > > > > > > > > > -ash > > > > > > > > > > On 12 Oct 2024, at 07:50, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > > > > Is there a recording available for that dev call? I would very much > > like > > > to > > > > hear especially the discussion about splitting packages as I think > this > > > is > > > > quite an important topic to discuss and I have quite a bit of > > experience > > > > there that might be useful to comment on :). > > > > > > > > I could not attend the meeting because I was at Community Over Code > > (and > > > > actually that was exactly conflicting with "State of The Foundation" > > talk > > > > that Dave Nalley, the president of the ASF opened the last day of the > > > > conference with - so I had to skip it ). > > > > > > > > BTW. Dave mentioned Airflow, security, and particularly pointed at > and > > > > mentioned me personally - underlying the efforts and trust we built > > > between > > > > us personally while working on the security initiatives - during the > > > > opening keynote :D, so it's good I was there. > > > > > > > > J. > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > >
