Re: [DISCUSS] Capturing Architectural decisions (ADRS?)

I like this idea! It would make it easier to follow why these decisions were made. But if we are to adopt this idea, we might need to decide which decisions we want to include in these ADRs. Or maybe we could include a checkbox on the PR template for the committers to decide whether to ask the a

Re: [PROPOSAL] Security team rotation introduction to our process

I really like the idea of having a rotation support system for the security team. It is rather important for a couple of members to have this skill as their forte because security is generally not a common area of expertise in my experience as an engineer. It is often looked at as "boring" or "har

Re: [DISCUSS] Common.sql interface / API

Thanks for taking this up, Jarek. I left a few comments on the pull request already. Thanks & Regards, Amogh Desai On Mon, Dec 4, 2023 at 5:24 AM Jarek Potiuk wrote: > Hey here, > > In parallel to the serialisation discussion, I also wanted to make sure we. > raise the topic of common.io. It's

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Congratulations, Utkarsh! Best, Wei > On Dec 5, 2023, at 5:58 AM, Oliveira, Niko > wrote: > > Congrats! Very well deserved! > > > From: Pankaj Koti > Sent: Monday, December 4, 2023 11:28:41 AM > To: dev@airflow.apache.org > Subject: RE: [EXTERNAL] [COURRIEL E

Re: [DISCUSS] Capturing Architectural decisions (ADRS?)

> > I love this idea! > > Another option, that I don't think we as a community are very good at, is > putting the context of the change in the git commit message itself. Those > messages are already tightly associated into git history and the code > itself via blame without needing to introduce an

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Congrats! Very well deserved! From: Pankaj Koti Sent: Monday, December 4, 2023 11:28:41 AM To: dev@airflow.apache.org Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [ANNOUNCE] New committer: Utkarsh Sharma CAUTION: This email originated from outside of the organizat

Re: [DISCUSS] Capturing Architectural decisions (ADRS?)

I love this idea! Another option, that I don't think we as a community are very good at, is putting the context of the change in the git commit message itself. Those messages are already tightly associated into git history and the code itself via blame without needing to introduce an new concep

Re: Helm Chart Version 1.9.0 Unavailable

You should follow the download page - https://airflow.apache.org/docs/helm-chart/1.9.0/installing-helm-chart-from-sources.html - not downloads* link. It will forward you to the direct location. Per ASF policies - downloads.apache.org contains generally only latest maintained releases. See the des

Helm Chart Version 1.9.0 Unavailable

Hi Team! First, thanks for all the work that you do in building and maintaining the project/helm/goodies! I’m working on deploying version 1.9.0 of the Helm chart, but the helm chart @ https://downloads.apache.org/airflow/helm-chart/1.9.0/airflow-1.9.0.tgz is currently unavailable. Is this e

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Great news, many congratulations Utkarsh 🕺🥳✌🏾 On Tue, 5 Dec 2023, 00:53 Pierre Jeambrun, wrote: > Congratulations! > > Le lun. 4 déc. 2023 à 20:18, Andrey Anshin a > écrit : > > > Congrats! 🔥💯 > > > > > > > > > > On Mon, 4 Dec 2023 at 22:45, Amogh Desai > wrote: > > > > > This is fantastic! Co

Re: [PROPOSAL] Security team rotation introduction to our process

Just to add on it - yeah - formal "shadowing" is not needed, we are friendly and welcoming and I can't imagine the rotation will be "big" - I think quite a few past team members will have to stay in the team every rotation to make sure we keep the experience :). Maybe just to add what security tea

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Congrats! On 2023/12/04 16:43:09 Jarek Potiuk wrote: > Hello everyone, > > [filling-in for Kaxil who is less available nowadays and travelling without > much access to computer] > > The Project Management Committee (PMC) for Apache Airflow > has invited Utkarsh Sharma to become a committer and w

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Congratulations! Le lun. 4 déc. 2023 à 20:18, Andrey Anshin a écrit : > Congrats! 🔥💯 > > > > > On Mon, 4 Dec 2023 at 22:45, Amogh Desai wrote: > > > This is fantastic! Congratulations Utkarsh. > > > > Well deserved! > > > > Thanks & Best Regards, > > Amogh Desai > > > > > > On Mon, 4 Dec 2023 a

Re: [DISCUSS] Capturing Architectural decisions (ADRS?)

I love this idea. I definitely think it can improve a lot the knowledge sharing across Airflow. Given the history and the number of components in Airflow, it is hard to keep up with everything, so having these ADRs would help a lot I think! On 2023/12/03 23:57:11 Jarek Potiuk wrote: > Hey every

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Congrats! 🔥💯 On Mon, 4 Dec 2023 at 22:45, Amogh Desai wrote: > This is fantastic! Congratulations Utkarsh. > > Well deserved! > > Thanks & Best Regards, > Amogh Desai > > > On Mon, 4 Dec 2023 at 10:56 PM, Pankaj Singh > wrote: > > > Congratulations Utkarsh 🎉 > > > > On Mon, Dec 4, 2023 at 10

Re: [DISCUSS] Allowlist in serializatin [was: Serialization in Apache Airflow]

> Pickle and the likes can execute arbitrary code that is inside the > serialized object. > > Yep. This is super dangerous indeed. My fifty cents. This sounds scarier than it actually is, it mostly covered by this simple things: - Do not open attachment from unknown senders / untrusted sources -

[DISCUSS] Using serde for PyODBC case/other cases [was Serialization in Apache Airflow]

Hey here, Also separating it out as another topic. > > It is a choice that needs to be made at the API level - here Common.sql. In > this case the particular form of the tuple was, somewhat implicit, part of > the spec. > Yep. I rectified the implicitness by storing this as an explicit decision

Re: [ANNOUNCE] New committer: Utkarsh Sharma

This is fantastic! Congratulations Utkarsh. Well deserved! Thanks & Best Regards, Amogh Desai On Mon, 4 Dec 2023 at 10:56 PM, Pankaj Singh wrote: > Congratulations Utkarsh 🎉 > > On Mon, Dec 4, 2023 at 10:45 PM Aritra Basu > wrote: > > > Amazing Utkarsh, congratulations! > > > > -- > > Regard

Re: [PROPOSAL] Security team rotation introduction to our process

I don't know that we'd need to make the shadow period too formal, we all come from diverse backgrounds.One of the reasons I didn't step up for the "full-time" position is that I have no real background in the security side of things and I didn't want to be a drain. But I'd consider a rotat

[DISCUSS] Allowlist in serializatin [was: Serialization in Apache Airflow]

Let me start separate threads from the main discussion. It will be easier to follow. I'd really like to understand the actors involved and what kind of threats the allowlist is supposed to protect us. I think it would be great if we see the "attack" scenario we think allowlist will protect us aga

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Congratulations Utkarsh 🎉 On Mon, Dec 4, 2023 at 10:45 PM Aritra Basu wrote: > Amazing Utkarsh, congratulations! > > -- > Regards, > Aritra Basu > > On Mon, Dec 4, 2023, 10:27 PM Phani Kumar .invalid> > wrote: > > > Congratulations Utkarsh ! Well deserved > > > > On Mon, Dec 4, 2023 at 10:13 PM

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Amazing Utkarsh, congratulations! -- Regards, Aritra Basu On Mon, Dec 4, 2023, 10:27 PM Phani Kumar wrote: > Congratulations Utkarsh ! Well deserved > > On Mon, Dec 4, 2023 at 10:13 PM Jarek Potiuk wrote: > > > Hello everyone, > > > > [filling-in for Kaxil who is less available nowadays and tr

Apache Airflow Newsletter | November 2023

IF YOU WOULD LIKE TO SEE THE FORMATTED VERSION OF THIS NEWSLETTER, PLEASE GO HERE Hey Team, In November, the Airflow community was busy finalizing the release of Airflow 2.8 (TBA), in addition to enjoying themselves at AWS re:Invent,

Re: [ANNOUNCE] New committer: Utkarsh Sharma

Congratulations Utkarsh ! Well deserved On Mon, Dec 4, 2023 at 10:13 PM Jarek Potiuk wrote: > Hello everyone, > > [filling-in for Kaxil who is less available nowadays and travelling without > much access to computer] > > The Project Management Committee (PMC) for Apache Airflow > has invited Utk

[ANNOUNCE] New committer: Utkarsh Sharma

Hello everyone, [filling-in for Kaxil who is less available nowadays and travelling without much access to computer] The Project Management Committee (PMC) for Apache Airflow has invited Utkarsh Sharma to become a committer and we are pleased to announce that they have accepted. Utkarsh had been

Re: [DISCUSS] Capturing Architectural decisions (ADRS?)

I am in support of having adrs, and perhaps it's be helpful if there can be a brief description on what kind of changes would need an adr. Also if we can have a reminder on precommit which tells a committer to add an adr if a change fits some criteria (lines of code change or any other kind of metr

Re: [PROPOSAL] Security team rotation introduction to our process

I think overall it is a great idea to slowly bring in more people into rotation. It should help with adding redundancy and help prevent burnout for the people who are doing it now. I would propose perhaps a gradual introduction via a brief shadow period where a new member would monitor the happeni

[PROPOSAL] Security team rotation introduction to our process

Hello everyone, *TL;DR; *I have a proposal of refinements we can apply to our security team and I am looking for comments and feedback (PR is out there in [1]). In short I am proposing that we introduce rotation of the security team members, so that we can avoid burnout, give a chance to others to

Re: [DISCUSS] Serialization in Apache Airflow: Clarifications and Future Directions

On Mon, 4 Dec 2023 at 02:10, Jarek Potiuk wrote: > > > > > > I am reaching out to initiate a discussion on a topic that has been > causing > > some confusion within the community: serialization in Apache Airflow. The > > purpose of this email is to shed light on the current state of > > serializa