Re: [VOTE] February PR of the Month

+1 for 27758On Feb 25, 2023, at 7:53 PM, Collin McNulty wrote:I’ll add my voice to the 27758 chorus. +1On Fri, Feb 24, 2023 at 6:09 PM Hussein Awala wrote: +1 to #27758 too! From: Vikram Koka Sent: Saturday, February 25, 2023 12:21:40 AM To: dev@airflow.apache.org Subj

Re: [VOTE] August 2023 PR of the Month

@eumiro deserves his own shoutout for the refactor work! That said, my vote is also #30259. > On Aug 28, 2023, at 6:53 PM, Hussein Awala wrote: > > I agree with Jed and Jarek on the great work by @eumiro, but my vote is for > #30259. > >> On Mon, Aug 28, 2023 at 10:49 PM Jarek Potiuk wrote:

Re: [RESULT][VOTE] PR of the Month for August Airflow Newsletter

+1 for special mention in newsletter. Fantastic idea Briana! > On Aug 30, 2023, at 3:59 PM, Briana Okyere > wrote: > > Hey All, > > Thanks for your engagement with our PR of the Month Vote for the > August Newsletter. > Lots of great PRs this month! > > The results are in at: > > + 7 to #30

Re: Lazy Consensus - Removing the Experimental tag for Pluggy

What’s the motivation to remove this now? > On Sep 8, 2023, at 2:55 AM, Cody Rich wrote: > > Hi Everyone, > > I'm calling a lazy consensus for removing the experimental tag for the > pluggy interface (pr #34174 ). > It's currently denoted as experi

Re: Lazy Consensus - Removing the Experimental tag for Pluggy

Oh! Yes I agree > and has been since it’s release in 2.6 Misunderstood. Thought it was only released a few months ago. Time flies! +1 (non-binding) > On Sep 8, 2023, at 3:44 PM, Hussein Awala wrote: > >  >> >> What’s the motivation to remove this now? > > The feature was introduced in Ai

Re: [VOTE] December PR of the Month

+1 for 35719 > On Jan 2, 2024, at 11:14 PM, Amogh Desai wrote: > > My vote goes to #35719. Having a nice UI fix come in, that too from a first > time contributor > makes it a winner for me! > > Thanks & Regards, > Amogh Desai > >> On Wed, Jan 3, 2024 at 7:51 AM Wei Lee wrote: >> >> I want t

Re: [ANNOUNCE] New committer: Wei Lee

Congrats Wei! > On Apr 8, 2024, at 5:31 AM, Pankaj Singh wrote: > > Congrats Wei, very well deserved! > >> On Mon, Apr 8, 2024 at 2:33 PM Rahul Vats wrote: >> >> Congrats Wei, very well deserved! >> >> Regards, >> Rahul Vats >> >> >>> On Mon, 8 Apr 2024 at 14:27, Amogh Desai wrote: >>>

Re: K8s version/support policy: as long as K8s project, or as long as cloud providers?

+1 > On Mar 7, 2022, at 7:08 PM, Vikram Koka wrote: > >  > +1 > >> On Mon, Mar 7, 2022 at 3:01 PM Kaxil Naik wrote: >> +1 >> >>> On Mon, 7 Mar 2022 at 21:17, Jarek Potiuk wrote: >>> +1 >>> On Mon, Mar 7, 2022 at 10:12 PM Ash Berlin-Taylor wrote: Hey everyone, So Kube

Re: [VOTE] AIP-48 Data Driven Scheduling

+1 (non-binding) > On Jun 4, 2022, at 7:45 PM, Igor Kholopov > wrote: > >  > +1 (non-binding) > Left some comments with my thoughts on the AIP wiki page. > >> On Fri, Jun 3, 2022 at 3:11 PM Ankit Chaurasia wrote: >> +1 (non-binding) >> >> Ankit Chaurasia >> HomePage | LinkedIn | +91-99873

Re: [DISCUSSION] Support policy for Airflow 2.x after the Airflow 3.0.0 release

Doing some back of envelope math: 6/12 (or 6+6) gets you to April 2026 (October 2025 for bug fixes). If we follow the pre-3.0 release cadence, we will release 3.1 beginning of August and 3.2 in December. Assuming the policy goes through, by the time we stop patching 2.10 with bug fixes, 2.10 wi

Re: [DISCUSS] confusing alert re SimpleAuthManager

Sounds great! Do we have something in the config linter to highlight this change? > On Mar 20, 2025, at 11:19 PM, Daniel Standish > wrote: > > It says this: > > Development-only auth manager configured > The auth manager configured in your environment is the Simple Auth Manager, > which is i

Re: [VOTE] June 2025 PR of the Month

Hard choice but +1 for 51153. Docs improvements are very needed. Massive shoutout to all the internationalization work!! Wavered between the two as that is also very needed > On Jun 27, 2025, at 8:37 AM, Rahul Vats wrote: > > My vote goes to #51153 as well. Great doc work. > > Regards, > Rah

Re: [VOTE] August 2022 PR of the Month

r the local Quick Start in docs >>>> https://github.com/apache/airflow/pull/25888 >>>> >>>> [25788] Properly check the existence of missing mapped TIs >>>> https://github.com/apache/airflow/pull/25788 >>>> >>>> [25610] Grid logs for mapped instances >>>> https://github.com/apache/airflow/pull/25610 >>>> >>>> [25857] Add `RedshiftCreateClusterSnapshotOperator` >>>> https://github.com/apache/airflow/pull/25857 >>>> >>>> -- > > Collin McNulty > Lead Airflow Engineer > > Email: col...@astronomer.io > Time zone: US Central (CST UTC-6 / CDT UTC-5) > > > <https://www.astronomer.io/> > -- Constance Martineau Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [VOTE] September 2022 PR of the Month

, Daniel Standish < > daniel.stand...@astronomer.io.INVALID> wrote: > > One vote for https://github.com/apache/airflow/pull/26400 (improved test > command) > > On Tue, Sep 27, 2022 at 10:50 AM Jed Cunningham > wrote: > >> My write-in is ExternalPythonOperator:

Re: [VOTE] November 2022 PR of the Month

ttps://github.com/apache/airflow/pull/27540 >> >> >> >> [27597] Add max_wait for exponential_backoff in BaseSensor >> >> https://github.com/apache/airflow/pull/27597 >> >> >> >> [27506] Fix mini scheduler expansion of mapped task >> >> https://github.com/apache/airflow/pull/27506 >> >> >> >> [27526] Clean backcompat code kpo >> >> https://github.com/apache/airflow/pull/27526 >> >> >> >> Regards, >> >> John >> > >> > >> > -- Constance Martineau Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [VOTE] AIP-50 Trigger DAG UI Extension with Flexible User Form Concept

gistergericht: Amtsgericht Stuttgart, HRB 14000; > Aufsichtsratsvorsitzender: Prof. Dr. Stefan Asenkerschbaumer; > Geschäftsführung: Dr. Stefan Hartung, > Dr. Christian Fischer, Filiz Albrecht, Dr. Markus Forschner, Dr. Markus > Heyn, Rolf Najork > ​ > -- Constance Martineau Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [VOTE] (extended) on AIP-50 (part 2) to finalize it

Hello, I think Option B is reasonable. +1 for B, non-binding. Constance On Mon, May 22, 2023 at 4:54 PM Scheffler Jens (XC-DX/ETV5) wrote: > Hi Airflow-Developers, > > It is not democracy if nobody makes a vote. I don't want to be a > "dictator" but still pr

Re: [VOTE] October 2023 PR of the Month

Hi Briana, My vote is for 34784. I love that we are resolving some inconsistencies within the AWS provider package Constance On Mon, Oct 30, 2023 at 1:19 PM Briana Okyere wrote: > Hey All, > > It’s once again time to vote for the PR of the Month. > > Please note that if th

Re: [VOTE] October 2023 PR of the Month

Tue, Oct 31, 2023 at 6:00 PM Jed Cunningham > wrote: > > > The new OpenSearch provider gets my vote - 34705. > > > -- Constance Martineau Senior Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [VOTE] New Airflow Community Provider: Teradata

tests/providers/teradata > > > > > > > > System Tests: > > > > > > > > https://github.com/Teradata/airflow/tree/td_develop/tests/system/providers/teradata > > > > System Tests Dashboard: https://teradata.github.io/airflow/ > > > > > > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > > -- Constance Martineau Senior Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [VOTE] January 2024 PR of the Month

or a future issue of the newsletter, please drop me a line at < > briana.oky...@astronomer.io>. > > -- > Briana Okyere > Community Manager > *Astronomer* > -- Constance Martineau Senior Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

t; >> > > >> these tenants, we currently have a complex setup: > >> > > >> > >> > > >>1. Containers run on a schedule to export metadata to CosmosDB > >> > (these > >> > > >>will be replaced by the listener). > >> > > >>2. Additional scheduled containers pull data from CosmosDB and > >> > write > >> > > >>it to a shared file system, enabling generated DAGS to read it > >> and > >> > > mirror a > >> > > >>dataset across tenants. > >> > > >> > >> > > >> > >> > > >> Proposed Workflow > >> > > >> Here's a breakdown of our proposed workflow: > >> > > >> > >> > > >>1. Cross-Tenant Dataset Interaction: We have Dags in Tenant 1 > >> > > >>producing Dataset A. We need a mechanism to trigger all Dags > >> > > consuming > >> > > >>Dataset A in Tenant 2. This interaction is crucial for our > data > >> > > pipeline's > >> > > >>efficiency and synchronicity. > >> > > >>2. Dataset Listener Implementation: Our approach involves > >> > > >>implementing a Dataset listener that programmatically creates > >> > > Dataset A in > >> > > >>all tenants where it's not present (like Tenant 2) and export > >> > Dataset > >> > > >>updates when they happen. This would trigger an update on all > >> Dags > >> > > >>consuming from that Dataset. > >> > > >>3. Standardized Dataset Names: We plan to use standardized > >> dataset > >> > > >>names, which makes sense since a URI is its identifier and > >> > > uniqueness is a > >> > > >>logical requirement. > >> > > >> > >> > > >> [image: image.png] > >> > > >> > >> > > >> Why This Matters: > >> > > >> > >> > > >>- It offers a streamlined, automated way to manage datasets > >> across > >> > > >>different Airflow instances. > >> > > >>- It aligns with a need for efficient, interconnected > workflows > >> in > >> > a > >> > > >>multi-tenant environment. > >> > > >> > >> > > >> > >> > > >> I invite the community to discuss: > >> > > >> > >> > > >>- Are there alternative methods within Airflow's current > >> framework > >> > > >>that could achieve similar goals? > >> > > >>- Any insights or experiences that could inform our approach? > >> > > >> > >> > > >> Your feedback and suggestions are invaluable, and I look forward > >> to a > >> > > >> collaborative discussion. > >> > > >> > >> > > >> Best Regards, > >> > > >> Eduardo Nicastro > >> > > >> > >> > > > > >> > > > >> > > >> > > > -- Constance Martineau Senior Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

it to be a task output as part of a dag. The only valid reason to now allow it IMHO is because they were designed to be defined within a dag file, similar to a dag, and we don't want to deal with the impediment I laid out. On Wed, Jan 24, 2024 at 12:45 PM Jarek Potiuk wrote: > On Wed, Jan

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

ating Airflow) my actual > experience with Datasets is limited, I've been mainly observing what was > going on, so I would love to hear what those who created (and continue to > think about future of) the datasets :). > > J, > > On Wed, Jan 24, 2024 at 7:27 PM Constance Ma

Re: [PROPOSE] Add A Code of Conduct for Slack and Meetups

Wow Briana! This is fantastic, what a great idea! I added a few comments. I also had a similar question as Jarek that I think merits a discussion: Should we have a committee or group to handle reported guideline violations? If we single out one person to report violations to, we'll have to continu

Re: [PROPOSE] Add A Code of Conduct for Slack and Meetups

t; we can ensure folks can anonymously submit a post as "breaking the > guidelines" if they cannot DM an individual. > > On Fri, Jan 26, 2024 at 10:16 AM Constance Martineau > wrote: > > > Wow Briana! This is fantastic, what a great idea! I added a few comments. > &

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

ed to think carefully about how API-created > datasets would coexist with those defined in DAG files. However, in my > opinion, this is a natural transition that will likely become necessary as > Airflow is used in increasingly diverse environments and organizations, a > shift that seems

Re: Idea for Discussion: custom TI dependencies

Naive question: Instead of running the code on the scheduler - could the condition check be delegated to the triggerer? On Fri, Feb 2, 2024 at 2:33 PM Pierre Jeambrun wrote: > But maybe it’s time to reconsider that :), curious to see what others > think. > > On Fri 2 Feb 2024 at 20:30, Pierre Je

Re: Idea for Discussion: custom TI dependencies

licy/task_instance_mutation_hook/pod_mutation_hook. > > So I do not think this violates our current preference in terms of > security. > > > Hi Constance, > > I thought the trigger is mainly for deferrable operator cases? It’s quite > different scenario from what I’m trying to

Re: [DISCUSS] Rename channels on slack

+1 for #contributing and leaving #troubleshooting. Shorter names in slack are nice where possible. No strong opinion on the actual names. Agree that #development needs to be renamed to something more obvious though. On Thu, Feb 8, 2024 at 9:30 AM Vincent Beck wrote: > I am +1 in renaming these

Re: [DISCUSS] Rename channels on slack

> Maybe we restrict who can post in development for a period of time with a message directing folks to the right places? As long as we don't make it committer only. If you're contributing something and want some help/feedback, it's not welcoming to find out that you're to be restricted from the de

Re: [HUGE DISCUSSION] Airflow3 and tactical (Airflow 2) vs strategic (Airflow 3) approach

at it continues to remain relevant for the next ten years. Constance On Mon, May 6, 2024 at 2:10 PM Ash Berlin-Taylor wrote: > There's a lot of technical debt hiding in Airflow, especially the > scheduler that makes it harder and harder to efficiently add new features. > > At some po

Re: [HUGE DISCUSSION] Airflow3 and tactical (Airflow 2) vs strategic (Airflow 3) approach

ns on AIP-67 and others change > if we consider Airflow 3 is "starting". > > The price for back-compat is speed of development and quality. More > combinations to test, more unexpected issues uncovered, necessity to keep > parallel paths (old/new) while adding new features

Re: [VOTE] Proposal for adding Telemetry via Scarf

+1 non-binding On Thu, May 9, 2024 at 7:17 AM Tomasz Urbaszek wrote: > +1 binding > > On Thu, 9 May 2024 at 12:40, Andrey Anshin > wrote: > > > +1 binding > > > > > > > > > > > > On Thu, 9 May 2024 at 13:25, Wei Lee wrote: > > > > > Got it. Thanks Jarek for pointing out! > > > > > > Best, > >

Re: [DISCUSS] AIP-63, AIP-64, and AIP-65: DAG Versioning

Agreed. When Jed and team wrote the AIP, we intentionally limited the scope to DAGs since the AIPs were already really large, but the intention is to extend the concept to datasets. Funny that you bring up point #2. A few of us met last week to talk about DAG Versioning, and that use-case came up

Re: [VOTE] May 2024 PR of the Month

While I know that #39336 was a lot of work and big from a dev perspective (thanks @Daniel Standish !), my vote goes to #39650 as task-level CPU and memory metrics are a long-standing feature request. On Tue, May 28, 2024 at 1:42 PM Jarek Potiuk wrote: > #39336 hands down > > On Tue, May 28, 2024

Re: Call with Nielsen team demoing their DAG debugging feature

Hi Jarek, Same :) Thanks, Constance On Mon, Jun 10, 2024 at 9:57 AM Amogh Desai wrote: > Hello Jarek, > > Please add me to the invite as well. > > Thanks & Regards, > Amogh Desai > > > On Mon, Jun 10, 2024 at 11:22 AM Abhishek Bhakat > wrote: > > >

Re: Call with Nielsen team demoing their DAG debugging feature

Hello again, Given all the enthusiasm - assuming Nielsen is ok with this - what if someone recorded the meeting so that it could be shared with those that are interested? Constance On Mon, Jun 10, 2024 at 1:48 PM Constance Martineau wrote: > Hi Jarek, > > Same :) > > Than

Proposal for Enhanced Data Awareness in Airflow

nt. We will follow up with a set of formal AIPs. Constance -- Constance Martineau Senior Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4)

Re: [DISCUSS] Number of queries to Airflow database in "DAG File Processing Stats"

I love the idea. If we were to store it in the DB, would we keep a history, or only the latest stats from the most recent dag parsing loop? DAG parsing by default is every 30s right? On Fri, Jun 14, 2024 at 6:53 AM Jarek Potiuk wrote: > > I think we still need to enable the ability for DAGs at p

Re: [DISCUSS][AIP-38 Modern Web Application]

I love it and 100% agree. Thinking "Dag Groups", where you can group dags (static & dynamic) into a subfolder. Tags are great for filtering, but they aren't a replacement for dirs especially at a large scale. We have some deployments with 20k dags and as designed today, it's not navigable at that s

Re: [DISCUSS] AIP-78 scheduler-managed backfill

Seems valid for default behaviour, but if I backfill for a year and realize there was something wrong with the code, I don't want to manually fail each dag run that is running. How about a force kill option? On Wed, Jul 10, 2024 at 9:28 AM Daniel Standish wrote: > Yup that's true @Tzu-ping Chung

Re: Naming things: What should the imports in dag files for DAG etc. be?

I'm partial to everything that we expect users to use to be importable from `airflow`, but would love to hear other people's thoughts. On Fri, Aug 30, 2024 at 5:48 AM Ash Berlin-Taylor wrote: > Hi everyone, > > It’s time to have a another discussion about everyone's favourite > discussion - nami

RE: [DISCUSS] [AIP-37] Virtualenv management inside Airflow

his work if one were using the KubernetesExecutor? Cheers, Constance / CONSTANCE MARTINEAU | Développeuse Principale, Platformes et Exploitation | Tél: 514-847-7992 | comartin...@cdpq.com -Original Message- From: Tomasz Urbaszek Sent: Friday, January 8, 2021 11:48 AM To: dev@airflow.ap

Re: [VOTE] AIP-38: Modern Web Application

+1 (non-binding) On Wed, Mar 3, 2021 at 10:31 AM Ryan Hamilton wrote: > Team, > > This email calls for a vote on the project proposed in AIP-38: > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-38+Modern+Web+Application > > Discussion thread: > > > https://lists.apache.org/thread.ht

Re: SSIS

but it is worth it. Your mileage may vary, but after a data scientist moved all his workloads to Airflow, he said that the time he spent debugging his pipelines went down 50%. Hope this helps, Constance On Tue, Jun 29, 2021 at 5:04 AM Neeku Endhuku Nenu cheppanu < idk.050...@gmail.com> wro

Re: Make first dag run optional when catchup is False

2022 at 3:00 PM Philippe Lanoe >>>> wrote: >>>> >>>> >>>> >>>> Hello Daniel, >>>> >>>> >>>> >>>> Thank you for your answer. In your example, as I experienced, the >>>> first run would n

Re: Make first dag run optional when catchup is False

the next day. The oddness is amplified when you consider a monthly >> dag, where if you deploy now, start date is now, first schedulable run is >> next month, therefore first run _more_ than a month away. To fix this I >> think we need to add support in our timetables for running

Re: Implicit DAG registration

Am intrigued. Curious about dynamic dag pattern, where you create the DAG object in a a create_dag function and adding the DAG to globals. Would this new way prevent someone from modifying the dag object within the function, or returning it? > On Apr 27, 2022, at 11:20 AM, Ferruzzi, Dennis >

Re: [Proposal] Creating DAG through the REST api

n modules where strong authentication such as >>> Kerberos can be used. >>> 2. Execute DAG code as the API identity, i.e. A DAG created through the >>> API service will have run_as_user set to be the API identity. >>> 3. To enforce data access control on DAGs, the API identity should also >>> be used to access the data warehouse. >>> >>> We shared a demo based on a prototype implementation in the summit and >>> some details are described in this ppt >>> <https://drive.google.com/file/d/1luDGvWRA-hwn2NjPoobis2SL4_UNYfcM/view>, >>> and would love to get feedback and comments from the community about this >>> initiative. >>> >>> thanks >>> Mocheng >>> >> -- Constance Martineau Product Manager Email: consta...@astronomer.io Time zone: US Eastern (EST UTC-5 / EDT UTC-4) <https://www.astronomer.io/>

Re: [DISCUSS] pools and deferrables

The main reason for pools is to control task execution parallelism, especially when tasks interact with systems like APIs or databases, to avoid overwhelming them. For deferrable operators, if a trigger is just 'sleeping' or waiting, it’s fine to exclude the task from the pool while in a deferred s

Re: [PROPOSAL] Add streaming support to PartialOperator

That was a lot to read through, and to be honest, it's hard for me to tell whether or not Jarek's proposal solves David's problem. However, if the debate is whether it's worthwhile or not to provide a first-class way for DAG authors to use Operators as part of TaskFlow Tasks, it is. Operators are

Re: Airflow should deprecate the term "DAG" for end users

In my experience, when you ask those with Airflow experience what a dag is, they'll start talking about workflow attributes - stuff like dags being a series of steps or tasks with owners. The structure doesn't come up. Echo-ing others, at this point, my vote is to embrace the name and de-emphasize

Re: [DISCUSS] Zipped dags

+1 to what Jarek has said I usually see zipped dags used in conjunction with blob storage, typically when dags are stored in multiple repos (either dags are split by use case, or teams are sharing a deployment). It's not perfect, but one of the benefits of zipping your dags is that if in zip_a I h

Re: [LAZY CONSENSUS] Remove `conf` from Task Context for AF 3.0

Will it still be possible to import and use configuration directly? For example: ``` from airflow.configuration import conf from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import KubernetesPodOperator namespace = conf.get("kubernetes", "NAMESPACE") KubernetesPodOperator( nam

Re: [DISCUSS] Mark Multi Exec Config as stable and remove old hardcoded hybrid execs

now that it’s considered stable. Curious to hear what others think! Cheers, Constance On Thu, Feb 13, 2025 at 8:16 PM Oliveira, Niko wrote: > Hello folks! > > Multiple Executor Configuration (aka hybrid executors, AIP-61) has been > out a while now and we haven't had many is

Re: [DISCUSS]: Proposing a "common message queue" abstraction

+1 Excited to see this On Thu, Feb 13, 2025 at 11:01 AM Pierre Jeambrun wrote: > +1 > > On Thu, Feb 13, 2025 at 4:36 PM Vincent Beck wrote: > > > Hi Vikram, > > > > Thanks for putting it together, I think this is great and will help users > > to use event driven scheduling in Airflow 3! > > > >

Re: Should we drop support for pre_execute & post_execute for AF 3.0

Just to add to the discussion: Removing these overrides will disproportionately impact teams at companies with more than a handful of Airflow users. Unless there’s a clear benefit to removing them *and* a well-defined migration path, I don’t think it’s worth the disruption. +1 to Bolke’s points.

Re: Discuss: lets' stop posting on X and muffle our account there.

I'm on the same page as TP and Wei. I think we should definitely stop posting and have a final post along with updating our profile to say we've moved on to Fosstodon (if not done already, I'm not on X/Twitter), but I'm not a fan of deleting past posts. If anything, leaving them up helps show the b

Re: AW: [ANNOUNCE] Apache Airflow 3.0.0 Released

Congrats everybody. What a journey!! On Thu, Apr 24, 2025 at 6:09 AM Pierre Jeambrun wrote: > Amazing! Thanks to everybody that was involved > > On Wed, Apr 23, 2025 at 10:31 PM Buğra Öztürk > wrote: > > > Amazing milestone! Great work and collaboration! Congrats everyone! > > > > On Wed, Apr 2

Re: [DISCUSS] Example dags

I know we rely on the example dags for test purposes, especially for UI development. As part of moving them out of core, can we update breeze to automatically load them from their new home? On Wed, Apr 30, 2025 at 7:12 AM Kaxil Naik wrote: > just an idea: With "git bundles", maybe example DAGs f

Re: [DISCUSS] Example dags

s/test/dev On Wed, Apr 30, 2025 at 8:49 AM Constance Martineau wrote: > I know we rely on the example dags for test purposes, especially for UI > development. As part of moving them out of core, can we update breeze to > automatically load them from their new home? > > On Wed, Ap

Re: [DISCUSS] When a DAG is paused, change the dag run state from running to failed.

I agree with Elad. If I pause a DAG, I expect for whatever is running to continue and for the scheduler to stop scheduling dag runs and new tasks. I don't think we should change the behaviour to fix a chart. That feels backwards. Separately: In my view, choosing to to prevent or run new tasks from

Re: Airflow should deprecate the term "DAG" for end users

t; > >>>>>>>>>>> -- Rajesh > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >> > >>>>>>>>>>> > >&

Re: [DISCUSS] Time to say goodbye to the old UI?

+1 I completely agree that now is the right time to remove the old UI. Keeping both UIs around at this stage is more of a drag than a safety net and is increasing the maintenance cost (things are starting to become very wonky in the old UI). Having a clean break sets clear expectations: Airflow 3

Re: [DISCUSSION] Changing catchup_by_default from True to False

> I strongly disagree with the proposal of changing the default for all DAGs. This requires every user that does not specify catchup to modify their DAGs. I don't think that's accurate. Tamara's proposal is just to change the default value of the global configuration that controls this

Re: [DISCUSS] removing pickling of executor config.... and any others?

+1 for #2. #1 for me is a non-starter. Yes, users can set a pod_template_file instead, but since Airflow will use that file instead of what is set at the deployment level, users have to know the internals of the cluster to define it properly, and become responsible for remembering to update things

Re: New committer: Rahul Vats

Congrats Rahul! Well deserved :) On Mon, Mar 31, 2025 at 2:28 PM Oliveira, Niko wrote: > Congrats Rahul, great work! > > > From: Jed Cunningham > Sent: Monday, March 31, 2025 11:19:53 AM > To: dev@airflow.apache.org > Subject: [EXT] New committer: Rahul Vats > >

Re: [DISCUSS] Human in the loop

I like the idea! We built something similar where a task would send an email with a report attached (Excel, of course). The user had to either approve the report or flag an issue. If they didn't respond in 2 hours, it escalated to a backup, then to the department lead. If no one responded, the DAG

Re: [RESULT][VOTE] April 2025 PR of the Month

I'm late to the party (I was out of office and offline for the last two weeks) so I only saw this now!! Thank you so much!! Means a lot to me On Sat, May 10, 2025 at 2:37 AM Amogh Desai wrote: > Superb! > > Happy to see Constance vs Constrance :) > > Thanks & Regards, &g

Re: [DISCUSS] Ray provider

We do see a not insignificant number of our customers using Ray & Airflow together on Astro and Astronomer-Software, so there is definitely interest and I believe this could be really valuable. https://github.com/astronomer/astro-provider-ray/ was created for validation purposes, but we were not ab

Re: [ANNOUNCE] New PMC member: Amogh Desai

Congrats Amogh! Very well deserved!! On Thu, Jun 26, 2025 at 9:03 AM Jarek Potiuk wrote: > Congrats Indeed very well deserved! > > On Thu, Jun 26, 2025 at 2:17 PM Aritra Basu > wrote: > > > Congrats Amogh! Very well deserved and love to see it!! > > -- > > Regards, > > Aritra Basu > > > > O