Looks like a good proposal.
Regards,
Poorvi Rohidekar
On Wed, 26 Jun 2024 at 00:28, Aritra Basu wrote:
> Agreed, overall sounds like a positive change. Don't see any issues with it
> --
> Regards,
> Aritra Basu
>
> On Tue, Jun 25, 2024, 10:28 PM Ferruzzi, Dennis
>
> wrote:
>
> > Sounds good, I
I was unaware of the Teradata dashboard! Outstanding to see.
I can take point on design discussion and documentation for this, but in the
end it'll be up to each provider to update their own infra, so there is only so
much I can do.
I didn't really think this would catch on so enthusiastically
Why would it be common.dataframe instead of providers.ibis?
Thanks Vikram, appreciate it.
On Tue, 25 Jun 2024 at 21:16, Vikram Koka
wrote:
> Thanks Kaxil, I really appreciate the diligent follow up here.
> Both the preparation and follow through is excellent!
>
> Vikram
>
>
> On Mon, Jun 24, 2024 at 5:09 PM Kaxil Naik wrote:
>
> > Hey all,
> >
> > Apolo
Thanks Kaxil, I really appreciate the diligent follow up here.
Both the preparation and follow through is excellent!
Vikram
On Mon, Jun 24, 2024 at 5:09 PM Kaxil Naik wrote:
> Hey all,
>
> Apologies for the delay!
>
> I have updated our meeting notes document to summarize the discussion
> from
Very interested in this.
I am quite positive and supportive of adding support for a generic
dataframe abstraction within Airflow.
However, I do have a few questions around how and where to include this
within Airflow from a dependency perspective.
I do wonder if this needs to be in Core Airflow o
Yeah I'm not saying there shouldn't be an airflow library. It's just
> unclear to me what its purpose would be and it would be helpful in
> evaluating the question to have some kind of a sketch of it. What
> interface it would introduce, how it would be used etc.
>
Yep. Very reasonable questio
Hey All,
It’s once again time to vote for the PR of the Month!
With the help of the `get_important_pr_candidates` script in dev/stats,
we've identified the following candidates:
PR #37948: [AIP-49] OpenTelemetry Traces for Apache Airflow <
https://github.com/apache/airflow/pull/37948>
PR #39936
Agreed, overall sounds like a positive change. Don't see any issues with it
--
Regards,
Aritra Basu
On Tue, Jun 25, 2024, 10:28 PM Ferruzzi, Dennis
wrote:
> Sounds good, I don't see a down side and "supply chain security" has been
> a big concern lately.
>
>
> - ferruzzi
>
>
> _
Sounds good, I don't see a down side and "supply chain security" has been a big
concern lately.
- ferruzzi
From: Wei Lee
Sent: Tuesday, June 25, 2024 8:07 AM
To: dev@airflow.apache.org
Subject: RE: [EXT] [PROPOSAL] Use Trusted Publishing workflow for Airflow
+1 non binding. All AWS system tests are running successfully against
apache-airflow-providers-amazon==8.25.0rc1. You can see the results here:
https://aws-mwaa.github.io/#/open-source/system-tests/version/c310159bc2363c12110b11febd5febaab8670210_8.25.0rc1.html
On 2024/06/24 12:43:05 Pankaj Koti
This proposal is great! PyPI security has been valued a lot these days. Glad
we're also joining.
Best,
Wei
> On Jun 25, 2024, at 8:01 PM, Jarek Potiuk wrote:
>
> Yes and no :)
>
> We publish alpha/betas - yes. No change there. But for RCs what we publish
> in SVN currently are the packages th
Yeah I'm not saying there shouldn't be an airflow library. It's just
unclear to me what its purpose would be and it would be helpful in
evaluating the question to have some kind of a sketch of it. What
interface it would introduce, how it would be used etc.
On Tue, Jun 25, 2024 at 6:51 AM Gil Fo
Hello!
Ibis core developer here. This is an exciting proposal, we'd love to see Ibis
and Airflow working together smoothly. To Daniel's point, yes, Ibis is a
common dataframe library on its own, but to reiterate Jarek's response, we
don't really handle orchestration, especially between multip
+1 to the idea of standardizing the format of the system test results output
On Tue, Jun 25, 2024 at 10:40 AM Jarek Potiuk wrote:
> So we have almost everyone on board!
>
> Now we also need the Teradata team to add whatever JSON/XML we come up with
> :). In case people have not noticed, among ou
Yes and no :)
We publish alpha/betas - yes. No change there. But for RCs what we publish
in SVN currently are the packages that are built fro RC tag but without rc
suffix - so that when they pass the voting we upload them to PyPI without
regenerating them (RC becomes final).
But we do not publish
Yeah, this is an excellent proposal. The part that the release manager
doesn't have to be added to the Apache Airflow organization in PYPI before
they can publish a package would be a significant improvement.
> small difference is that we will also have to upload RC/BETA packages to
SVN - we are c
So we have almost everyone on board!
Now we also need the Teradata team to add whatever JSON/XML we come up with
:). In case people have not noticed, among our dashboards [1] we also have
Teradata dashboard [2]
[1]
https://airflow.apache.org/ecosystem/#airflow-provider-system-test-dashboards
[2]
For context, the Astronomer LLM providers dashboard operates as follows:
1. Fetch the latest source code for providers and system tests/example DAGs
from the Airflow repository, deploy them to an Airflow instance, and
execute the
DAGs.
2. Use the Airflow API to retrieve the DAG run statuses and pr
Hello everyone,
TL;DR; I wanted to propose switching to PyPI Trusted Publishing workflow
for Airflow releases with Github Actions being the trusted publisher.
A bit of background:
In the security team and also in the security committee of ASF w have been
discussing ways how we can plugin into Tr
20 matches
Mail list logo
