Hi All,
just wanted to give a quick update on the effort discussed here:
The action items from the retrospective are tracked in
https://github.com/apache/beam/issues/25652.
Many outdated dependencies were updated in
https://github.com/apache/beam/pull/24599 by +Anand Inguva
and remaining older
Happy to help and I hope we can work together with Valentyn and others to
get the "google clients" approach improved :)
J.
On Fri, Aug 26, 2022 at 3:40 PM Kerry Donny-Clark via dev <
dev@beam.apache.org> wrote:
> Jarek, I really appreciate you sharing your experience and expertise here.
> I thi
Jarek, I really appreciate you sharing your experience and expertise here.
I think Beam would benefit from adopting some of these practices.
Kerry
On Fri, Aug 26, 2022, 7:35 AM Jarek Potiuk wrote:
>
>> I'm curious Jarek, does Airflow take any dependencies on popular
>> libraries like pandas, num
>
> I'm curious Jarek, does Airflow take any dependencies on popular libraries
> like pandas, numpy, pyarrow, scipy, etc... which users are likely to have
> their own dependency on? I think these dependencies are challenging in a
> different way than the client libraries - ideally we would support
Thanks for writing this up Valentyn!
I'm curious Jarek, does Airflow take any dependencies on popular libraries
like pandas, numpy, pyarrow, scipy, etc... which users are likely to have
their own dependency on? I think these dependencies are challenging in a
different way than the client libraries
Hi Jarek,
Thanks a lot for detailed feedback and sharing the Airflow story, this is
exactly what I was hoping to hear in response from the mailing list!
600+ dependencies is very impressive, so I'd be happy to chat more and
learn from your experience.
On Wed, Aug 24, 2022 at 5:50 AM Jarek Potiuk
Comment (from a bit outsider)
Fantastic document Valentyn.
Very, very insightful and interesting. We feel a lot of the same pain in
Apache Airflow (actually even more because we have not 20 but 620+
dependencies) but we are also a bit more advanced in the way how we are
managing the dependencies