;opensearch" is separate from "elasticsearch".
>
> Gesendet: Dienstag, den 15.07.2025 um 21:38 Uhr
> > Von: "Jarek Potiuk"
> > An: dev@airflow.apache.org
> > Cc: gha...@amazon.com.invalid
> > Betreff: Re: [DISCUSSION] New Provider: Valke
pache.org
> Cc: gha...@amazon.com.invalid
> Betreff: Re: [DISCUSSION] New Provider: Valkey (Redis OSS Fork)
>
> > Can you please explain why it should be a provider managed by Airflow
> rather than on your own?
>
> Just to explain it Elad - and yes, I know I am usually the one that is
>
I still see no arguments why it should be a provider maintained by airflow.
Is the only reason because we have a Redis provider?
What is the downside of running it as a 3rd party provider and if/when it
gains popularity donate it to airflow?
On Tue, Jul 15, 2025 at 10:39 PM Jarek Potiuk wrote:
> Can you please explain why it should be a provider managed by Airflow
rather than on your own?
Just to explain it Elad - and yes, I know I am usually the one that is
sceptical -> valley is truly community driven replacement for redis
https://valkey.io/participants/ - you see a lot of well-known
> * Helm Chart support to run valley as Celery Broker instead of Redis
Oh yes! didn't consider its usage as a broker but is it related to the
provider package?
Amazon SQS can be used as broker but it's not related to the SQS
integrations in the Airflow amazon provider (hook,operator,sensor)
I also
Yes. Valkey is a good replacement/parallel implementation to Redis. One
thing however is important - I would see this even more useful if it also
allows for the Celery Redis replacement for broker - means that it is fully
supported in our:
* unit tests (of course)
* integration tests with ("valkey
Can you please explain why it should be a provider managed by Airflow
rather than on your own?
Redis provider is not very popular as normally redis isn't that
involved with batch processing. Why should Valkey be different?
Please also clarify if Amazon are committing to the maintenance of the
prov
