Thanks, also thanks Fares for pointing me to Keda.
On Sun, Mar 6, 2022 at 4:18 PM Liam Clarke-Hutchinson
wrote:
> > I was trying to see what the goals of enabling Hpa on the consumer would
> be. Since like you say there is a partition upper limit which will limit
> the consumer throughput. so
> I was trying to see what the goals of enabling Hpa on the consumer would
be. Since like you say there is a partition upper limit which will limit
the consumer throughput. so in the end you have to tweak partitions on
kafka and then reassess the maxReplicas config of hpa.
It seems hpa in this scen
HI Liam,
I was trying to see what the goals of enabling Hpa on the consumer would
be. Since like you say there is a partition upper limit which will limit
the consumer throughput. so in the end you have to tweak partitions on
kafka and then reassess the maxReplicas config of hpa.
It seems hpa in t
Hi David,
Scaling on CPU can be fine, what you scale on depends on what resource
constrains your consuming application. CPU is a good proxy for "I'm working
really hard", so not a bad one to start with.
Main thing to be aware of is tuning the HPA to minimise scaling that causes
"stop-the-world" c
Hello
You can look at keda.sh which allows to auto scale workloads on kubernetes
according to many custom metrics including Kafka lag for instance.
Fares
Le mer. 2 mars 2022 à 01:07, David Ballano Fernandez <
dfernan...@demonware.net> a écrit :
> Thanks Liam,
>
> I am trying hpa but using cpu u
Thanks Liam,
I am trying hpa but using cpu utilization, but since everything is tied to
partition number etc i wonder what the benefits of running on hpa really
are.
thanks!
On Mon, Feb 28, 2022 at 12:59 PM Liam Clarke-Hutchinson
wrote:
> I've used HPAs scaling on lag before by feeding lag met
I've used HPAs scaling on lag before by feeding lag metrics from Prometheus
into the K8s metrics server as custom metrics.
That said, you need to carefully control scaling frequency to avoid
excessive consumer group rebalances. The cooperative sticky assignor can
minimise pauses, but not remove th
Hello Guys,
I was wondering how you guys do autoscaling of you consumers in kubernetes
if you do any.
We have a mirrormaker-like app that mirrors data from cluster to cluster at
the same time does some topic routing. I would like to add hpa to the app
in order to scale up/down depending on avg c
