DEBUG level for some classes could have dramatic performance impact.
I tested DEBUG level for *kafka.authorizer.logger *for audit purposes, like
a naive approach to have audit logs about actions taken by engineers when
working with cluster directly. I don't have exact data right now, since it
was around 2 years ago, but I remember we managed to kill the cluster
completely by this change and had to rollback since it caused quite a wide
incident.

On Sat, Nov 2, 2024 at 12:53 AM Brebner, Paul
<paul.breb...@netapp.com.invalid> wrote:

> And the flip side, if the cluster is heavily loaded then increasing log
> levels is likely to have a detectable impact on performance! Paul
>
> From: Brebner, Paul <paul.breb...@netapp.com>
> Date: Saturday, 2 November 2024 at 9:50 am
> To: users@kafka.apache.org <users@kafka.apache.org>, om22sh...@gmail.com <
> om22sh...@gmail.com>
> Subject: Re: Q: Does Kafka log level affect performance and latency
> Hi Om,  I asked some of our techops people about this, and their general
> advice is that increasing the log level from the default (INFO?) is likely
> to increase the I/O (the amount depending on a variety of factors including
> the cluster traffic etc) – and my take on this is that assuming there is
> sufficient I/O headroom it shouldn’t impact the cluster throughput or
> latency.  However, in previous performance benchmarking I haven’t tried
> changing log levels so I would suggest you do some more detailed
> benchmarking with realistic workloads and increased log levels, good luck,
> Paul Brebner
>
> From: Om Shree <om22sh...@gmail.com>
> Date: Friday, 1 November 2024 at 2:47 am
> To: users@kafka.apache.org <users@kafka.apache.org>
> Subject: Q: Does Kafka log level affect performance and latency
> [You don't often get email from om22sh...@gmail.com. Learn why this is
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> EXTERNAL EMAIL - USE CAUTION when clicking links or attachments
>
>
>
>
> Hello community,
>
> We use Kafka extensively at our organisation.
> Our SLAs are strict and require:-
> — throughputs north of 1000TPS and
> — 60ms latency per transaction
>
> We run:-
> — 6 brokers and 6 zookeepers
> — brokers and zookeepers are hosted on ec2 instances with sufficient iops,
> throughput and network bandwidth to meet our requirements
>
> We were under the impression that using log levels (on brokers) such as
> INFO or DEBUG would produce too many server logs and have an adverse impact
> on performance.
>
> Can someone with experience please confirm that log levels won’t have an
> adverse effect on latency and throughput of these clusters or vice-versa,
> given that we come up with a strategy to clean out these logs on broker
> servers after a defined unit of time ?
> Does generation of system logs have potential to impact disk or compute
> iops and slow us down ?
>
> Thanks!
>

Reply via email to