And the flip side, if the cluster is heavily loaded then increasing log levels is likely to have a detectable impact on performance! Paul
From: Brebner, Paul <paul.breb...@netapp.com> Date: Saturday, 2 November 2024 at 9:50 am To: users@kafka.apache.org <users@kafka.apache.org>, om22sh...@gmail.com <om22sh...@gmail.com> Subject: Re: Q: Does Kafka log level affect performance and latency Hi Om, I asked some of our techops people about this, and their general advice is that increasing the log level from the default (INFO?) is likely to increase the I/O (the amount depending on a variety of factors including the cluster traffic etc) – and my take on this is that assuming there is sufficient I/O headroom it shouldn’t impact the cluster throughput or latency. However, in previous performance benchmarking I haven’t tried changing log levels so I would suggest you do some more detailed benchmarking with realistic workloads and increased log levels, good luck, Paul Brebner From: Om Shree <om22sh...@gmail.com> Date: Friday, 1 November 2024 at 2:47 am To: users@kafka.apache.org <users@kafka.apache.org> Subject: Q: Does Kafka log level affect performance and latency [You don't often get email from om22sh...@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Hello community, We use Kafka extensively at our organisation. Our SLAs are strict and require:- — throughputs north of 1000TPS and — 60ms latency per transaction We run:- — 6 brokers and 6 zookeepers — brokers and zookeepers are hosted on ec2 instances with sufficient iops, throughput and network bandwidth to meet our requirements We were under the impression that using log levels (on brokers) such as INFO or DEBUG would produce too many server logs and have an adverse impact on performance. Can someone with experience please confirm that log levels won’t have an adverse effect on latency and throughput of these clusters or vice-versa, given that we come up with a strategy to clean out these logs on broker servers after a defined unit of time ? Does generation of system logs have potential to impact disk or compute iops and slow us down ? Thanks!
