Hi everyone I have updated the content of KIP. I want to highlight that although we set log4j2 as the default server-side logging framework we couldn't change its scope to implementation in `build.gradle`. This will lead the downstream projects to encounter dependency conflicts.
Please take a look and share your feedback. Best, TengYao TengYao Chi <kiting...@gmail.com> 於 2025年3月23日 週日 下午2:23寫道： > Hi Stig > > Sorry for the late reply, and thanks for your question. 😀 > > The title "Upgrade slf4j to 2.x" represents the core change enabling > improved logging backend selection. > The fundamental change is the SLF4J upgrade, which brings the new provider > selection mechanism (`-Dslf4j.provider`) as a key feature. > Given that, I think the current title reflects the primary technical > change, while the KIP details explain the resulting benefits and > implementation approach. > > Best, > TengYao > > > Stig Rohde Døssing <stigdoess...@gmail.com> 於 2025年3月21日 週五 下午11:46寫道： > >> The title of the KIP seems a little odd, because if I understand >> correctly, >> the main change you want to make is to bundle multiple logging backends >> with Kafka and make them selectable via a system property, and upgrading >> sfl4j is a means to achieve that, not the goal itself? >> >> Den fre. 21. mar. 2025 kl. 12.18 skrev Chia-Ping Tsai <chia7...@gmail.com >> >: >> >> > hi Teng >> > >> > > The KIP will document that log4j2 is the only officially supported >> > server-side logging framework, and we will expose its configuration >> file >> > to >> > users. >> > >> > on the server-side, we should keep current scope - compileOnly and >> > releaseOnly - Otherwise, downstream projects could encounter dependency >> > conflicts [0] >> > >> > > I will revise the motivation section of the KIP to emphasize that the >> > >> > Please include the following description. >> > >> > "The rationale for this KIP is that upgrading SLF4J necessitates >> > corresponding provider upgrades, which constitutes a breaking change." >> > >> > Also, we must upgrade the Log4j2 dependency based on SLF4J 2 (i.e., >> > log4j-slf4j-impl to log4j-slf4j2-impl) in 5.0 if we upgrade to slf4j2 >> > >> > [0] https://github.com/apache/kafka/pull/17373#issuecomment-2577813317 >> > >> > Best, >> > >> > Chia-Ping >> > >> > >> > TengYao Chi <kiting...@gmail.com> 於 2025年3月21日 週五 下午6:52寫道： >> > >> > > Hello everyone, >> > > >> > > Sorry for the late reply. >> > > Based on the previous discussion, I would like to conclude with the >> > > following points: >> > > >> > > 1. The upgrade to slf4j2 should be postponed to Kafka 5.0 due to >> > > compatibility issues. >> > > 2. I will revise the motivation section of the KIP to emphasize >> that >> > the >> > > key benefit is allowing users to select logging backends through >> > > configuration rather than modifying JAR files. >> > > 3. After the slf4j upgrade, users will be able to use the >> > > `-Dslf4j.provider` system property to configure their preferred >> > logging >> > > backend. >> > > 4. The KIP will document that log4j2 is the only officially >> supported >> > > server-side logging framework, and we will expose its configuration >> > > file to >> > > users. >> > > To avoid breaking downstream compatibility, we will not bind the >> > client >> > > side to any specific logging framework. Users will need to manage >> > their >> > > own >> > > logging libraries, but they can utilize the `-Dslf4j.provider` >> > property >> > > once slf4j is upgraded. >> > > 5. We have rejected alternatives that involve warnings and >> classpath >> > > ordering as they do not provide a solid solution to compatibility >> > > issues. >> > > >> > > Does this summary make sense? >> > > >> > > Best, >> > > TengYao >> > > >> > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月21日 週五 上午9:32寫道： >> > > >> > > > A solution that involves a warning and classpath ordering doesn't >> meet >> > > the >> > > > bar for me. Good clarification though. >> > > > >> > > > Ismael >> > > > >> > > > On Thu, Mar 20, 2025 at 8:37 AM Farid Zakaria >> > > > <fzaka...@confluent.io.invalid> >> > > > wrote: >> > > > >> > > > > AFAIR SLF4J you don't have to remove the other backends; merely >> make >> > > > > sure yours is first on the CLASSPATH list :P >> > > > > (SLF4J pre 2.0 would always emit a warning that it found 2+ >> > > > StaticBinders) >> > > > > >> > > > > Interestingly, you could still whatever backend (i.e. Log4J) and >> pipe >> > > > > it through to another backend via another appender. >> > > > > This is what SLF4J refers to bridges -- although you have to be >> sure >> > > > > not to create a circular loop. >> > > > > >> > > > > Then there is something also general like syslog. >> > > > > >> > > > > On Thu, Mar 20, 2025 at 1:15 AM Chia-Ping Tsai < >> chia7...@gmail.com> >> > > > wrote: >> > > > > > >> > > > > > hi Ismael >> > > > > > >> > > > > > thanks for all your response. >> > > > > > >> > > > > > > All that said, I am not actually sure we can do what I >> described >> > > > above >> > > > > > while maintaining the compatibility required by a minor release. >> > > > > > >> > > > > > Excuse me, are your concerns related to version matching, as >> > > discussed >> > > > in >> > > > > > [0]? If so, I concur that this represents a compatibility issue, >> > and >> > > > the >> > > > > > target version for this change should be 5.0. Additionally, >> there >> > > was a >> > > > > > related discussion previously documented in [1]. While we have >> not >> > > > > strictly >> > > > > > enforced version matching during prior SLF4J updates, this KIP >> > > provides >> > > > > an >> > > > > > opportunity to establish guidelines for upgrading sl4fj that >> have >> > > > direct >> > > > > > compatibility implications. >> > > > > > >> > > > > > [0] https://www.slf4j.org/faq.html#compatibility >> > > > > > [1] >> > > https://github.com/apache/kafka/pull/16324#discussion_r1644671854 >> > > > > > >> > > > > > To Teng >> > > > > > >> > > > > > Could you please revise the KIP according to following benefit? >> > > > > > >> > > > > > > The key benefit of this KIP is that you can add new logging >> > > backends >> > > > > and >> > > > > > select it via a config. This is how most pluggable things work. >> But >> > > it >> > > > is >> > > > > > *not* how slf4j 1.x works. slf4j 1.x requires you to *remove* >> the >> > > > default >> > > > > > logging library picked by the project as well. That's much more >> > > > > intrusive. >> > > > > > >> > > > > > Best, >> > > > > > Chia-Ping >> > > > > > >> > > > > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月20日 週四 上午8:10寫道： >> > > > > > >> > > > > > > Hi Chia-Ping, >> > > > > > > >> > > > > > > I don't think we're in the business of shipping multiple >> logging >> > > > > libraries. >> > > > > > > Here's my take: >> > > > > > > >> > > > > > > 1. We should pick one logging library for services/servers, >> ship >> > it >> > > > and >> > > > > > > include a configuration file for it. >> > > > > > > 2. For clients, we leave it to the users to select the logging >> > > > library >> > > > > - >> > > > > > > clients run alongside applications and it's desirable to use >> the >> > > same >> > > > > > > logging library for both (if we were starting from scratch, we >> > may >> > > > have >> > > > > > > decided to also include our default logging library for >> clients >> > as >> > > > > well, >> > > > > > > but it's hard to make that change now). >> > > > > > > 3. For the cases where users want to use a different logging >> > > library >> > > > > for >> > > > > > > services/servers (perhaps because they have standardized on a >> > > > different >> > > > > > > logging library), they would have to add the additional jar to >> > the >> > > > > > > classpath and change the relevant logging config. This is no >> > > > different >> > > > > than >> > > > > > > adding a different authorizer or any other pluggable >> component. >> > > > > > > >> > > > > > > The key benefit of this KIP is that you can add new logging >> > > backends >> > > > > and >> > > > > > > select it via a config. This is how most pluggable things >> work. >> > But >> > > > it >> > > > > is >> > > > > > > *not* how slf4j 1.x works. slf4j 1.x requires you to *remove* >> the >> > > > > default >> > > > > > > logging library picked by the project as well. That's much >> more >> > > > > intrusive. >> > > > > > > >> > > > > > > All that said, I am not actually sure we can do what I >> described >> > > > above >> > > > > > > while maintaining the compatibility required by a minor >> release. >> > > > > > > >> > > > > > > Ismael >> > > > > > > >> > > > > > > On Wed, Mar 19, 2025, 2:03 PM Chia-Ping Tsai < >> chia7...@gmail.com >> > > >> > > > > wrote: >> > > > > > > >> > > > > > > > hi Ismael >> > > > > > > > >> > > > > > > > > but they must also add whichever logging library they >> want to >> > > > use. >> > > > > > > > >> > > > > > > > If users are required to modify JAR files to alter the SLF4J >> > > > > provider, >> > > > > > > the >> > > > > > > > value of this KIP is significantly diminished. I believe the >> > > > primary >> > > > > > > > benefit of this KIP lies in enabling users to configure a >> > system >> > > > > property >> > > > > > > > for SLF4J provider changes without JAR modifications. >> > > Furthermore, >> > > > by >> > > > > > > > managing all SLF4J and provider JARs within Kafka, we can >> > ensure >> > > > > SLF4J >> > > > > > > > version updates without compatibility concerns, as we can >> > > guarantee >> > > > > > > > provider JAR consistency with the SLF4J version in the >> > > > distribution. >> > > > > > > > >> > > > > > > > Best, >> > > > > > > > Chia-Ping >> > > > > > > > >> > > > > > > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月19日 週三 下午12:00寫道： >> > > > > > > > >> > > > > > > > > Hi TengYao, >> > > > > > > > > >> > > > > > > > > It's a bit difficult to review the KIP. I don't follow >> most >> > of >> > > > the >> > > > > > > > > motivation. The only one that I follow is: >> > > > > > > > > >> > > > > > > > > "Our current build configuration employs fragile >> dependency >> > > > > management >> > > > > > > > > tricks to handle SLF4J backends. We can eliminate these >> > brittle >> > > > > build >> > > > > > > > > mechanisms by transitioning to explicit provider >> dependencies >> > > > after >> > > > > > > > > upgrading to 2.0." >> > > > > > > > > >> > > > > > > > > Additionally, I don't think we should do the following: >> > > > > > > > > >> > > > > > > > > "Add other popular slf4j backend binding provider >> > > dependencies." >> > > > > > > > > >> > > > > > > > > A reasonable approach, in my opinion, would be: >> > > > > > > > > >> > > > > > > > > 1. Include the log4j2 dependency with the server modules >> and >> > > not >> > > > > > > include >> > > > > > > > > them with the client modules. >> > > > > > > > > 2. Automatically configure the log4j2 dependency for the >> > server >> > > > > > > modules. >> > > > > > > > > Users can override them via the system property, but they >> > must >> > > > > also add >> > > > > > > > > whichever logging library they want to use. >> > > > > > > > > 3. Somehow configure slf4j 2.x to work like 1.x out of the >> > box >> > > > for >> > > > > the >> > > > > > > > > client module (for compatibility reasons). >> > > > > > > > > >> > > > > > > > > But I don't know if `3` is possible. If `3` is not >> possible, >> > I >> > > > > don't >> > > > > > > see >> > > > > > > > > how we can make this a compatible change. >> > > > > > > > > >> > > > > > > > > Ismael >> > > > > > > > > >> > > > > > > > > On Tue, Mar 18, 2025 at 7:41 PM TengYao Chi < >> > > kiting...@gmail.com >> > > > > >> > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > Hello everyone, >> > > > > > > > > > >> > > > > > > > > > I want to bump this thread manually. >> > > > > > > > > > Any feedback or vote would be appreciated. >> > > > > > > > > > >> > > > > > > > > > Best, >> > > > > > > > > > TengYao >> > > > > > > > > > >> > > > > > > > > > TengYao Chi <kiting...@gmail.com> 於 2025年3月10日 週一 >> > 上午11:47寫道： >> > > > > > > > > > >> > > > > > > > > > > Hello guys, >> > > > > > > > > > > I would like to remind you in the vote thread that the >> > KIP >> > > > has >> > > > > been >> > > > > > > > > > > updated, and I apologize for repeating it. >> > > > > > > > > > > I have taken over this KIP from Muralidhar. >> > > > > > > > > > > Since the original content is outdated as the logging >> > > > > framework has >> > > > > > > > > been >> > > > > > > > > > > widely changed, I have updated the content of the KIP. >> > > > > > > > > > > Please take a look and share your thoughts. >> > > > > > > > > > > >> > > > > > > > > > > Best Regards, >> > > > > > > > > > > TengYao >> > > > > > > > > > > >> > > > > > > > > > > Muralidhar Basani <muralidhar.bas...@aiven.io.invalid> >> 於 >> > > > > > > 2024年9月24日 >> > > > > > > > 週二 >> > > > > > > > > > > 上午5:09寫道： >> > > > > > > > > > > >> > > > > > > > > > >> Hi, >> > > > > > > > > > >> >> > > > > > > > > > >> I wanted to gently follow up on this thread in case >> > anyone >> > > > > has any >> > > > > > > > > > >> thoughts >> > > > > > > > > > >> or would like to take a look. >> > > > > > > > > > >> >> > > > > > > > > > >> Thanks, >> > > > > > > > > > >> Murali >> > > > > > > > > > >> >> > > > > > > > > > >> On Mon, Sep 16, 2024 at 10:23 AM Muralidhar Basani < >> > > > > > > > > > >> muralidhar.bas...@aiven.io> wrote: >> > > > > > > > > > >> >> > > > > > > > > > >> > Thanks Chia. >> > > > > > > > > > >> > >> > > > > > > > > > >> > I have updated KIP with this quote, in the >> migration >> > > plan >> > > > > > > section. >> > > > > > > > > > >> > >> > > > > > > > > > >> > Thanks, >> > > > > > > > > > >> > Murali >> > > > > > > > > > >> > >> > > > > > > > > > >> > On Sun, Sep 15, 2024 at 3:30 PM Chia-Ping Tsai < >> > > > > > > > chia7...@gmail.com> >> > > > > > > > > > >> wrote: >> > > > > > > > > > >> > >> > > > > > > > > > >> >> >> > > > > > > > > > >> >> > Muralidhar Basani <muralidhar.bas...@aiven.io >> > > .invalid> >> > > > 於 >> > > > > > > > > > 2024年9月15日 >> > > > > > > > > > >> >> 晚上9:02 寫道： >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > With this, I think, users don't have to make any >> > > > explicit >> > > > > > > > changes >> > > > > > > > > > in >> > > > > > > > > > >> >> their >> > > > > > > > > > >> >> > code, if their provider is reload4j. And if >> it's a >> > > > > different >> > > > > > > > > > provider >> > > > > > > > > > >> >> (like >> > > > > > > > > > >> >> > logback, log4j), they would have to upgrade >> that to >> > > > > match it >> > > > > > > > with >> > > > > > > > > > >> slf4j. >> > > > > > > > > > >> >> >> > > > > > > > > > >> >> If upgrading the matched provider is the only >> > explicit >> > > > > change >> > > > > > > and >> > > > > > > > > we >> > > > > > > > > > >> >> expect users have responsibility to keep >> consistent >> > > > version >> > > > > > > when >> > > > > > > > > > using >> > > > > > > > > > >> >> other providers , could we write it down to the >> KIP? >> > > > > > > > > > >> >> >> > > > > > > > > > >> >> That means we will update slf4j without KIP in the >> > > future >> > > > > > > except >> > > > > > > > > for >> > > > > > > > > > >> >> specific reason. >> > > > > > > > > > >> >> >> > > > > > > > > > >> >> Best, >> > > > > > > > > > >> >> Chia-Ping >> > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > >> >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > >> > > > >> > > >> > >> >
