Thanks TengYao, I note that the KIP's scope has shrunk a bit, and the goal now is only to upgrade slf4j to make it easy for users to replace log4j2 with another slf4j binding, but Kafka will only ship with log4j2 and not alternatives, so people who want to use something else will have to bring their own jars.
+1 (non-binding, not a committer) for that plan Den søn. 23. mar. 2025 kl. 12.13 skrev TengYao Chi <kiting...@gmail.com>: > Hi everyone > > I have updated the content of KIP. > I want to highlight that although we set log4j2 as the default server-side > logging framework we couldn't change its scope to implementation in > `build.gradle`. > This will lead the downstream projects to encounter dependency conflicts. > > Please take a look and share your feedback. > > Best, > TengYao > > TengYao Chi <kiting...@gmail.com> 於 2025年3月23日 週日 下午2:23寫道： > > > Hi Stig > > > > Sorry for the late reply, and thanks for your question. 😀 > > > > The title "Upgrade slf4j to 2.x" represents the core change enabling > > improved logging backend selection. > > The fundamental change is the SLF4J upgrade, which brings the new > provider > > selection mechanism (`-Dslf4j.provider`) as a key feature. > > Given that, I think the current title reflects the primary technical > > change, while the KIP details explain the resulting benefits and > > implementation approach. > > > > Best, > > TengYao > > > > > > Stig Rohde Døssing <stigdoess...@gmail.com> 於 2025年3月21日 週五 下午11:46寫道： > > > >> The title of the KIP seems a little odd, because if I understand > >> correctly, > >> the main change you want to make is to bundle multiple logging backends > >> with Kafka and make them selectable via a system property, and upgrading > >> sfl4j is a means to achieve that, not the goal itself? > >> > >> Den fre. 21. mar. 2025 kl. 12.18 skrev Chia-Ping Tsai < > chia7...@gmail.com > >> >: > >> > >> > hi Teng > >> > > >> > > The KIP will document that log4j2 is the only officially supported > >> > server-side logging framework, and we will expose its configuration > >> file > >> > to > >> > users. > >> > > >> > on the server-side, we should keep current scope - compileOnly and > >> > releaseOnly - Otherwise, downstream projects could encounter > dependency > >> > conflicts [0] > >> > > >> > > I will revise the motivation section of the KIP to emphasize that > the > >> > > >> > Please include the following description. > >> > > >> > "The rationale for this KIP is that upgrading SLF4J necessitates > >> > corresponding provider upgrades, which constitutes a breaking change." > >> > > >> > Also, we must upgrade the Log4j2 dependency based on SLF4J 2 (i.e., > >> > log4j-slf4j-impl to log4j-slf4j2-impl) in 5.0 if we upgrade to slf4j2 > >> > > >> > [0] > https://github.com/apache/kafka/pull/17373#issuecomment-2577813317 > >> > > >> > Best, > >> > > >> > Chia-Ping > >> > > >> > > >> > TengYao Chi <kiting...@gmail.com> 於 2025年3月21日 週五 下午6:52寫道： > >> > > >> > > Hello everyone, > >> > > > >> > > Sorry for the late reply. > >> > > Based on the previous discussion, I would like to conclude with the > >> > > following points: > >> > > > >> > > 1. The upgrade to slf4j2 should be postponed to Kafka 5.0 due to > >> > > compatibility issues. > >> > > 2. I will revise the motivation section of the KIP to emphasize > >> that > >> > the > >> > > key benefit is allowing users to select logging backends through > >> > > configuration rather than modifying JAR files. > >> > > 3. After the slf4j upgrade, users will be able to use the > >> > > `-Dslf4j.provider` system property to configure their preferred > >> > logging > >> > > backend. > >> > > 4. The KIP will document that log4j2 is the only officially > >> supported > >> > > server-side logging framework, and we will expose its > configuration > >> > > file to > >> > > users. > >> > > To avoid breaking downstream compatibility, we will not bind the > >> > client > >> > > side to any specific logging framework. Users will need to manage > >> > their > >> > > own > >> > > logging libraries, but they can utilize the `-Dslf4j.provider` > >> > property > >> > > once slf4j is upgraded. > >> > > 5. We have rejected alternatives that involve warnings and > >> classpath > >> > > ordering as they do not provide a solid solution to compatibility > >> > > issues. > >> > > > >> > > Does this summary make sense? > >> > > > >> > > Best, > >> > > TengYao > >> > > > >> > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月21日 週五 上午9:32寫道： > >> > > > >> > > > A solution that involves a warning and classpath ordering doesn't > >> meet > >> > > the > >> > > > bar for me. Good clarification though. > >> > > > > >> > > > Ismael > >> > > > > >> > > > On Thu, Mar 20, 2025 at 8:37 AM Farid Zakaria > >> > > > <fzaka...@confluent.io.invalid> > >> > > > wrote: > >> > > > > >> > > > > AFAIR SLF4J you don't have to remove the other backends; merely > >> make > >> > > > > sure yours is first on the CLASSPATH list :P > >> > > > > (SLF4J pre 2.0 would always emit a warning that it found 2+ > >> > > > StaticBinders) > >> > > > > > >> > > > > Interestingly, you could still whatever backend (i.e. Log4J) and > >> pipe > >> > > > > it through to another backend via another appender. > >> > > > > This is what SLF4J refers to bridges -- although you have to be > >> sure > >> > > > > not to create a circular loop. > >> > > > > > >> > > > > Then there is something also general like syslog. > >> > > > > > >> > > > > On Thu, Mar 20, 2025 at 1:15 AM Chia-Ping Tsai < > >> chia7...@gmail.com> > >> > > > wrote: > >> > > > > > > >> > > > > > hi Ismael > >> > > > > > > >> > > > > > thanks for all your response. > >> > > > > > > >> > > > > > > All that said, I am not actually sure we can do what I > >> described > >> > > > above > >> > > > > > while maintaining the compatibility required by a minor > release. > >> > > > > > > >> > > > > > Excuse me, are your concerns related to version matching, as > >> > > discussed > >> > > > in > >> > > > > > [0]? If so, I concur that this represents a compatibility > issue, > >> > and > >> > > > the > >> > > > > > target version for this change should be 5.0. Additionally, > >> there > >> > > was a > >> > > > > > related discussion previously documented in [1]. While we have > >> not > >> > > > > strictly > >> > > > > > enforced version matching during prior SLF4J updates, this KIP > >> > > provides > >> > > > > an > >> > > > > > opportunity to establish guidelines for upgrading sl4fj that > >> have > >> > > > direct > >> > > > > > compatibility implications. > >> > > > > > > >> > > > > > [0] https://www.slf4j.org/faq.html#compatibility > >> > > > > > [1] > >> > > https://github.com/apache/kafka/pull/16324#discussion_r1644671854 > >> > > > > > > >> > > > > > To Teng > >> > > > > > > >> > > > > > Could you please revise the KIP according to following > benefit? > >> > > > > > > >> > > > > > > The key benefit of this KIP is that you can add new logging > >> > > backends > >> > > > > and > >> > > > > > select it via a config. This is how most pluggable things > work. > >> But > >> > > it > >> > > > is > >> > > > > > *not* how slf4j 1.x works. slf4j 1.x requires you to *remove* > >> the > >> > > > default > >> > > > > > logging library picked by the project as well. That's much > more > >> > > > > intrusive. > >> > > > > > > >> > > > > > Best, > >> > > > > > Chia-Ping > >> > > > > > > >> > > > > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月20日 週四 上午8:10寫道： > >> > > > > > > >> > > > > > > Hi Chia-Ping, > >> > > > > > > > >> > > > > > > I don't think we're in the business of shipping multiple > >> logging > >> > > > > libraries. > >> > > > > > > Here's my take: > >> > > > > > > > >> > > > > > > 1. We should pick one logging library for services/servers, > >> ship > >> > it > >> > > > and > >> > > > > > > include a configuration file for it. > >> > > > > > > 2. For clients, we leave it to the users to select the > logging > >> > > > library > >> > > > > - > >> > > > > > > clients run alongside applications and it's desirable to use > >> the > >> > > same > >> > > > > > > logging library for both (if we were starting from scratch, > we > >> > may > >> > > > have > >> > > > > > > decided to also include our default logging library for > >> clients > >> > as > >> > > > > well, > >> > > > > > > but it's hard to make that change now). > >> > > > > > > 3. For the cases where users want to use a different logging > >> > > library > >> > > > > for > >> > > > > > > services/servers (perhaps because they have standardized on > a > >> > > > different > >> > > > > > > logging library), they would have to add the additional jar > to > >> > the > >> > > > > > > classpath and change the relevant logging config. This is no > >> > > > different > >> > > > > than > >> > > > > > > adding a different authorizer or any other pluggable > >> component. > >> > > > > > > > >> > > > > > > The key benefit of this KIP is that you can add new logging > >> > > backends > >> > > > > and > >> > > > > > > select it via a config. This is how most pluggable things > >> work. > >> > But > >> > > > it > >> > > > > is > >> > > > > > > *not* how slf4j 1.x works. slf4j 1.x requires you to > *remove* > >> the > >> > > > > default > >> > > > > > > logging library picked by the project as well. That's much > >> more > >> > > > > intrusive. > >> > > > > > > > >> > > > > > > All that said, I am not actually sure we can do what I > >> described > >> > > > above > >> > > > > > > while maintaining the compatibility required by a minor > >> release. > >> > > > > > > > >> > > > > > > Ismael > >> > > > > > > > >> > > > > > > On Wed, Mar 19, 2025, 2:03 PM Chia-Ping Tsai < > >> chia7...@gmail.com > >> > > > >> > > > > wrote: > >> > > > > > > > >> > > > > > > > hi Ismael > >> > > > > > > > > >> > > > > > > > > but they must also add whichever logging library they > >> want to > >> > > > use. > >> > > > > > > > > >> > > > > > > > If users are required to modify JAR files to alter the > SLF4J > >> > > > > provider, > >> > > > > > > the > >> > > > > > > > value of this KIP is significantly diminished. I believe > the > >> > > > primary > >> > > > > > > > benefit of this KIP lies in enabling users to configure a > >> > system > >> > > > > property > >> > > > > > > > for SLF4J provider changes without JAR modifications. > >> > > Furthermore, > >> > > > by > >> > > > > > > > managing all SLF4J and provider JARs within Kafka, we can > >> > ensure > >> > > > > SLF4J > >> > > > > > > > version updates without compatibility concerns, as we can > >> > > guarantee > >> > > > > > > > provider JAR consistency with the SLF4J version in the > >> > > > distribution. > >> > > > > > > > > >> > > > > > > > Best, > >> > > > > > > > Chia-Ping > >> > > > > > > > > >> > > > > > > > Ismael Juma <m...@ismaeljuma.com> 於 2025年3月19日 週三 > 下午12:00寫道： > >> > > > > > > > > >> > > > > > > > > Hi TengYao, > >> > > > > > > > > > >> > > > > > > > > It's a bit difficult to review the KIP. I don't follow > >> most > >> > of > >> > > > the > >> > > > > > > > > motivation. The only one that I follow is: > >> > > > > > > > > > >> > > > > > > > > "Our current build configuration employs fragile > >> dependency > >> > > > > management > >> > > > > > > > > tricks to handle SLF4J backends. We can eliminate these > >> > brittle > >> > > > > build > >> > > > > > > > > mechanisms by transitioning to explicit provider > >> dependencies > >> > > > after > >> > > > > > > > > upgrading to 2.0." > >> > > > > > > > > > >> > > > > > > > > Additionally, I don't think we should do the following: > >> > > > > > > > > > >> > > > > > > > > "Add other popular slf4j backend binding provider > >> > > dependencies." > >> > > > > > > > > > >> > > > > > > > > A reasonable approach, in my opinion, would be: > >> > > > > > > > > > >> > > > > > > > > 1. Include the log4j2 dependency with the server modules > >> and > >> > > not > >> > > > > > > include > >> > > > > > > > > them with the client modules. > >> > > > > > > > > 2. Automatically configure the log4j2 dependency for the > >> > server > >> > > > > > > modules. > >> > > > > > > > > Users can override them via the system property, but > they > >> > must > >> > > > > also add > >> > > > > > > > > whichever logging library they want to use. > >> > > > > > > > > 3. Somehow configure slf4j 2.x to work like 1.x out of > the > >> > box > >> > > > for > >> > > > > the > >> > > > > > > > > client module (for compatibility reasons). > >> > > > > > > > > > >> > > > > > > > > But I don't know if `3` is possible. If `3` is not > >> possible, > >> > I > >> > > > > don't > >> > > > > > > see > >> > > > > > > > > how we can make this a compatible change. > >> > > > > > > > > > >> > > > > > > > > Ismael > >> > > > > > > > > > >> > > > > > > > > On Tue, Mar 18, 2025 at 7:41 PM TengYao Chi < > >> > > kiting...@gmail.com > >> > > > > > >> > > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > > > Hello everyone, > >> > > > > > > > > > > >> > > > > > > > > > I want to bump this thread manually. > >> > > > > > > > > > Any feedback or vote would be appreciated. > >> > > > > > > > > > > >> > > > > > > > > > Best, > >> > > > > > > > > > TengYao > >> > > > > > > > > > > >> > > > > > > > > > TengYao Chi <kiting...@gmail.com> 於 2025年3月10日 週一 > >> > 上午11:47寫道： > >> > > > > > > > > > > >> > > > > > > > > > > Hello guys, > >> > > > > > > > > > > I would like to remind you in the vote thread that > the > >> > KIP > >> > > > has > >> > > > > been > >> > > > > > > > > > > updated, and I apologize for repeating it. > >> > > > > > > > > > > I have taken over this KIP from Muralidhar. > >> > > > > > > > > > > Since the original content is outdated as the > logging > >> > > > > framework has > >> > > > > > > > > been > >> > > > > > > > > > > widely changed, I have updated the content of the > KIP. > >> > > > > > > > > > > Please take a look and share your thoughts. > >> > > > > > > > > > > > >> > > > > > > > > > > Best Regards, > >> > > > > > > > > > > TengYao > >> > > > > > > > > > > > >> > > > > > > > > > > Muralidhar Basani <muralidhar.bas...@aiven.io > .invalid> > >> 於 > >> > > > > > > 2024年9月24日 > >> > > > > > > > 週二 > >> > > > > > > > > > > 上午5:09寫道： > >> > > > > > > > > > > > >> > > > > > > > > > >> Hi, > >> > > > > > > > > > >> > >> > > > > > > > > > >> I wanted to gently follow up on this thread in case > >> > anyone > >> > > > > has any > >> > > > > > > > > > >> thoughts > >> > > > > > > > > > >> or would like to take a look. > >> > > > > > > > > > >> > >> > > > > > > > > > >> Thanks, > >> > > > > > > > > > >> Murali > >> > > > > > > > > > >> > >> > > > > > > > > > >> On Mon, Sep 16, 2024 at 10:23 AM Muralidhar Basani > < > >> > > > > > > > > > >> muralidhar.bas...@aiven.io> wrote: > >> > > > > > > > > > >> > >> > > > > > > > > > >> > Thanks Chia. > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > I have updated KIP with this quote, in the > >> migration > >> > > plan > >> > > > > > > section. > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > Thanks, > >> > > > > > > > > > >> > Murali > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > On Sun, Sep 15, 2024 at 3:30 PM Chia-Ping Tsai < > >> > > > > > > > chia7...@gmail.com> > >> > > > > > > > > > >> wrote: > >> > > > > > > > > > >> > > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > Muralidhar Basani <muralidhar.bas...@aiven.io > >> > > .invalid> > >> > > > 於 > >> > > > > > > > > > 2024年9月15日 > >> > > > > > > > > > >> >> 晚上9:02 寫道： > >> > > > > > > > > > >> >> > > >> > > > > > > > > > >> >> > With this, I think, users don't have to make > any > >> > > > explicit > >> > > > > > > > changes > >> > > > > > > > > > in > >> > > > > > > > > > >> >> their > >> > > > > > > > > > >> >> > code, if their provider is reload4j. And if > >> it's a > >> > > > > different > >> > > > > > > > > > provider > >> > > > > > > > > > >> >> (like > >> > > > > > > > > > >> >> > logback, log4j), they would have to upgrade > >> that to > >> > > > > match it > >> > > > > > > > with > >> > > > > > > > > > >> slf4j. > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> If upgrading the matched provider is the only > >> > explicit > >> > > > > change > >> > > > > > > and > >> > > > > > > > > we > >> > > > > > > > > > >> >> expect users have responsibility to keep > >> consistent > >> > > > version > >> > > > > > > when > >> > > > > > > > > > using > >> > > > > > > > > > >> >> other providers , could we write it down to the > >> KIP? > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> That means we will update slf4j without KIP in > the > >> > > future > >> > > > > > > except > >> > > > > > > > > for > >> > > > > > > > > > >> >> specific reason. > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> Best, > >> > > > > > > > > > >> >> Chia-Ping > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > > >> > > > > > > > > > >> > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
