On Wed, Jan 12, 2022, at 02:37, Viktor Somogyi-Vass wrote: > Hi Dongjin, > > We're also looking into this matter as our stack was also affected by all > the log4j hell and users increasingly pushing us to upgrade to log4j2 or > logback because of the existing vulnerabilities in log4j1. > Regarding the points raised by Haruki: >
I've heard the same thing from other people -- that there is now more interest in moving from log4j1.x to log4j2.x, after all the recent vulnerabilities in the latter. I found this a bit strange. Kafka avoided all the log4shell vulnerabilities exactly because we DIDN'T move to log4j 2.x. (Yes, I am aware that there is a longstanding vulnerability in that one log sink in log4j 1.x, but you can just not use that one.) I haven't thought about this very hard. Maybe it's still a good idea to move to log4j2. But it's odd that nobody is commenting about how in this case, not updating actually prevented a major security incident for Kafka. best, Colin > > a) In my opinion the best would be to make the dynamic logger support > (Log4jController and LoggingResource) pluggable for log4j2 and logback (so > an interface could be used to define the dynamic logging control methods > and a config to specify the implementation). That way we're not bound to > either logback or log4j and seems like a low-effort thing to do. > Additionally this could be used in Connect too in LoggingResource. > > b) I think testing dependencies aren't that important from the user > perspective, it's fine to either use log4j2 or logback, whichever is > easier. Kafka is either used from the distribution (tgz) or pulled in > through maven, but test dependencies shouldn't be exposed to the world. > > c) I would support deprecating the appender in favor of the log4j2 Kafka > appender. VerifiableLog4jAppender is intended as a testing tool anyway, so > I think it's less important to change this to logback. > > Future vulnerabilities will always be found in either logback or log4j2 or > any other logging framework, so I think the safest approach is to allow > users to choose their implementation, while in tests I think we're free to > use whatever we want as that shouldn't be constrained by vulnerabilities. > > Viktor > > On Thu, Dec 23, 2021 at 9:37 AM Haruki Okada <ocadar...@gmail.com> wrote: > >> Thanks for the clarification. >> >> About 2, I wan't aware of those concerns. >> Let me check them first. >> >> >> Thanks, >> >> 2021年12月23日(木) 13:37 Dongjin Lee <dong...@apache.org>: >> >> > Hi Haruki, >> > >> > >> > Thanks for organizing the issue. >> > >> > >> > If the community prefers logback, I will gladly change the dependency and >> > update the PR. However, it has the following issues: >> > >> > >> > 1. The log4j2 vulnerabilities seem mostly fixed, and KIP-653 + KIP-719 >> are >> > not released yet. So, using log4j2 (whose recent update pace is so high) >> > will not affect the users. >> > >> > >> > 2. To switch to logback, the following features should be reworked: >> > >> > >> > a. Dynamic logger level configuration (core, connect) >> > >> > b. Logging tests (streams) >> > >> > c. Kafka Appender (tools) >> > >> > >> > a and b are the most challenging ones since there is little documentation >> > on how to do this, so it requires analyzing the implementation itself. >> > (what I actually did with log4j2) About c, logback does not provide a >> Kafka >> > Appender so we have to provide an equivalent. >> > >> > >> > It is why I prefer to use log4j2. How do you think? >> > >> > >> > Thanks, >> > >> > Dongjin >> > >> > >> > On Thu, Dec 23, 2021 at 9:01 AM Haruki Okada <ocadar...@gmail.com> >> wrote: >> > >> > > Hi, Dongjin, >> > > >> > > Sorry for interrupting the discussion. >> > > And thank you for your hard work about KIP-653, KIP-719. >> > > >> > > I understand that KIP-653 is already accepted so log4j2 is the choice >> of >> > > the Kafka community though, I'm now feeling that logback is a better >> > choice >> > > here. >> > > >> > > Reasons: >> > > >> > > - even after "log4shell", several vulnerabilities found on log4j2 so >> new >> > > versions are released and users have to update in high-pace >> > > * actually, a CVE was also reported for logback (CVE-2021-42550) >> but >> > it >> > > requires edit-permission of the config file for an attacker so it's >> much >> > > less threatening >> > > - log4j1.x and logback are made by same developer (ceki), so >> > substantially >> > > the successor of log4j1 is logback rather than log4j2 >> > > - in Hadoop project, seems similar suggestion was made from a PMC >> > > * https://issues.apache.org/jira/browse/HADOOP-12956 >> > > >> > > >> > > What do you think about adopting logback instead? >> > > >> > > >> > > Thanks, >> > > >> > > 2021年12月21日(火) 18:02 Dongjin Lee <dong...@apache.org>: >> > > >> > > > Hi Mickael, >> > > > >> > > > > In the meantime, you may want to bump the VOTE thread too. >> > > > >> > > > Sure, I just reset the voting thread with a brief context. >> > > > >> > > > Thanks, >> > > > Dongjin >> > > > >> > > > On Tue, Dec 21, 2021 at 2:13 AM Mickael Maison < >> > mickael.mai...@gmail.com >> > > > >> > > > wrote: >> > > > >> > > > > Thanks Dongjin! >> > > > > >> > > > > I'll take a look soon. >> > > > > In the meantime, you may want to bump the VOTE thread too. >> > > > > >> > > > > Best, >> > > > > Mickael >> > > > > >> > > > > >> > > > > On Sat, Dec 18, 2021 at 10:00 AM Dongjin Lee <dong...@apache.org> >> > > wrote: >> > > > > > >> > > > > > Hi Mickael, >> > > > > > >> > > > > > Finally, I did it! As you can see at the PR >> > > > > > <https://github.com/apache/kafka/pull/10244>, KIP-719 now uses >> > > > log4j2's >> > > > > > Kafka appender, and log4j-appender is not used by the other >> modules >> > > > > > anymore. You can see how it will work with KIP-653 at this >> preview >> > > > > > < >> http://home.apache.org/~dongjin/post/apache-kafka-log4j2-support/ >> > >, >> > > > > based >> > > > > > on Apache Kafka 3.0.0. The proposal document >> > > > > > < >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Deprecate+Log4J+Appender >> > > > > > >> > > > > > is also updated accordingly, with its title. >> > > > > > >> > > > > > There is a minor issue on log4j2 >> > > > > > <https://issues.apache.org/jira/browse/LOG4J2-3256>, but it >> seems >> > > like >> > > > > it >> > > > > > will be resolved soon. >> > > > > > >> > > > > > Best, >> > > > > > Dongjin >> > > > > > >> > > > > > On Wed, Dec 15, 2021 at 9:28 PM Dongjin Lee <dong...@apache.org> >> > > > wrote: >> > > > > > >> > > > > > > Hi Mickael, >> > > > > > > >> > > > > > > > Can we do step 3 without breaking any compatibility? If so >> then >> > > > that >> > > > > > > sounds like a good idea. >> > > > > > > >> > > > > > > As far as I know, the answer is yes; I am now updating my PR, >> so >> > I >> > > > will >> > > > > > > notify you as soon as I complete the work. >> > > > > > > >> > > > > > > Best, >> > > > > > > Dongjin >> > > > > > > >> > > > > > > On Wed, Dec 15, 2021 at 2:00 AM Mickael Maison < >> > > > > mickael.mai...@gmail.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > >> Hi Dongjin, >> > > > > > >> >> > > > > > >> Sorry for the late reply. Can we do step 3 without breaking >> any >> > > > > > >> compatibility? If so then that sounds like a good idea. >> > > > > > >> >> > > > > > >> Thanks, >> > > > > > >> Mickael >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> On Tue, Nov 23, 2021 at 2:08 PM Dongjin Lee < >> dong...@apache.org >> > > >> > > > > wrote: >> > > > > > >> > >> > > > > > >> > Hi Mickael, >> > > > > > >> > >> > > > > > >> > I also thought over the issue thoroughly and would like to >> > > > propose a >> > > > > > >> minor >> > > > > > >> > change to your proposal: >> > > > > > >> > >> > > > > > >> > 1. Deprecate log4j-appender now >> > > > > > >> > 2. Document how to migrate into logging-log4j2 >> > > > > > >> > 3. (Changed) Replace the log4j-appender (in turn log4j 1.x) >> > > > > > >> dependencies in >> > > > > > >> > tools, trogdor, and shell and upgrade to log4j2 in 3.x, >> > removing >> > > > > log4j >> > > > > > >> 1.x >> > > > > > >> > dependencies. >> > > > > > >> > 4. (Changed) Remove log4j-appender in Kafka 4.0 >> > > > > > >> > >> > > > > > >> > What we need to do for the log4j2 upgrade is just removing >> the >> > > > log4j >> > > > > > >> > dependencies only, for they can cause a classpath error. And >> > > > > actually, >> > > > > > >> we >> > > > > > >> > can do it without discontinuing publishing the >> log4j-appender >> > > > > artifact. >> > > > > > >> So, >> > > > > > >> > I suggest separating the upgrade to log4j2 and removing the >> > > > > > >> log4j-appender >> > > > > > >> > module. >> > > > > > >> > >> > > > > > >> > How do you think? If you agree, I will update the KIP and >> the >> > PR >> > > > > > >> > accordingly ASAP. >> > > > > > >> > >> > > > > > >> > Thanks, >> > > > > > >> > Dongjin >> > > > > > >> > >> > > > > > >> > On Mon, Nov 15, 2021 at 8:06 PM Mickael Maison < >> > > > > > >> mickael.mai...@gmail.com> >> > > > > > >> > wrote: >> > > > > > >> > >> > > > > > >> > > Hi Dongjin, >> > > > > > >> > > >> > > > > > >> > > Thanks for the clarifications. >> > > > > > >> > > >> > > > > > >> > > I wonder if a simpler course of action could be: >> > > > > > >> > > - Deprecate log4j-appender now >> > > > > > >> > > - Document how to use logging-log4j2 >> > > > > > >> > > - Remove log4j-appender and all the log4j dependencies in >> > > Kafka >> > > > > 4.0 >> > > > > > >> > > >> > > > > > >> > > This delays KIP-653 till Kafka 4.0 but (so far) Kafka is >> not >> > > > > directly >> > > > > > >> > > affected by the log4j CVEs. At least this gives us a clear >> > and >> > > > > simple >> > > > > > >> > > roadmap to follow. >> > > > > > >> > > >> > > > > > >> > > What do you think? >> > > > > > >> > > >> > > > > > >> > > On Tue, Nov 9, 2021 at 12:12 PM Dongjin Lee < >> > > dong...@apache.org >> > > > > >> > > > > > >> wrote: >> > > > > > >> > > > >> > > > > > >> > > > Hi Mickael, >> > > > > > >> > > > >> > > > > > >> > > > I greatly appreciate you for reading the proposal so >> > > > carefully! >> > > > > I >> > > > > > >> wrote >> > > > > > >> > > it >> > > > > > >> > > > quite a while ago and rechecked it today. >> > > > > > >> > > > >> > > > > > >> > > > > Is the KIP proposing to replace the existing >> > log4-appender >> > > > or >> > > > > > >> simply >> > > > > > >> > > add >> > > > > > >> > > > a new one for log4j2? Reading the KIP and with its >> current >> > > > > title, >> > > > > > >> it's >> > > > > > >> > > not >> > > > > > >> > > > entirely explicit. >> > > > > > >> > > > >> > > > > > >> > > > Oh, After re-reading it, I realized that this is not >> > clear. >> > > > Let >> > > > > me >> > > > > > >> > > clarify; >> > > > > > >> > > > >> > > > > > >> > > > 1. Provide a lo4j2 equivalent of traditional >> > log4j-appender, >> > > > > > >> > > > log4j2-appender. >> > > > > > >> > > > 2. Migrate the modules depending on log4j-appender >> (i.e., >> > > > tools, >> > > > > > >> trogdor, >> > > > > > >> > > > shell) into log4j2-appender, removing log4j-appender >> from >> > > > > > >> dependencies. >> > > > > > >> > > > 3. Entirely remove log4j-appender from the project >> > > > dependencies, >> > > > > > >> along >> > > > > > >> > > with >> > > > > > >> > > > log4j. >> > > > > > >> > > > >> > > > > > >> > > > I think log4j-appender may be published for every new >> > > release >> > > > > like >> > > > > > >> > > before, >> > > > > > >> > > > but the committee should make a decision on the policy. >> > > > > > >> > > > >> > > > > > >> > > > > Under Rejected Alternative, the KIP states: "the Kafka >> > > > > appender >> > > > > > >> > > provided >> > > > > > >> > > > by log4j2 community stores log message in the Record >> key". >> > > > > Looking >> > > > > > >> at the >> > > > > > >> > > > code, it looks like the log message is stored in the >> > Record >> > > > > value: >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://github.com/apache/logging-log4j2/blob/master/log4j-kafka/src/main/java/org/apache/logging/log4j/kafka/appender/KafkaManager.java#L135 >> > > > > > >> > > > Am I missing something? >> > > > > > >> > > > >> > > > > > >> > > > It's totally my fault; I confused it with another >> > appender. >> > > > The >> > > > > > >> > > > compatibility problem in the logging-log4j2 Kafka >> appender >> > > is >> > > > > not >> > > > > > >> the >> > > > > > >> > > > format but the configuration. logging-log4j2 Kafka >> > appender >> > > > > supports >> > > > > > >> > > > `properties` configuration, which will be directly used >> to >> > > > > > >> instantiate a >> > > > > > >> > > > Kafka producer. However, log4j-appender has been using >> > > > > non-producer >> > > > > > >> > > config >> > > > > > >> > > > names like brokerList (=bootstrap.servers), >> > requiredNumAcks >> > > > > (=acks). >> > > > > > >> > > > Instead, logging-log4j2 Kafka appender supports >> > retryCount, >> > > > > > >> > > > sendEventTimestamp. >> > > > > > >> > > > >> > > > > > >> > > > On second thought, using logging-log4j2 Kafka appender >> > > > > internally >> > > > > > >> and >> > > > > > >> > > > making log4j2-appender to focus on compatibility facade >> > only >> > > > > would >> > > > > > >> be a >> > > > > > >> > > > better approach; As I described above, the goal of this >> > > module >> > > > > is >> > > > > > >> just >> > > > > > >> > > > keeping the backward-compatibility, and (as you pointed >> > out) >> > > > the >> > > > > > >> current >> > > > > > >> > > > implementation has little value. Since >> > > > > > >> > > org.apache.logging.log4j:log4j-core >> > > > > > >> > > > already includes Kafka appender, we can make use of the >> > > > 'proven >> > > > > > >> wheel' >> > > > > > >> > > > without adding more dependencies. I have not tried it >> yet, >> > > > but I >> > > > > > >> think it >> > > > > > >> > > > is well worth it. (One additional advantage of this >> > approach >> > > > is >> > > > > > >> > > providing a >> > > > > > >> > > > bridge to the users who hope to move from/into >> > > logging-log4j2 >> > > > > Kafka >> > > > > > >> > > > appender.) >> > > > > > >> > > > >> > > > > > >> > > > > As the current log4j-appender is not even deprecated >> > yet, >> > > in >> > > > > > >> theory we >> > > > > > >> > > > can't remove it till Kafka 4. If we want to speed up the >> > > > > process, I >> > > > > > >> > > wonder >> > > > > > >> > > > if the lack of documentation and a migration guide could >> > > help >> > > > > us. >> > > > > > >> What do >> > > > > > >> > > > you think? >> > > > > > >> > > > >> > > > > > >> > > > In fact, this is what I am doing nowadays. While working >> > > with >> > > > > > >> > > > log4j-appender, I found that despite a lack of >> > > documentation, >> > > > > > >> > > considerable >> > > > > > >> > > > users are already using it[^1][^2][^3][^4][^5]. So, I >> > think >> > > > > > >> providing a >> > > > > > >> > > > documentation to those who are already using >> > log4j-appender >> > > is >> > > > > > >> > > > indispensable. It should include: >> > > > > > >> > > > >> > > > > > >> > > > - What is the difference between log4j-appender vs. >> > > > > log4j2-appender. >> > > > > > >> > > > - Which options are supported and deprecated. >> > > > > > >> > > > - Exemplar configurations that show how to migrate. >> > > > > > >> > > > >> > > > > > >> > > > Here is the summary: >> > > > > > >> > > > >> > > > > > >> > > > 1. The goal of this proposal is to replace the >> traditional >> > > > > > >> log4j-appender >> > > > > > >> > > > for compatibility concerns. But log4j-appender may be >> > > > published >> > > > > > >> after the >> > > > > > >> > > > deprecation. >> > > > > > >> > > > 2. As of present, the description about logging-log4j2 >> > Kafka >> > > > > > >> appender is >> > > > > > >> > > > entirely wrong. The problem is interface compatibility, >> > not >> > > > > record >> > > > > > >> > > format. >> > > > > > >> > > > Focusing on the compatibility facade is a good approach. >> > > > > > >> > > > 3. A documentation focus on migration should be >> provided. >> > > > > > >> > > > >> > > > > > >> > > > If you have any questions or suggestions, don't hesitate >> > to >> > > > > tell me. >> > > > > > >> > > Thanks >> > > > > > >> > > > again for your comments! >> > > > > > >> > > > >> > > > > > >> > > > Best, >> > > > > > >> > > > Dongjin >> > > > > > >> > > > >> > > > > > >> > > > [^1]: >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://docs.cloudera.com/csa/1.2.0/monitoring/topics/csa-kafka-logging.html >> > > > > > >> > > > [^2]: >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://stackoverflow.com/questions/22034895/how-to-use-kafka-0-8-log4j-appender >> > > > > > >> > > > [^3]: >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://stackoverflow.com/questions/32402405/delay-in-kafka-log4j-appender >> > > > > > >> > > > [^4]: >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://stackoverflow.com/questions/32301129/kafka-log4j-appender-not-sending-messages >> > > > > > >> > > > [^5]: >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://stackoverflow.com/questions/35628706/kafka-log4j-appender-0-9-does-not-work >> > > > > > >> > > > >> > > > > > >> > > > On Mon, Nov 8, 2021 at 9:04 PM Mickael Maison < >> > > > > > >> mickael.mai...@gmail.com> >> > > > > > >> > > > wrote: >> > > > > > >> > > > >> > > > > > >> > > > > Hi Dongjin, >> > > > > > >> > > > > >> > > > > > >> > > > > Thanks for working on the update to log4j2, it's >> > > > definitively >> > > > > > >> > > > > something we should complete. >> > > > > > >> > > > > I have a couple of comments: >> > > > > > >> > > > > >> > > > > > >> > > > > 1) Is the KIP proposing to replace the existing >> > > > log4-appender >> > > > > or >> > > > > > >> > > > > simply add a new one for log4j2? Reading the KIP and >> > with >> > > > its >> > > > > > >> current >> > > > > > >> > > > > title, it's not entirely explicit. For example I don't >> > > see a >> > > > > > >> statement >> > > > > > >> > > > > under the proposed changes section. The PR seems to >> only >> > > add >> > > > > a new >> > > > > > >> > > > > appender but the KIP mentions we want to fully remove >> > > > > > >> dependencies to >> > > > > > >> > > > > log4j. >> > > > > > >> > > > > >> > > > > > >> > > > > 2) Under Rejected Alternative, the KIP states: "the >> > Kafka >> > > > > appender >> > > > > > >> > > > > provided by log4j2 community stores log message in the >> > > > Record >> > > > > > >> key". >> > > > > > >> > > > > Looking at the code, it looks like the log message is >> > > stored >> > > > > in >> > > > > > >> the >> > > > > > >> > > > > Record value: >> > > > > > >> > > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://github.com/apache/logging-log4j2/blob/master/log4j-kafka/src/main/java/org/apache/logging/log4j/kafka/appender/KafkaManager.java#L135 >> > > > > > >> > > > > Am I missing something? >> > > > > > >> > > > > Comparing it with the proposed new appender, apart >> from >> > > > their >> > > > > > >> > > > > configuration format (hence the backwards >> compatibility >> > > > > issues), >> > > > > > >> they >> > > > > > >> > > > > both work pretty much the same way, so it's not clear >> it >> > > > would >> > > > > > >> add a >> > > > > > >> > > > > ton a value. >> > > > > > >> > > > > >> > > > > > >> > > > > At a glance, _I've not extensively looked at it_, it >> > does >> > > > not >> > > > > look >> > > > > > >> > > > > very hard to migrate to the appender from the logging >> > > team. >> > > > I >> > > > > was >> > > > > > >> > > > > wondering if we should mention it in our documentation >> > > but I >> > > > > was >> > > > > > >> not >> > > > > > >> > > > > able to find any references to the log4j-appender in >> the >> > > > Kafka >> > > > > > >> docs: >> > > > > > >> > > > > >> > > > > https://github.com/apache/kafka-site/search?q=KafkaLog4jAppender >> > > > > > >> > > > > >> > > > > > >> > > > > As the current log4j-appender is not even deprecated >> > yet, >> > > in >> > > > > > >> theory we >> > > > > > >> > > > > can't remove it till Kafka 4. If we want to speed up >> the >> > > > > process, >> > > > > > >> I >> > > > > > >> > > > > wonder if the lack of documentation and a migration >> > guide >> > > > > could >> > > > > > >> help >> > > > > > >> > > > > us. What do you think? >> > > > > > >> > > > > >> > > > > > >> > > > > Thanks, >> > > > > > >> > > > > Mickael >> > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > On Fri, Jun 11, 2021 at 4:57 PM Boojapho O < >> > > > > booja...@gmail.com> >> > > > > > >> wrote: >> > > > > > >> > > > > > >> > > > > > >> > > > > > Continuing to use log4j would leave several known >> > > security >> > > > > > >> > > > > vulnerabilities in Apache Kafka, including >> > > > > > >> > > > > https://nvd.nist.gov/vuln/detail/CVE-2019-17571. The >> > > > Apache >> > > > > > >> log4j >> > > > > > >> > > team >> > > > > > >> > > > > will not fix this vulnerability and is urging an >> upgrade >> > > to >> > > > > > >> log4j2. >> > > > > > >> > > See >> > > > > > >> > > > > https://logging.apache.org/log4j/1.2/ for further >> > > > > information. >> > > > > > >> > > > > > >> > > > > > >> > > > > > This is desperately needed in Apache 3.0 to keep the >> > > > > software >> > > > > > >> secure. >> > > > > > >> > > > > > >> > > > > > >> > > > > > On 2021/05/26 12:31:20, Dongjin Lee < >> > dong...@apache.org >> > > > >> > > > > wrote: >> > > > > > >> > > > > > > CC'd the +1ers of KIP-653 with detailed context: >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > When I submitted and got the approval of KIP-653: >> > > > Upgrade >> > > > > > >> log4j to >> > > > > > >> > > > > log4j2 >> > > > > > >> > > > > > > < >> > > > > > >> > > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-653%3A+Upgrade+log4j+to+log4j2 >> > > > > > >> > > > > >, >> > > > > > >> > > > > > > I thought the log4j2-appender should not be the >> > scope >> > > of >> > > > > the >> > > > > > >> work. >> > > > > > >> > > But >> > > > > > >> > > > > it >> > > > > > >> > > > > > > was wrong. >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > Since the VerifiableLog4jAppender tool is built >> upon >> > > > > > >> > > log4j-appender, >> > > > > > >> > > > > log4j >> > > > > > >> > > > > > > 1.x artifact will co-exist with log4j2 artifact in >> > the >> > > > > > >> classpath >> > > > > > >> > > within >> > > > > > >> > > > > > > this scheme. Since the log4j 1.x code is not >> called >> > > > > anymore, I >> > > > > > >> > > thought >> > > > > > >> > > > > it >> > > > > > >> > > > > > > is not problematic but actually, it was not - >> when I >> > > > > started >> > > > > > >> to >> > > > > > >> > > > > provide a >> > > > > > >> > > > > > > preview of KIP-653 >> > > > > > >> > > > > > > < >> > > > > > >> >> > http://home.apache.org/~dongjin/post/apache-kafka-log4j2-support/ >> > > > > > >> > > >, >> > > > > > >> > > > > some >> > > > > > >> > > > > > > users reported that sometimes slf4j fails to find >> > the >> > > > > > >> appropriate >> > > > > > >> > > > > binding >> > > > > > >> > > > > > > within the classpath, resulting fail to append the >> > log >> > > > > > >> message. >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > To resolve this problem, I subtly adjusted the >> scope >> > > of >> > > > > the >> > > > > > >> work; I >> > > > > > >> > > > > > > excluded Tools and Trogdor from KIP-653 and >> extended >> > > > > KIP-719 >> > > > > > >> to >> > > > > > >> > > take >> > > > > > >> > > > > care >> > > > > > >> > > > > > > of them instead, along with providing >> > log4j2-appender. >> > > > It >> > > > > is >> > > > > > >> why >> > > > > > >> > > the >> > > > > > >> > > > > > > current WIP implementations include some classpath >> > > logic >> > > > > in >> > > > > > >> the >> > > > > > >> > > shell >> > > > > > >> > > > > > > script and *why KIP-653 only can't complete the >> > log4j2 >> > > > > > >> migration*. >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > I hope you will check this proposal out. >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > Best, >> > > > > > >> > > > > > > Dongjin >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > On Tue, May 25, 2021 at 10:43 PM Dongjin Lee < >> > > > > > >> dong...@apache.org> >> > > > > > >> > > > > wrote: >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > > Bumping up the discussion thread. >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > Recently, I updated the document of KIP-653: >> > Upgrade >> > > > > log4j >> > > > > > >> to >> > > > > > >> > > log4j2 >> > > > > > >> > > > > > > > < >> > > > > > >> > > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-653%3A+Upgrade+log4j+to+log4j2 >> > > > > > >> > > > >> > > > > > >> > > > > (accepted) >> > > > > > >> > > > > > > > and KIP-719: Add Log4J2 Appender >> > > > > > >> > > > > > > > < >> > > > > > >> > > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Add+Log4J2+Appender >> > > > > > >> > > > >> > > > > > >> > > > > (under >> > > > > > >> > > > > > > > discussion) reflecting the recent changes to our >> > > > > codebase. >> > > > > > >> > > > > Especially: >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > 1. KIP-653 document >> > > > > > >> > > > > > > > < >> > > > > > >> > > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-653%3A+Upgrade+log4j+to+log4j2 >> > > > > > >> > > > >> > > > > > >> > > > > now >> > > > > > >> > > > > > > > explains which modules will be migrated and why. >> > > > > > >> > > > > > > > 2. KIP-719 document >> > > > > > >> > > > > > > > < >> > > > > > >> > > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Add+Log4J2+Appender >> > > > > > >> > > > >> > > > > > >> > > > > now >> > > > > > >> > > > > > > > explains not only the log4j2-appender plan but >> > also >> > > > > > >> upgrading the >> > > > > > >> > > > > omitted >> > > > > > >> > > > > > > > modules in KIP-653 into log4j2. >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > As you can see here, those two KIPs are the >> > > different >> > > > > parts >> > > > > > >> of >> > > > > > >> > > the >> > > > > > >> > > > > same >> > > > > > >> > > > > > > > problem. I believe the community will have a >> good >> > > > grasp >> > > > > on >> > > > > > >> why >> > > > > > >> > > both >> > > > > > >> > > > > KIPs >> > > > > > >> > > > > > > > are best if released altogether. >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > I will open the voting thread now, and please >> > leave >> > > a >> > > > > vote >> > > > > > >> if >> > > > > > >> > > you are >> > > > > > >> > > > > > > > interested in this issue. >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > Best, >> > > > > > >> > > > > > > > Dongjin >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > On Tue, Mar 2, 2021 at 5:00 PM Dongjin Lee < >> > > > > > >> dong...@apache.org> >> > > > > > >> > > > > wrote: >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> Hi Kafka dev, >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> I would like to start the discussion of >> KIP-719: >> > > Add >> > > > > Log4J2 >> > > > > > >> > > > > Appender. >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > >> > > > > > >> > > >> > > > > > >> >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Add+Log4J2+Appender >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> All kinds of feedbacks are greatly appreciated! >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> Best, >> > > > > > >> > > > > > > >> Dongjin >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> -- >> > > > > > >> > > > > > > >> *Dongjin Lee* >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> *A hitchhiker in the mathematical world.* >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > >> *github: <http://goog_969573159/> >> > > > > github.com/dongjinleekr >> > > > > > >> > > > > > > >> <https://github.com/dongjinleekr>keybase: >> > > > > > >> > > > > https://keybase.io/dongjinleekr >> > > > > > >> > > > > > > >> <https://keybase.io/dongjinleekr>linkedin: >> > > > > > >> > > > > kr.linkedin.com/in/dongjinleekr >> > > > > > >> > > > > > > >> <https://kr.linkedin.com/in/dongjinleekr >> > > > >speakerdeck: >> > > > > > >> > > > > speakerdeck.com/dongjin >> > > > > > >> > > > > > > >> <https://speakerdeck.com/dongjin>* >> > > > > > >> > > > > > > >> >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > -- >> > > > > > >> > > > > > > > *Dongjin Lee* >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > *A hitchhiker in the mathematical world.* >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > *github: <http://goog_969573159/> >> > > > > github.com/dongjinleekr >> > > > > > >> > > > > > > > <https://github.com/dongjinleekr>keybase: >> > > > > > >> > > > > https://keybase.io/dongjinleekr >> > > > > > >> > > > > > > > <https://keybase.io/dongjinleekr>linkedin: >> > > > > > >> > > > > kr.linkedin.com/in/dongjinleekr >> > > > > > >> > > > > > > > <https://kr.linkedin.com/in/dongjinleekr >> > > >speakerdeck: >> > > > > > >> > > > > speakerdeck.com/dongjin >> > > > > > >> > > > > > > > <https://speakerdeck.com/dongjin>* >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > -- >> > > > > > >> > > > > > > *Dongjin Lee* >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > *A hitchhiker in the mathematical world.* >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > *github: <http://goog_969573159/> >> > > > github.com/dongjinleekr >> > > > > > >> > > > > > > <https://github.com/dongjinleekr>keybase: >> > > > > > >> > > > > https://keybase.io/dongjinleekr >> > > > > > >> > > > > > > <https://keybase.io/dongjinleekr>linkedin: >> > > > > > >> > > > > kr.linkedin.com/in/dongjinleekr >> > > > > > >> > > > > > > <https://kr.linkedin.com/in/dongjinleekr >> > >speakerdeck: >> > > > > > >> > > > > speakerdeck.com/dongjin >> > > > > > >> > > > > > > <https://speakerdeck.com/dongjin>* >> > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > -- >> > > > > > >> > > > *Dongjin Lee* >> > > > > > >> > > > >> > > > > > >> > > > *A hitchhiker in the mathematical world.* >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > *github: <http://goog_969573159/> >> github.com/dongjinleekr >> > > > > > >> > > > <https://github.com/dongjinleekr>keybase: >> > > > > > >> > > https://keybase.io/dongjinleekr >> > > > > > >> > > > <https://keybase.io/dongjinleekr>linkedin: >> > > > > > >> > > kr.linkedin.com/in/dongjinleekr >> > > > > > >> > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: >> > > > > > >> > > speakerdeck.com/dongjin >> > > > > > >> > > > <https://speakerdeck.com/dongjin>* >> > > > > > >> > > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > -- >> > > > > > >> > *Dongjin Lee* >> > > > > > >> > >> > > > > > >> > *A hitchhiker in the mathematical world.* >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > *github: <http://goog_969573159/>github.com/dongjinleekr >> > > > > > >> > <https://github.com/dongjinleekr>keybase: >> > > > > > >> https://keybase.io/dongjinleekr >> > > > > > >> > <https://keybase.io/dongjinleekr>linkedin: >> > > > > > >> kr.linkedin.com/in/dongjinleekr >> > > > > > >> > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: >> > > > > > >> speakerdeck.com/dongjin >> > > > > > >> > <https://speakerdeck.com/dongjin>* >> > > > > > >> >> > > > > > > >> > > > > > > >> > > > > > > -- >> > > > > > > *Dongjin Lee* >> > > > > > > >> > > > > > > *A hitchhiker in the mathematical world.* >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr >> > > > > > > <https://github.com/dongjinleekr>keybase: >> > > > > https://keybase.io/dongjinleekr >> > > > > > > <https://keybase.io/dongjinleekr>linkedin: >> > > > > kr.linkedin.com/in/dongjinleekr >> > > > > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: >> > > > > speakerdeck.com/dongjin >> > > > > > > <https://speakerdeck.com/dongjin>* >> > > > > > > >> > > > > > >> > > > > > >> > > > > > -- >> > > > > > *Dongjin Lee* >> > > > > > >> > > > > > *A hitchhiker in the mathematical world.* >> > > > > > >> > > > > > >> > > > > > >> > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr >> > > > > > <https://github.com/dongjinleekr>keybase: >> > > > > https://keybase.io/dongjinleekr >> > > > > > <https://keybase.io/dongjinleekr>linkedin: >> > > > > kr.linkedin.com/in/dongjinleekr >> > > > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: >> > > > > speakerdeck.com/dongjin >> > > > > > <https://speakerdeck.com/dongjin>* >> > > > > >> > > > >> > > > >> > > > -- >> > > > *Dongjin Lee* >> > > > >> > > > *A hitchhiker in the mathematical world.* >> > > > >> > > > >> > > > >> > > > *github: <http://goog_969573159/>github.com/dongjinleekr >> > > > <https://github.com/dongjinleekr>keybase: >> > > https://keybase.io/dongjinleekr >> > > > <https://keybase.io/dongjinleekr>linkedin: >> > > kr.linkedin.com/in/dongjinleekr >> > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: >> > > > speakerdeck.com/dongjin >> > > > <https://speakerdeck.com/dongjin>* >> > > > >> > > >> > > >> > > -- >> > > ======================== >> > > Okada Haruki >> > > ocadar...@gmail.com >> > > ======================== >> > > >> > >> > >> > -- >> > *Dongjin Lee* >> > >> > *A hitchhiker in the mathematical world.* >> > >> > >> > >> > *github: <http://goog_969573159/>github.com/dongjinleekr >> > <https://github.com/dongjinleekr>keybase: >> https://keybase.io/dongjinleekr >> > <https://keybase.io/dongjinleekr>linkedin: >> kr.linkedin.com/in/dongjinleekr >> > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: >> > speakerdeck.com/dongjin >> > <https://speakerdeck.com/dongjin>* >> > >> >> >> -- >> ======================== >> Okada Haruki >> ocadar...@gmail.com >> ======================== >>
