Bumping up the voting thread for the 3.1.0 KIP freeze deadline.
As of present:
Binding: +1 (Konstantine)
Non-binding: +1 (Boojapho)
Again, without this feature, we can't entirely remove the log4j dependency
from the classpath - and it can cause a logging problem after the log4j2
upgrade provided by KIP-653. I found this problem while testing the preview
build.
I also updated the proposal document discussing the details of the problem
and why the other approaches were all rejected.
Thanks,
Dongjin.
On Mon, Jun 14, 2021 at 3:54 PM Dongjin Lee <dong...@apache.org> wrote:
> > VerifiableLog4jAppender is used for system tests, it's not a user facing
> tool. We don't need it to support log4j 2.
>
> I mean, as long as there is VerifiableLog4jAppender which uses
> log4j-appender, we can't entirely remove log4j 1.x artifact from the
> classpath, regardless of if it is a user-facing tool or not.
>
> As I wrote above, I found that some users with the preview build reported
> that if log4j 1.x and 2.x artifacts co-exist in the classpath, sometimes
> the slg4j can't find the appropriate binding, resulting in logging failing.
>
> Providing a log4j2 appender is not only for providing an updated one for
> the log4j-appender users, but for removing a potential problem in the
> classpath also.
>
> Regards,
> Dongjin
>
> On Mon, Jun 14, 2021 at 12:17 AM Ismael Juma <ism...@juma.me.uk> wrote:
>
>> VerifiableLog4jAppender is used for system tests, it's not a user facing
>> tool. We don't need it to support log4j 2.
>>
>> Ismael
>>
>> On Sun, Jun 13, 2021 at 8:12 AM Dongjin Lee <dong...@apache.org> wrote:
>>
>> > Hi Ismael,
>> >
>> > > Can't we work with the log4j community to support the alternative
>> format?
>> >
>> > It seems not.
>> >
>> > 1. Changing the format of current log4j2's Kafka Appender implementation
>> > means causing inconvenience to the existing users by changing API
>> > semantics. (In my opinion, the log4j community did not take the
>> > compatibility with the log4j-appender into account when they developed
>> this
>> > module.)
>> >
>> > 2. Providing a new implementation (which is compatible with
>> log4j-appender)
>> > alongside with the existing one means there will be two APIs with
>> similar
>> > goals and functionality under the same artifact coordinate of log4j.
>> This
>> > confuses the users.
>> >
>> > Either of the approaches above is not feasible to the log4j community.
>> >
>> > Moreover, this approach does not resolve the classpath problem; To
>> entirely
>> > remove the log4j 1.x artifact from the classpath, the log4j2 appender
>> > should be released first, since VerifiableLog4jAppender uses it. This
>> means
>> > that the release of Kafka 3.0 depends upon when the log4j community
>> > releases the appender - it is also not feasible for us.
>> >
>> > Thanks,
>> > Dongjin
>> >
>> > On Sun, Jun 13, 2021 at 12:20 AM Ismael Juma <ism...@juma.me.uk> wrote:
>> >
>> > > Can't we work with the log4j community to support the alternative
>> format?
>> > >
>> > > Ismael
>> > >
>> > > On Fri, Jun 11, 2021, 10:54 PM Dongjin Lee <dong...@apache.org>
>> wrote:
>> > >
>> > > > Hi all,
>> > > >
>> > > > @Ismael
>> > > >
>> > > > As I stated in the KIP (see subsection 2 of 'Motivation'), the log4j
>> > > > community's implementation can't be an alternative for the existing
>> > > > 'log4j-appender' users since their message format is different,
>> > breaking
>> > > > the related, already running pipelines.
>> > > >
>> > > > Add to this, the log4j appender can be best maintained when log4j2
>> and
>> > > > Kafka versions are updated together. (see subsection 1 of
>> > 'Motivation'.)
>> > > >
>> > > > @Israel
>> > > >
>> > > > Then, you mean instead of creating a new artifact
>> ('log4j2-appender'),
>> > > just
>> > > > substituting the traditional artifact ('log4j-appender') with a new
>> > > > implementation would be better. Do I understand correctly?
>> > > >
>> > > > After all, one main reason I hurried this proposal is that for the
>> > > > VerifiableLog4jAppender tool, we can't entirely remove log4j 1.x
>> > artifact
>> > > > from the classpath - making classpath logic more complex. (see here
>> > > > <
>> > > >
>> > >
>> >
>> https://github.com/apache/kafka/pull/7898/commits/f56f491e68ef2a976c0e3331a48dd881b74a06b3
>> > > > >
>> > > > for KIP-653
>> > > > <
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-653%3A+Upgrade+log4j+to+log4j2
>> > > > >
>> > > > and here
>> > > > <
>> > > >
>> > >
>> >
>> https://github.com/apache/kafka/pull/10244/commits/b66fce3d04e005b1eaeae006d78bd8e698f417c6
>> > > > >
>> > > > for KIP-719
>> > > > <
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Add+Log4J2+Appender
>> > > > >
>> > > > .)
>> > > >
>> > > > @Boojapho, @Konstantine
>> > > >
>> > > > Thanks for the voting. Currently:
>> > > >
>> > > > - binding: +1 (Konstantine)
>> > > > - non-binding: +1 (Boojapho)
>> > > >
>> > > > Regards,
>> > > > Dongjin
>> > > >
>> > > > On Sat, Jun 12, 2021 at 10:29 AM Israel Ekpo <israele...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > It looks like some of the language in the KIP is suggesting that
>> the
>> > > > Kafka
>> > > > > Dev community is going to provide the log4j2 equivalent and
>> would be
>> > > > > responsible for maintaining it: “ User-interfacing configurations
>> > (like
>> > > > > broker logging config), provide additional log4j2-equivalent
>> > > > configuration
>> > > > > with backward compatibility”
>> > > > >
>> > > > > I think the external dependencies should just be used as is from
>> the
>> > > > > original project to minimize overhead. Trying to recreate an
>> > external
>> > > > > dependency to provide backward compatibility could end up being a
>> lot
>> > > of
>> > > > > work in the long term.
>> > > > >
>> > > > > Do you think we need to push this to 3.1 to allow more time to
>> think
>> > > > about
>> > > > > it and discuss it further?
>> > > > >
>> > > > > Those are my thoughts at this time
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Jun 11, 2021 at 12:04 PM Ismael Juma <ism...@juma.me.uk>
>> > > wrote:
>> > > > >
>> > > > > > I'm personally not sure about this. We have not maintained the
>> > > existing
>> > > > > > log4j-appender very actively, so are we sure that we can do a
>> > better
>> > > > job
>> > > > > > than the log4j community?
>> > > > > >
>> > > > > > Ismael
>> > > > > >
>> > > > > > On Tue, May 25, 2021, 6:46 AM Dongjin Lee <dong...@apache.org>
>> > > wrote:
>> > > > > >
>> > > > > > > Hi Kafka dev,
>> > > > > > >
>> > > > > > > I'd like to kick-off the voting for KIP-719: Add Log4J2
>> Appender.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Add+Log4J2+Appender
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Dongjin
>> > > > > > >
>> > > > > > > --
>> > > > > > > *Dongjin Lee*
>> > > > > > >
>> > > > > > > *A hitchhiker in the mathematical world.*
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > *github: <http://goog_969573159/>github.com/dongjinleekr
>> > > > > > > <https://github.com/dongjinleekr>keybase:
>> > > > > > https://keybase.io/dongjinleekr
>> > > > > > > <https://keybase.io/dongjinleekr>linkedin:
>> > > > > > kr.linkedin.com/in/dongjinleekr
>> > > > > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck:
>> > > > > > > speakerdeck.com/dongjin
>> > > > > > > <https://speakerdeck.com/dongjin>*
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Dongjin Lee*
>> > > >
>> > > > *A hitchhiker in the mathematical world.*
>> > > >
>> > > >
>> > > >
>> > > > *github: <http://goog_969573159/>github.com/dongjinleekr
>> > > > <https://github.com/dongjinleekr>keybase:
>> > > https://keybase.io/dongjinleekr
>> > > > <https://keybase.io/dongjinleekr>linkedin:
>> > > kr.linkedin.com/in/dongjinleekr
>> > > > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck:
>> > > > speakerdeck.com/dongjin
>> > > > <https://speakerdeck.com/dongjin>*
>> > > >
>> > >
>> >
>> >
>> > --
>> > *Dongjin Lee*
>> >
>> > *A hitchhiker in the mathematical world.*
>> >
>> >
>> >
>> > *github: <http://goog_969573159/>github.com/dongjinleekr
>> > <https://github.com/dongjinleekr>keybase:
>> https://keybase.io/dongjinleekr
>> > <https://keybase.io/dongjinleekr>linkedin:
>> kr.linkedin.com/in/dongjinleekr
>> > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck:
>> > speakerdeck.com/dongjin
>> > <https://speakerdeck.com/dongjin>*
>> >
>>
>
>
> --
> *Dongjin Lee*
>
> *A hitchhiker in the mathematical world.*
>
>
>
> *github: <http://goog_969573159/>github.com/dongjinleekr
> <https://github.com/dongjinleekr>keybase: https://keybase.io/dongjinleekr
> <https://keybase.io/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
> <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: speakerdeck.com/dongjin
> <https://speakerdeck.com/dongjin>*
>
--
*Dongjin Lee*
*A hitchhiker in the mathematical world.*
*github: <http://goog_969573159/>github.com/dongjinleekr
<https://github.com/dongjinleekr>keybase: https://keybase.io/dongjinleekr
<https://keybase.io/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
<https://kr.linkedin.com/in/dongjinleekr>speakerdeck: speakerdeck.com/dongjin
<https://speakerdeck.com/dongjin>*
