Great. In that case we can fix the license issue retrospectively. I have
created a JIRA for it https://issues.apache.org/jira/browse/KAFKA-16278 and
also updated the release process (which redirects to
https://issues.apache.org/jira/browse/KAFKA-12622) to check for the correct
license in both the kafka binaries.
I am +1 (binding) assuming Mickael's concerns about update notes to 3.7 are
addressed before release.
--
Divij Vaidya
On Mon, Feb 19, 2024 at 6:08 PM Mickael Maison <mickael.mai...@gmail.com>
wrote:
> Hi,
>
> I agree with Josep, I don't think it's worth making a new RC just for this.
>
> Thanks Stanislav for sharing the test results. The last thing holding
> me from casting my vote is the missing upgrade notes for 3.7.0.
>
> Thanks,
> Mickael
>
>
>
> On Mon, Feb 19, 2024 at 4:28 PM Josep Prat <josep.p...@aiven.io.invalid>
> wrote:
> >
> > I think I remember finding a similar problem (NOTICE_binary) and it
> didn't
> > qualify for an extra RC
> >
> > Best,
> >
> > On Mon, Feb 19, 2024 at 3:44 PM Divij Vaidya <divijvaidy...@gmail.com>
> > wrote:
> >
> > > I have performed the following checks. The only thing I would like to
> call
> > > out is the missing licenses before providing a vote. How do we want
> > > to proceed on this? What have we done in the past? (Creating a new RC
> is
> > > overkill IMO for this license issue).
> > >
> > > ## License check
> > >
> > > Test: Validate license of dependencies for both 2.12 & 2.13 binary.
> > > Result: Missing license for some scala* libraries specifically for
> 2.12.
> > > Seems like we have been missing these licenses for quite some version
> now.
> > >
> > > ```
> > > for f in $(ls libs | grep -v "^kafka\|connect\|trogdor"); do if ! grep
> -q
> > > ${f%.*} LICENSE; then echo "${f%.*} is missing in license file"; fi;
> done
> > > scala-collection-compat_2.12-2.10.0 is missing in license file
> > > scala-java8-compat_2.12-1.0.2 is missing in license file
> > > scala-library-2.12.18 is missing in license file
> > > scala-logging_2.12-3.9.4 is missing in license file
> > > scala-reflect-2.12.18 is missing in license file
> > > ```
> > >
> > > ## Long running tests for memory leak (on ARM machine with zstd)
> > >
> > > Test: Run produce/consume for a few hours and verify no gradual
> increase in
> > > heap.
> > > Result: No heap increase observed. The overall CPU utilization is lower
> > > compared to 3.5.1.
> > >
> > > ## Verify system test results
> > >
> > > Test: Spot check the results of system tests.
> > > Result: I have verified that the system tests are passing across
> different
> > > runs.
> > >
> > > --
> > > Divij Vaidya
> > >
> > >
> > >
> > > On Sun, Feb 18, 2024 at 2:50 PM Stanislav Kozlovski
> > > <stanis...@confluent.io.invalid> wrote:
> > >
> > > > The latest system test build completed successfully -
> > > >
> > > >
> > >
> https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708250728--apache--3.7--02197edaaa/2024-02-18--001./2024-02-18--001./report.html
> > > >
> > > > *System tests are therefore all good*. We just have some flakes
> > > >
> > > > On Sun, Feb 18, 2024 at 10:45 AM Stanislav Kozlovski <
> > > > stanis...@confluent.io>
> > > > wrote:
> > > >
> > > > > The upgrade test passed ->
> > > > >
> > > >
> > >
> https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708103771--apache--3.7--bb6990114b/2024-02-16--001./2024-02-16--001./report.html
> > > > >
> > > > > The replica verification test succeeded in ZK mode, but failed in
> > > > > ISOLATED_KRAFT. It just seems to be very flaky.
> > > > >
> > > >
> > >
> https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708100119--apache--3.7--bb6990114b/2024-02-16--001./2024-02-16--001./report.html
> > > > >
> > > > > Scheduling another run in
> > > > >
> > >
> https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6062/
> > > > >
> > > > > On Fri, Feb 16, 2024 at 6:39 PM Stanislav Kozlovski <
> > > > > stanis...@confluent.io> wrote:
> > > > >
> > > > >> Thanks all for the help in verifying.
> > > > >>
> > > > >> I have updated
> > > > >>
> > > >
> > >
> https://gist.github.com/stanislavkozlovski/820976fc7bfb5f4dcdf9742fd96a9982
> > > > >> with the system tests.
> > > > >> There were two builds ran, and across those - the following tests
> > > failed
> > > > >> two times in a row:
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> *kafkatest.tests.tools.replica_verification_test.ReplicaVerificationToolTest#test_replica_lagsArguments:{
> > > > >> "metadata_quorum": "ZK"}*Fails with the same error of
> > > > >> *`TimeoutError('Timed out waiting to reach non-zero number of
> replica
> > > > >> lags.')`*
> > > > >> I have scheduled a re-run of this specific test here ->
> > > > >>
> > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6057
> > > > >>
> > > > >>
> *kafkatest.tests.core.upgrade_test.TestUpgrade#test_upgradeArguments:{
> > > > >> "compression_types": [ "zstd" ], "from_kafka_version": "2.4.1",
> > > > >> "to_message_format_version": null}*
> > > > >> Fails with the same error of
> > > > >> *`TimeoutError('Producer failed to produce messages for 20s.')`*
> > > > >>
> *kafkatest.tests.core.upgrade_test.TestUpgrade#test_upgradeArguments:{
> > > > >> "compression_types": [ "lz4" ], "from_kafka_version": "3.0.2",
> > > > >> "to_message_format_version": null}*
> > > > >> Fails with the same error of *`TimeoutError('Producer failed to
> > > produce
> > > > >> messages for 20s.')`*
> > > > >>
> > > > >> I have scheduled a re-run of this test here ->
> > > > >>
> > >
> https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6058/
> > > > >>
> > > > >> On Fri, Feb 16, 2024 at 12:15 PM Vedarth Sharma <
> > > > vedarth.sha...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hey Stanislav,
> > > > >>>
> > > > >>> Thanks for the release candidate.
> > > > >>>
> > > > >>> +1 (non-binding)
> > > > >>>
> > > > >>> I tested and verified the docker image artifact
> > > > apache/kafka:3.7.0-rc4:-
> > > > >>> - verified create topic, produce messages and consume messages
> flow
> > > > when
> > > > >>> running the docker image with
> > > > >>> - default configs
> > > > >>> - configs provided via env variables
> > > > >>> - configs provided via file input
> > > > >>> - verified the html documentation for docker image.
> > > > >>> - ran the example docker compose files successfully.
> > > > >>>
> > > > >>> All looks good for the docker image artifact!
> > > > >>>
> > > > >>> Thanks and regards,
> > > > >>> Vedarth
> > > > >>>
> > > > >>>
> > > > >>> On Thu, Feb 15, 2024 at 10:58 PM Mickael Maison <
> > > > >>> mickael.mai...@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Hi Stanislav,
> > > > >>> >
> > > > >>> > Thanks for running the release.
> > > > >>> >
> > > > >>> > I did the following testing:
> > > > >>> > - verified the check sums and signatures
> > > > >>> > - ran ZooKeeper and KRaft quickstarts with Scala 2.13 binaries
> > > > >>> > - ran a successful migration from ZooKeeper to KRaft
> > > > >>> >
> > > > >>> > We seem to be missing the upgrade notes for 3.7.0 in the docs.
> See
> > > > >>> > https://kafka.apache.org/37/documentation.html#upgrade that
> still
> > > > >>> > points to 3.6.0
> > > > >>> > Before voting I'd like to see results from the system tests
> too.
> > > > >>> >
> > > > >>> > Thanks,
> > > > >>> > Mickael
> > > > >>> >
> > > > >>> > On Thu, Feb 15, 2024 at 6:06 PM Andrew Schofield
> > > > >>> > <andrew_schofield_j...@outlook.com> wrote:
> > > > >>> > >
> > > > >>> > > +1 (non-binding). I used the staged binaries with Scala
> 2.13. I
> > > > tried
> > > > >>> > the new group coordinator
> > > > >>> > > and consumer group protocol which is included with the Early
> > > Access
> > > > >>> > release of KIP-848.
> > > > >>> > > Also verified the availability of the new APIs. All working
> as
> > > > >>> expected.
> > > > >>> > >
> > > > >>> > > Thanks,
> > > > >>> > > Andrew
> > > > >>> > >
> > > > >>> > > > On 15 Feb 2024, at 05:07, Paolo Patierno <
> > > > paolo.patie...@gmail.com
> > > > >>> >
> > > > >>> > wrote:
> > > > >>> > > >
> > > > >>> > > > +1 (non-binding). I used the staged binaries with Scala
> 2.13
> > > and
> > > > >>> mostly
> > > > >>> > > > focused on the ZooKeeper to KRaft migration with multiple
> > > tests.
> > > > >>> > Everything
> > > > >>> > > > works fine.
> > > > >>> > > >
> > > > >>> > > > Thanks
> > > > >>> > > > Paolo
> > > > >>> > > >
> > > > >>> > > > On Mon, 12 Feb 2024, 22:06 Jakub Scholz, <ja...@scholz.cz>
> > > > wrote:
> > > > >>> > > >
> > > > >>> > > >> +1 (non-binding). I used the staged binaries with Scala
> 2.13
> > > and
> > > > >>> the
> > > > >>> > staged
> > > > >>> > > >> Maven artifacts to run my tests. All seems to work fine.
> > > Thanks.
> > > > >>> > > >>
> > > > >>> > > >> Jakub
> > > > >>> > > >>
> > > > >>> > > >> On Fri, Feb 9, 2024 at 4:20 PM Stanislav Kozlovski
> > > > >>> > > >> <stanis...@confluent.io.invalid> wrote:
> > > > >>> > > >>
> > > > >>> > > >>> Hello Kafka users, developers and client-developers,
> > > > >>> > > >>>
> > > > >>> > > >>> This is the second candidate we are considering for
> release
> > > of
> > > > >>> Apache
> > > > >>> > > >> Kafka
> > > > >>> > > >>> 3.7.0.
> > > > >>> > > >>>
> > > > >>> > > >>> Major changes include:
> > > > >>> > > >>> - Early Access to KIP-848 - the next generation of the
> > > consumer
> > > > >>> > rebalance
> > > > >>> > > >>> protocol
> > > > >>> > > >>> - Early Access to KIP-858: Adding JBOD support to KRaft
> > > > >>> > > >>> - KIP-714: Observability into Client metrics via a
> > > standardized
> > > > >>> > interface
> > > > >>> > > >>>
> > > > >>> > > >>> Release notes for the 3.7.0 release:
> > > > >>> > > >>>
> > > > >>> > > >>>
> > > > >>> > > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/RELEASE_NOTES.html
> > > > >>> > > >>>
> > > > >>> > > >>> *** Please download, test and vote by Thursday, February
> > > 15th,
> > > > >>> 9AM
> > > > >>> > PST
> > > > >>> > > >> ***
> > > > >>> > > >>>
> > > > >>> > > >>> Kafka's KEYS file containing PGP keys we use to sign the
> > > > release:
> > > > >>> > > >>> https://kafka.apache.org/KEYS
> > > > >>> > > >>>
> > > > >>> > > >>> * Release artifacts to be voted upon (source and binary):
> > > > >>> > > >>>
> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/
> > > > >>> > > >>>
> > > > >>> > > >>> * Docker release artifact to be voted upon:
> > > > >>> > > >>> apache/kafka:3.7.0-rc4
> > > > >>> > > >>>
> > > > >>> > > >>> * Maven artifacts to be voted upon:
> > > > >>> > > >>>
> > > > >>> >
> > > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > >>> > > >>>
> > > > >>> > > >>> * Javadoc:
> > > > >>> > > >>>
> > > > >>>
> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/javadoc/
> > > > >>> > > >>>
> > > > >>> > > >>> * Tag to be voted upon (off 3.7 branch) is the 3.7.0 tag:
> > > > >>> > > >>> https://github.com/apache/kafka/releases/tag/3.7.0-rc4
> > > > >>> > > >>>
> > > > >>> > > >>> * Documentation:
> > > > >>> > > >>> https://kafka.apache.org/37/documentation.html
> > > > >>> > > >>>
> > > > >>> > > >>> * Protocol:
> > > > >>> > > >>> https://kafka.apache.org/37/protocol.html
> > > > >>> > > >>>
> > > > >>> > > >>> * Successful Jenkins builds for the 3.7 branch:
> > > > >>> > > >>>
> > > > >>> > > >>> Unit/integration tests: I am in the process of running
> and
> > > > >>> analyzing
> > > > >>> > > >> these.
> > > > >>> > > >>> System tests: I am in the process of running these.
> > > > >>> > > >>>
> > > > >>> > > >>> Expect a follow-up over the weekend
> > > > >>> > > >>>
> > > > >>> > > >>> * Successful Docker Image Github Actions Pipeline for 3.7
> > > > branch:
> > > > >>> > > >>> Docker Build Test Pipeline:
> > > > >>> > > >>> https://github.com/apache/kafka/actions/runs/7845614846
> > > > >>> > > >>>
> > > > >>> > > >>> /**************************************
> > > > >>> > > >>>
> > > > >>> > > >>> Best,
> > > > >>> > > >>> Stanislav
> > > > >>> > > >>>
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Best,
> > > > >> Stanislav
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Best,
> > > > > Stanislav
> > > > >
> > > >
> > > >
> > > > --
> > > > Best,
> > > > Stanislav
> > > >
> > >
> >
> >
> > --
> > [image: Aiven] <https://www.aiven.io>
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.p...@aiven.io | +491715557497
> > aiven.io <https://www.aiven.io> | <
> https://www.facebook.com/aivencloud>
> > <https://www.linkedin.com/company/aiven/> <
> https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > Amtsgericht Charlottenburg, HRB 209739 B
>
