Hi, Thanks David for investigating this issue. If it's not a regression and if it's not a critical bug then I agree that it's not a blocker.
Mickael On Tue, Dec 5, 2023 at 7:16 AM Justine Olshan <jols...@confluent.io.invalid> wrote: > > Not sure if all folks can see this link but the results are here: > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1701739886--apache--3.6--ec5faaa90d/2023-12-04--001./2023-12-04--001./report.html > > > Looks like there were a few failures but most of them were due to known > versioning issues with tests (ie the dev version string causes issues when > trying to assert on it or versions 2.6.3, 2.7.2, and 3.3.2 are not in the > tests -- see PR here where I tried to fix this > https://github.com/apache/kafka/pull/14641) I believe the quota tests have > also been broken for a while and I've talked to folks about fixing them in > the past. > > Taking in the dev (3.6.1-SNAPSHOT), 2.6.3, 2.7.2, and 3.3.2, and the quota > tests I think the failures look reasonable. > > +1 binding from me given I did the previous validations. > > Justine > > P.S. I will get to that versioning fix again soon! > > On Mon, Dec 4, 2023 at 2:10 PM Justine Olshan <jols...@confluent.io> wrote: > > > Just wanted to give an update on the system tests since I said I started > > mine last week. I accidentally misconfigured my build, so I restarted them > > today. I should have results and if things look good, I should be able to > > make a vote early tomorrow. > > > > Justine > > > > On Mon, Dec 4, 2023 at 1:55 PM David Arthur > > <david.art...@confluent.io.invalid> wrote: > > > >> I have a fix for KAFKA-15968 > >> <https://issues.apache.org/jira/browse/KAFKA-15968> here > >> https://github.com/apache/kafka/pull/14919/. After a bit of digging, I > >> found that this behavior has existed in the KRaft controller since the > >> beginning, so it is not a regression. > >> > >> Another thing I observed while investigating this is that MetadataLoader > >> *does* treat CorruptRecordExceptions as fatal, which leads to the crash we > >> want. RaftClient calls handleCommit serially for all its listeners, so if > >> QuorumController#handleCommit is called first and does not crash, the call > >> to MetadataLoader#handleCommit will crash. > >> > >> Considering these two factors, I don't strongly feel like we need to block > >> the release for this fix. > >> > >> -David > >> > >> > >> On Mon, Dec 4, 2023 at 10:49 AM David Arthur <david.art...@confluent.io> > >> wrote: > >> > >> > Mickael, > >> > > >> > I just filed https://issues.apache.org/jira/browse/KAFKA-15968 while > >> > investigating a log corruption issue on the controller. I'm still > >> > investigating the issue to see how far back this goes, but I think this > >> > could be a blocker. > >> > > >> > Essentially, the bug is that the controller does not treat a > >> > CorruptRecordException as fatal, so the process will continue running. > >> If > >> > this happens on an active controller, it could corrupt the cluster's > >> > metadata in general (since missing a single metadata record can cause > >> lots > >> > of downstream problems). > >> > > >> > I'll update this thread by the end of day with a stronger > >> > blocker/non-blocker opinion. > >> > > >> > Thanks, > >> > David > >> > > >> > > >> > On Mon, Dec 4, 2023 at 6:48 AM Luke Chen <show...@gmail.com> wrote: > >> > > >> >> Hi Mickael: > >> >> > >> >> I did: > >> >> 1. Validated all checksums, signatures, and hashes > >> >> 2. Ran quick start for KRaft using scala 2.12 artifacts > >> >> 3. Spot checked the documentation and Javadoc > >> >> 4. Validated the licence file > >> >> > >> >> When running the validation to scala 2.12 package, I found these > >> libraries > >> >> are missing: (We only include scala 2.13 libraries in licence file) > >> >> scala-java8-compat_2.12-1.0.2 is missing in license file > >> >> scala-library-2.12.18 is missing in license file > >> >> scala-logging_2.12-3.9.4 is missing in license file > >> >> scala-reflect-2.12.18 is missing in license file > >> >> > >> >> It looks like this issue has been there for a long time, so it won't > >> be a > >> >> block issue for v3.6.1. > >> >> > >> >> +1 (binding) from me. > >> >> > >> >> Thank you. > >> >> Luke > >> >> > >> >> On Sat, Dec 2, 2023 at 5:46 AM Bill Bejeck <bbej...@gmail.com> wrote: > >> >> > >> >> > Hi Mickael, > >> >> > > >> >> > I did the following: > >> >> > > >> >> > 1. Validated all checksums, signatures, and hashes > >> >> > 2. Built from source > >> >> > 3. Ran all the unit tests > >> >> > 4. Spot checked the documentation and Javadoc > >> >> > 5. Ran the ZK, Kraft, and Kafka Streams quickstart guides > >> >> > > >> >> > I did notice that the `fillDotVersion` in `js/templateData.js` needs > >> >> > updating to `3.6.1`, but this is minor and should not block the > >> release. > >> >> > > >> >> > It's a +1(binding) for me, pending the successful system test run > >> >> > > >> >> > Thanks, > >> >> > Bill > >> >> > > >> >> > On Fri, Dec 1, 2023 at 1:49 PM Justine Olshan > >> >> <jols...@confluent.io.invalid > >> >> > > > >> >> > wrote: > >> >> > > >> >> > > I've started a system test run on my end. > >> >> > > > >> >> > > Justine > >> >> > > > >> >> > > On Wed, Nov 29, 2023 at 1:55 PM Justine Olshan < > >> jols...@confluent.io> > >> >> > > wrote: > >> >> > > > >> >> > > > I built from source and ran a simple transactional produce > >> bench. I > >> >> > ran a > >> >> > > > handful of unit tests as well. > >> >> > > > I scanned the docs and everything looked reasonable. > >> >> > > > > >> >> > > > I was wondering if we got the system test results mentioned > > >> System > >> >> > > > tests: Still running I'll post an update once they complete. > >> >> > > > > >> >> > > > Justine > >> >> > > > > >> >> > > > On Wed, Nov 29, 2023 at 6:33 AM Mickael Maison < > >> >> > mickael.mai...@gmail.com > >> >> > > > > >> >> > > > wrote: > >> >> > > > > >> >> > > >> Hi Josep, > >> >> > > >> > >> >> > > >> Good catch! > >> >> > > >> If it's the only issue we find, I don't think we should block > >> the > >> >> > > >> release just to fix that. > >> >> > > >> > >> >> > > >> If we find another issue, I'll backport it before running > >> another > >> >> RC, > >> >> > > >> otherwise I'll backport it once 3.6.1 is released. > >> >> > > >> > >> >> > > >> Thanks, > >> >> > > >> Mickael > >> >> > > >> > >> >> > > >> On Wed, Nov 29, 2023 at 11:55 AM Josep Prat > >> >> > <josep.p...@aiven.io.invalid > >> >> > > > > >> >> > > >> wrote: > >> >> > > >> > > >> >> > > >> > Hi Mickael, > >> >> > > >> > This PR[1] made me realize NOTICE-binary is missing the notice > >> >> for > >> >> > > >> > commons-io. I don't know if it's a blocker or not. I can > >> cherry > >> >> pick > >> >> > > the > >> >> > > >> > commit to the 3.6 branch if you want. > >> >> > > >> > > >> >> > > >> > Best, > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > [1]: https://github.com/apache/kafka/pull/14865 > >> >> > > >> > > >> >> > > >> > On Tue, Nov 28, 2023 at 10:25 AM Josep Prat < > >> josep.p...@aiven.io > >> >> > > >> >> > > >> wrote: > >> >> > > >> > > >> >> > > >> > > Hi Mickael, > >> >> > > >> > > Thanks for running the release. It's a +1 for me > >> (non-binding). > >> >> > > >> > > I did the following: > >> >> > > >> > > - Verified artifact's signatures and hashes > >> >> > > >> > > - Checked JavaDoc (with navigation to Oracle JavaDoc) > >> >> > > >> > > - Compiled source code > >> >> > > >> > > - Run unit tests and integration tests > >> >> > > >> > > - Run getting started with ZK and KRaft > >> >> > > >> > > > >> >> > > >> > > Best, > >> >> > > >> > > > >> >> > > >> > > On Tue, Nov 28, 2023 at 8:51 AM Kamal Chandraprakash < > >> >> > > >> > > kamal.chandraprak...@gmail.com> wrote: > >> >> > > >> > > > >> >> > > >> > >> +1 (non-binding) > >> >> > > >> > >> > >> >> > > >> > >> 1. Built the source from 3.6.1-rc0 tag in scala 2.12 and > >> 2.13 > >> >> > > >> > >> 2. Ran all the unit and integration tests. > >> >> > > >> > >> 3. Ran quickstart and verified the produce-consume on a 3 > >> node > >> >> > > >> cluster. > >> >> > > >> > >> 4. Verified the tiered storage functionality with > >> local-tiered > >> >> > > >> storage. > >> >> > > >> > >> > >> >> > > >> > >> On Tue, Nov 28, 2023 at 12:55 AM Federico Valeri < > >> >> > > >> fedeval...@gmail.com> > >> >> > > >> > >> wrote: > >> >> > > >> > >> > >> >> > > >> > >> > Hi Mickael, > >> >> > > >> > >> > > >> >> > > >> > >> > - Build from source (Java 17, Scala 2.13) > >> >> > > >> > >> > - Run unit and integration tests > >> >> > > >> > >> > - Run custom client apps using staging artifacts > >> >> > > >> > >> > > >> >> > > >> > >> > +1 (non binding) > >> >> > > >> > >> > > >> >> > > >> > >> > Thanks > >> >> > > >> > >> > Fede > >> >> > > >> > >> > > >> >> > > >> > >> > > >> >> > > >> > >> > > >> >> > > >> > >> > On Sun, Nov 26, 2023 at 11:34 AM Jakub Scholz < > >> >> ja...@scholz.cz > >> >> > > > >> >> > > >> wrote: > >> >> > > >> > >> > > > >> >> > > >> > >> > > +1 non-binding. I used the staged Scala 2.13 artifacts > >> and > >> >> > the > >> >> > > >> staged > >> >> > > >> > >> > Maven > >> >> > > >> > >> > > repo for my tests. All seems to work fine. > >> >> > > >> > >> > > > >> >> > > >> > >> > > Thanks > >> >> > > >> > >> > > Jakub > >> >> > > >> > >> > > > >> >> > > >> > >> > > On Fri, Nov 24, 2023 at 4:37 PM Mickael Maison < > >> >> > > >> mimai...@apache.org> > >> >> > > >> > >> > wrote: > >> >> > > >> > >> > > > >> >> > > >> > >> > > > Hello Kafka users, developers and client-developers, > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > This is the first candidate for release of Apache > >> Kafka > >> >> > > 3.6.1. > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > This is a bugfix release with several fixes, > >> including > >> >> > > >> dependency > >> >> > > >> > >> > > > version bumps for CVEs. > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > Release notes for the 3.6.1 release: > >> >> > > >> > >> > > > > >> >> > > >> > >> > >> >> > > > >> https://home.apache.org/~mimaison/kafka-3.6.1-rc0/RELEASE_NOTES.html > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > *** Please download, test and vote by Friday, > >> December 1 > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > Kafka's KEYS file containing PGP keys we use to sign > >> the > >> >> > > >> release: > >> >> > > >> > >> > > > https://kafka.apache.org/KEYS > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > * Release artifacts to be voted upon (source and > >> >> binary): > >> >> > > >> > >> > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/ > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > * Maven artifacts to be voted upon: > >> >> > > >> > >> > > > > >> >> > > >> > >> > >> >> > > >> > >> >> > > >> https://repository.apache.org/content/groups/staging/org/apache/kafka/ > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > * Javadoc: > >> >> > > >> > >> > > > > >> >> https://home.apache.org/~mimaison/kafka-3.6.1-rc0/javadoc/ > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.1 > >> >> tag: > >> >> > > >> > >> > > > > >> https://github.com/apache/kafka/releases/tag/3.6.1-rc0 > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > PR for updating docs: > >> >> > > >> > >> > > > https://github.com/apache/kafka-site/pull/568 > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > * Documentation: > >> >> > > >> > >> > > > https://kafka.apache.org/36/documentation.html > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > * Protocol: > >> >> > > >> > >> > > > https://kafka.apache.org/36/protocol.html > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > * Successful Jenkins builds for the 3.6 branch: > >> >> > > >> > >> > > > Unit/integration tests: We still have a lot of flaky > >> >> tests > >> >> > in > >> >> > > >> the > >> >> > > >> > >> 3.6 > >> >> > > >> > >> > > > branch. Looking at the last few 3.6 builds in > >> >> > > >> > >> > > > > >> >> https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/ > >> >> > it > >> >> > > >> seems > >> >> > > >> > >> all > >> >> > > >> > >> > > > tests passed at least once apart from > >> >> > > >> > >> > > > ClusterConnectionStatesTest.testSingleIP(). There's > >> >> > > >> > >> > > > https://issues.apache.org/jira/browse/KAFKA-15762 to > >> >> fix > >> >> > > that > >> >> > > >> test. > >> >> > > >> > >> > > > System tests: Still running I'll post an update once > >> >> they > >> >> > > >> complete. > >> >> > > >> > >> > > > > >> >> > > >> > >> > > > Thanks, > >> >> > > >> > >> > > > Mickael > >> >> > > >> > >> > > > > >> >> > > >> > >> > > >> >> > > >> > >> > >> >> > > >> > > > >> >> > > >> > > > >> >> > > >> > > -- > >> >> > > >> > > [image: Aiven] <https://www.aiven.io> > >> >> > > >> > > > >> >> > > >> > > *Josep Prat* > >> >> > > >> > > Open Source Engineering Director, *Aiven* > >> >> > > >> > > josep.p...@aiven.io | +491715557497 > >> >> > > >> > > aiven.io <https://www.aiven.io> | > >> >> > > >> > > <https://www.facebook.com/aivencloud> > >> >> > > >> > > <https://www.linkedin.com/company/aiven/> < > >> >> > > >> https://twitter.com/aiven_io> > >> >> > > >> > > *Aiven Deutschland GmbH* > >> >> > > >> > > Alexanderufer 3-7, 10117 Berlin > >> >> > > >> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> >> > > >> > > Amtsgericht Charlottenburg, HRB 209739 B > >> >> > > >> > > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > -- > >> >> > > >> > [image: Aiven] <https://www.aiven.io> > >> >> > > >> > > >> >> > > >> > *Josep Prat* > >> >> > > >> > Open Source Engineering Director, *Aiven* > >> >> > > >> > josep.p...@aiven.io | +491715557497 > >> >> > > >> > aiven.io <https://www.aiven.io> | < > >> >> > > >> https://www.facebook.com/aivencloud> > >> >> > > >> > <https://www.linkedin.com/company/aiven/> < > >> >> > > >> https://twitter.com/aiven_io> > >> >> > > >> > *Aiven Deutschland GmbH* > >> >> > > >> > Alexanderufer 3-7, 10117 Berlin > >> >> > > >> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> >> > > >> > Amtsgericht Charlottenburg, HRB 209739 B > >> >> > > >> > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > > >> > > >> > -- > >> > -David > >> > > >> > >> > >> -- > >> -David > >> > >
