Hi Konstantine, Given that we are making a new RC, I would suggest that we merge "KAFKA-13277; Fix size calculation for tagged string fields in message generator" to 3.0. What do you think?
best, Colin On Tue, Sep 7, 2021, at 13:20, Konstantine Karantasis wrote: > Thanks David, > > Assuming we'll have the PRs you mention merged soon, I'll include them in > RC2 for 3.0.0 given their low risk and the fact that we need to generate a > new RC anyways. > > Konstantine > > On Tue, Sep 7, 2021 at 4:43 PM David Jacot <dja...@confluent.io.invalid> > wrote: > > > Hi Konstantine, > > > > I would like to propose https://github.com/apache/kafka/pull/11300 as a > > blocker > > as well. The PR fixes KAFKA-13258/13259/13260. There are all very small but > > annoying issues. The PR is trivial. > > > > Regarding KAFKA-13266, the fix is quite simple, so low risk in my opinion. > > Jason > > will review it soon. > > > > Best, > > David > > > > On Tue, Sep 7, 2021 at 3:27 PM Ismael Juma <ism...@juma.me.uk> wrote: > > > > > Hi Konstantine, > > > > > > I will remove the final modifier for now. > > > > > > I added it because the removal of the deprecated `close` overload could > > > lead to weird behavior if the no-args `close` was overridden (the > > > implementation of the no-arg `close` delegated to the removed `close`, > > but > > > that's no longer the case). However, I didn't realize `MockConsumer` was > > a > > > public class (which is a mistake imo since we tweak its behavior > > regularly > > > and sometimes add methods without a KIP). > > > > > > Ismael > > > > > > On Tue, Sep 7, 2021 at 1:46 AM Konstantine Karantasis < > > > kkaranta...@apache.org> wrote: > > > > > > > Hi Gary, > > > > > > > > Regarding KAFKA-13262, this might need a more detailed and strong > > > > justification to be considered as a blocker at this point in the > > release. > > > > But we are already moving towards RC2 because of the blockers mentioned > > > > above, so I'd be in favor of a PR that would revert the change that > > made > > > > this method final if we could get one this week. I'm not sure why the > > > > MockConsumer::close method was marked as final in > > > > https://github.com/apache/kafka/pull/10438. The change is not included > > > in > > > > the description. Maybe it wasn't obvious that this class is used > > outside > > > > this project or maybe we thought we should discourage such use. > > > > > > > > Gary, is this the only change you are referring to? > > > > Ismael, I see you authored the PR originally. Any thoughts on the > > > suggested > > > > change? > > > > > > > > Konstantine > > > > > > > > On Tue, Sep 7, 2021 at 11:35 AM Konstantine Karantasis < > > > > kkaranta...@apache.org> wrote: > > > > > > > > > Hi Magnus, > > > > > > > > > > Thanks for reporting the numbers. > > > > > > > > > > I agree that what you observe warrants further investigation, not so > > > much > > > > > because these are clear performance regressions but because they > > might > > > be > > > > > indications of underlying issues, as you noted. However, from a major > > > > > release standpoint I'd say that such preliminary evidence probably > > > falls > > > > > within the range of acceptable differences in the presence of major > > > > changes > > > > > and can be optimized over time in subsequent releases. > > > > > > > > > > Having said that, I'd take Israel's comment one step further and say > > > that > > > > > in order to be able to give more definitive answers it'd be good to > > > have > > > > a > > > > > description of the setup that is reproducible. Briefly, that would > > > > include > > > > > both the benchmark setup (specs etc) as well as the benchmark suite > > and > > > > the > > > > > way to run it (links to benchmark code including configurations and > > > > > description of how to run and reproduce). > > > > > > > > > > Small note that the attached plot is missing units on the x-axis > > > (y-axis > > > > I > > > > > assume is milliseconds from your description). > > > > > > > > > > Konstantine > > > > > > > > > > > > > > > On Fri, Sep 3, 2021 at 12:59 AM Israel Ekpo <israele...@gmail.com> > > > > wrote: > > > > > > > > > >> Magnus, > > > > >> > > > > >> Please could you share the machine and network specs? > > > > >> > > > > >> How much CPU, RAM is available on each node? > > > > >> > > > > >> What JDK, JRE version are you using? > > > > >> > > > > >> What are your broker and client configuration values? Please could > > you > > > > >> share this info if possible? > > > > >> > > > > >> Thanks. > > > > >> > > > > >> > > > > >> > > > > >> On Wed, Sep 1, 2021 at 10:25 AM Magnus Edenhill <mag...@edenhill.se > > > > > > > >> wrote: > > > > >> > > > > >> > Hi Konstantine, > > > > >> > > > > > >> > Some findings from running 3.0.0-RC1 with the librdkafka test > > suite: > > > > >> > > > > > >> > * Compaction seems to take slightly longer to kick in when segment > > > > sizes > > > > >> > exceed their threshold. (Used to take less than 20 seconds, now > > > > takes > > > > >> > 20..30 seconds.) > > > > >> > > > > > >> > * CreateTopic seems to take slightly longer to propagate through > > the > > > > >> > cluster, > > > > >> > e.g., before a new topic is available in metadata from other > > > > brokers. > > > > >> > > > > > >> > * CreateTopics seems to take longer when the Admin request timeout > > > is > > > > >> set, > > > > >> > looks like a plateau at 10 seconds: > > > > >> > https://imgur.com/a/n6y76sj > > > > >> > > > > > >> > (This is a 3 broker cluster with identical configs between 2.8 and > > > > >> 3.0.0.) > > > > >> > > > > > >> > Nothing critical, but could be an indication of regression so I > > > > thought > > > > >> I'd > > > > >> > mention it. > > > > >> > > > > > >> > Regards, > > > > >> > Magnus > > > > >> > > > > > >> > > > > > >> > Den tis 31 aug. 2021 kl 17:51 skrev Konstantine Karantasis < > > > > >> > kkaranta...@apache.org>: > > > > >> > > > > > >> > > Small correction to my previous email. > > > > >> > > The actual link for public preview of the 3.0.0 blog post draft > > > is: > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache6 > > > > >> > > > > > > >> > > (see also the email thread with title: [DISCUSS] Please review > > the > > > > >> 3.0.0 > > > > >> > > blog post) > > > > >> > > > > > > >> > > Best, > > > > >> > > Konstantine > > > > >> > > > > > > >> > > On Tue, Aug 31, 2021 at 6:34 PM Konstantine Karantasis < > > > > >> > > kkaranta...@apache.org> wrote: > > > > >> > > > > > > >> > > > > > > > >> > > > Hello Kafka users, developers and client-developers, > > > > >> > > > > > > > >> > > > This is the second release candidate for Apache Kafka 3.0.0. > > > > >> > > > It corresponds to a major release that includes many new > > > features, > > > > >> > > > including: > > > > >> > > > > > > > >> > > > * The deprecation of support for Java 8 and Scala 2.12. > > > > >> > > > * Kafka Raft support for snapshots of the metadata topic and > > > > >> > > > other improvements in the self-managed quorum. > > > > >> > > > * Deprecation of message formats v0 and v1. > > > > >> > > > * Stronger delivery guarantees for the Kafka producer enabled > > by > > > > >> > default. > > > > >> > > > * Optimizations in OffsetFetch and FindCoordinator requests. > > > > >> > > > * More flexible Mirror Maker 2 configuration and deprecation > > of > > > > >> > > > Mirror Maker 1. > > > > >> > > > * Ability to restart a connector's tasks on a single call in > > > Kafka > > > > >> > > Connect. > > > > >> > > > * Connector log contexts and connector client overrides are > > now > > > > >> enabled > > > > >> > > > by default. > > > > >> > > > * Enhanced semantics for timestamp synchronization in Kafka > > > > Streams. > > > > >> > > > * Revamped public API for Stream's TaskId. > > > > >> > > > * Default serde becomes null in Kafka Streams and several > > > > >> > > > other configuration changes. > > > > >> > > > > > > > >> > > > You may read and review a more detailed list of changes in the > > > > 3.0.0 > > > > >> > blog > > > > >> > > > post draft here: > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > https://blogs.apache.org/roller-ui/authoring/preview/kafka/?previewEntry=what-s-new-in-apache6 > > > > >> > > > > > > > >> > > > Release notes for the 3.0.0 release: > > > > >> > > > > > > > >> > > > > > > > https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/RELEASE_NOTES.html > > > > >> > > > > > > > >> > > > *** Please download, test and vote by Wednesday, September 8, > > > 2021 > > > > >> *** > > > > >> > > > > > > > >> > > > Kafka's KEYS file containing PGP keys we use to sign the > > > release: > > > > >> > > > https://kafka.apache.org/KEYS > > > > >> > > > > > > > >> > > > * Release artifacts to be voted upon (source and binary): > > > > >> > > > https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/ > > > > >> > > > > > > > >> > > > * Maven artifacts to be voted upon: > > > > >> > > > > > > > >> > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > >> > > > > > > > >> > > > * Javadoc: > > > > >> > > > https://home.apache.org/~kkarantasis/kafka-3.0.0-rc1/javadoc/ > > > > >> > > > > > > > >> > > > * Tag to be voted upon (off 3.0 branch) is the 3.0.0 tag: > > > > >> > > > https://github.com/apache/kafka/releases/tag/3.0.0-rc1 > > > > >> > > > > > > > >> > > > * Documentation: > > > > >> > > > https://kafka.apache.org/30/documentation.html > > > > >> > > > > > > > >> > > > * Protocol: > > > > >> > > > https://kafka.apache.org/30/protocol.html > > > > >> > > > > > > > >> > > > * Successful Jenkins builds for the 3.0 branch: > > > > >> > > > Unit/integration tests: > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka/detail/3.0/121/pipeline/ > > > > >> > > > (only few flaky failures) > > > > >> > > > System tests: > > > > >> > > > > > https://jenkins.confluent.io/job/system-test-kafka/job/3.0/57/ > > > > >> > > > > > > > >> > > > /************************************** > > > > >> > > > > > > > >> > > > Thanks, > > > > >> > > > Konstantine > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > >
