Found a bug while testing TS feature in 3.6 - https://issues.apache.org/jira/browse/KAFKA-15481
I don't consider it as a blocker for release since it's a concurrency bug that should occur rarely for a feature which is early access. Sharing it here as FYI in case someone else thinks differently. -- Divij Vaidya On Fri, Sep 22, 2023 at 1:26 AM Satish Duggana <satish.dugg...@gmail.com> wrote: > > Thanks Divij for raising a PR for doc formatting issue. > > On Thu, 21 Sep, 2023, 2:22 PM Divij Vaidya, <divijvaidy...@gmail.com> wrote: > > > Hey Satish > > > > I filed a PR to fix the website formatting bug in 3.6 documentation - > > https://github.com/apache/kafka/pull/14419 > > Please take a look when you get a chance. > > > > -- > > Divij Vaidya > > > > On Tue, Sep 19, 2023 at 5:36 PM Chris Egerton <chr...@aiven.io.invalid> > > wrote: > > > > > > Hi Satish, > > > > > > I think this qualifies as a blocker. This API has been around for years > > now > > > and, while we don't document it as not exposing duplicates*, it has come > > > with that implicit contract since its inception. More importantly, it has > > > also never exposed plugins that cannot be used on the worker. This change > > > in behavior not only introduces duplicates*, it causes unreachable > > plugins > > > to be displayed. With this in mind, it seems to qualify pretty clearly > > as a > > > regression and we should not put out a release that includes it. > > > > > > * - Really, these aren't duplicates; rather, they're multiple copies of > > the > > > same plugin that come from different locations on the worker > > > > > > Best, > > > > > > Chris > > > > > > On Tue, Sep 19, 2023 at 4:31 AM Satish Duggana <satish.dugg...@gmail.com > > > > > > wrote: > > > > > > > Hi Greg, > > > > Is this API documented that it does not return duplicate entries? > > > > > > > > Can we also get an opinion from PMC/Committers who have KafkaConnect > > > > expertise on whether this issue is a release blocker? > > > > > > > > If we agree that it is not a release blocker then we can have a > > > > release note clarifying this behaviour and add a reference to the JIRA > > > > that follows up on the possible solutions. > > > > > > > > Thanks, > > > > Satish. > > > > > > > > > > > > On Tue, 19 Sept 2023 at 03:29, Greg Harris > > <greg.har...@aiven.io.invalid> > > > > wrote: > > > > > > > > > > Hey Satish, > > > > > > > > > > After investigating further, I believe that this is a regression, but > > > > > mostly a cosmetic one. > > > > > I don't think there is significant risk of breaking clients with this > > > > > change, but it would be confusing for users, so I'd still like to get > > > > > the fix into the next RC. > > > > > I've opened a PR here: https://github.com/apache/kafka/pull/14398 > > and > > > > > I'll work to get it merged promptly. > > > > > > > > > > Thanks! > > > > > > > > > > On Mon, Sep 18, 2023 at 11:54 AM Greg Harris <greg.har...@aiven.io> > > > > wrote: > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > While validating 3.6.0-rc0, I noticed this regression as compared > > to > > > > > > 3.5.1: https://issues.apache.org/jira/browse/KAFKA-15473 > > > > > > > > > > > > Impact: The `connector-plugins` endpoint lists duplicates which may > > > > > > cause confusion for users, or poor behavior in clients. > > > > > > Using the other REST API endpoints appears unaffected. > > > > > > I'll open a PR for this later today. > > > > > > > > > > > > Thanks, > > > > > > Greg > > > > > > > > > > > > On Thu, Sep 14, 2023 at 11:56 AM Satish Duggana > > > > > > <satish.dugg...@gmail.com> wrote: > > > > > > > > > > > > > > Thanks Justine for the update. I saw in the morning that these > > > > changes > > > > > > > are pushed to trunk and 3.6. > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > On Thu, 14 Sept 2023 at 21:54, Justine Olshan > > > > > > > <jols...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > We were able to merge > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15459 yesterday > > > > > > > > and pick to 3.6. > > > > > > > > > > > > > > > > Hopefully nothing more from me on this release. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Justine > > > > > > > > > > > > > > > > On Wed, Sep 13, 2023 at 9:51 PM Satish Duggana < > > > > satish.dugg...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks Luke for the update. > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > On Thu, 14 Sept 2023 at 07:29, Luke Chen <show...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > Since this PR: > > > > > > > > > > https://github.com/apache/kafka/pull/14366 only changes > > the > > > > doc, I've > > > > > > > > > > backported to 3.6 branch. FYI. > > > > > > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > Luke > > > > > > > > > > > > > > > > > > > > On Thu, Sep 14, 2023 at 12:15 AM Justine Olshan > > > > > > > > > > <jols...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > Hey Satish -- yes, you are correct. KAFKA-15459 only > > affects > > > > 3.6. > > > > > > > > > > > PR should be finalized soon. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Justine > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 13, 2023 at 1:41 AM Federico Valeri < > > > > fedeval...@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, this is a small documentation fix about ZK > > to > > > > KRaft > > > > > > > > > > > > migration, that we would like to backport to 3.5 and > > 3.6 > > > > branches. > > > > > > > > > Are > > > > > > > > > > > > you ok with that? > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14366 > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 13, 2023 at 3:13 AM Satish Duggana < > > > > > > > > > satish.dugg...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks David for the quick resolution. > > > > > > > > > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 12 Sept 2023 at 22:51, David Arthur > > > > > > > > > > > > > <david.art...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > KAFKA-15450 is merged to 3.6 (as well as trunk, > > 3.5, > > > > and 3.4) > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > David > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 11:44 AM Ismael Juma < > > > > m...@ismaeljuma.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Justine, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Probably best to have the conversation in the > > JIRA > > > > ticket vs > > > > > > > > > the > > > > > > > > > > > > release > > > > > > > > > > > > > > > thread. Generally, we want to only include low > > risk > > > > bug fixes > > > > > > > > > that > > > > > > > > > > > > are > > > > > > > > > > > > > > > fully compatible in patch releases. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 7:16 AM Justine Olshan > > > > > > > > > > > > > > > <jols...@confluent.io.invalid> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks Satish. I understand. > > > > > > > > > > > > > > > > Just curious, is this something that could be > > > > added to > > > > > > > > > 3.6.1? It > > > > > > > > > > > > would be > > > > > > > > > > > > > > > > nice to say that hanging transactions are fully > > > > covered in a > > > > > > > > > 3.6 > > > > > > > > > > > > release. > > > > > > > > > > > > > > > > I'm not as familiar with the rules around minor > > > > releases, but > > > > > > > > > > > > adding it > > > > > > > > > > > > > > > > there would give more time to ensure stability. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Justine > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 12, 2023 at 5:49 AM Satish Duggana > > < > > > > > > > > > > > > satish.dugg...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Justine, > > > > > > > > > > > > > > > > > We can skip this change into 3.6 now as it is > > > > not a > > > > > > > > > blocker or > > > > > > > > > > > > > > > > > regression and it involves changes to the API > > > > > > > > > implementation. > > > > > > > > > > > > Let us > > > > > > > > > > > > > > > > > plan to add the gap in the release notes as > > you > > > > mentioned. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 12 Sept 2023 at 04:44, Justine Olshan > > > > > > > > > > > > > > > > > <jols...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We just discovered a gap in KIP-890 part > > 1. We > > > > currently > > > > > > > > > > > don't > > > > > > > > > > > > verify > > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > > txn offset commits, so it is still > > possible to > > > > have > > > > > > > > > hanging > > > > > > > > > > > > > > > > transactions > > > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > > the consumer offsets partitions. > > > > > > > > > > > > > > > > > > I've opened a jira to wire the > > verification in > > > > that > > > > > > > > > request. > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15449 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This also isn't a regression, but it would > > be > > > > nice to > > > > > > > > > have > > > > > > > > > > > > part 1 > > > > > > > > > > > > > > > fully > > > > > > > > > > > > > > > > > > complete. I have opened a PR with the fix: > > > > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14370 > > . > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I understand if there are concerns about > > last > > > > minute > > > > > > > > > changes > > > > > > > > > > > > to this > > > > > > > > > > > > > > > > API > > > > > > > > > > > > > > > > > > and we can hold off if that makes the most > > > > sense. > > > > > > > > > > > > > > > > > > If we take that route, I think we should > > still > > > > keep > > > > > > > > > > > > verification for > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > data partitions since it still provides > > full > > > > protection > > > > > > > > > there > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > improves > > > > > > > > > > > > > > > > > > the transactions experience. We will need > > to > > > > call out > > > > > > > > > the gap > > > > > > > > > > > > in the > > > > > > > > > > > > > > > > > > release notes for consumer offsets > > partitions > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > > > > > Justine > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 11, 2023 at 12:29 PM David > > Arthur > > > > > > > > > > > > > > > > > > <david.art...@confluent.io.invalid> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Another (small) ZK migration issue was > > > > identified. > > > > > > > > > This one > > > > > > > > > > > > isn't a > > > > > > > > > > > > > > > > > > > regression (it has existed since 3.4), > > but I > > > > think it's > > > > > > > > > > > > reasonable > > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > include. It's a small configuration check > > > > that could > > > > > > > > > > > > potentially > > > > > > > > > > > > > > > save > > > > > > > > > > > > > > > > > end > > > > > > > > > > > > > > > > > > > users from some headaches down the line. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15450 > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/kafka/pull/14367 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think we can get this one committed to > > > > trunk today. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Sep 10, 2023 at 7:50 PM Ismael > > Juma < > > > > > > > > > > > > m...@ismaeljuma.com> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That sounds great. I think we should > > aim > > > > to only > > > > > > > > > allow > > > > > > > > > > > > blockers > > > > > > > > > > > > > > > > > > > > (regressions, impactful security > > issues, > > > > etc.) on > > > > > > > > > the 3.6 > > > > > > > > > > > > branch > > > > > > > > > > > > > > > > > until > > > > > > > > > > > > > > > > > > > > 3.6.0 is out. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Sep 9, 2023, 12:20 AM Satish > > > > Duggana < > > > > > > > > > > > > > > > > > satish.dugg...@gmail.com> > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Ismael, > > > > > > > > > > > > > > > > > > > > > It looks like we will publish RC0 by > > > > 14th Sep. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 8 Sept 2023 at 19:23, Ismael > > > > Juma < > > > > > > > > > > > > m...@ismaeljuma.com> > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Do you have a sense of when we'll > > > > publish RC0? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023 at 6:27 AM > > David > > > > Arthur > > > > > > > > > > > > > > > > > > > > > > <david.art...@confluent.io > > .invalid> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Quick update on my two blockers: > > > > KAFKA-15435 is > > > > > > > > > > > > merged to > > > > > > > > > > > > > > > > > trunk and > > > > > > > > > > > > > > > > > > > > > > > cherry-picked to 3.6. I have a PR > > > > open for > > > > > > > > > > > > KAFKA-15441 and > > > > > > > > > > > > > > > > will > > > > > > > > > > > > > > > > > > > > > hopefully > > > > > > > > > > > > > > > > > > > > > > > get it merged today. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023 at 5:26 AM > > Ivan > > > > Yurchenko < > > > > > > > > > > > > > > > > i...@ivanyu.me> > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish and all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wonder if > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-14993 > > > > > > > > > > > > > > > > > > > > > should be > > > > > > > > > > > > > > > > > > > > > > > > included in the 3.6 release > > plan. > > > > I'm > > > > > > > > > thinking > > > > > > > > > > > > that when > > > > > > > > > > > > > > > > > > > > > implemented, it > > > > > > > > > > > > > > > > > > > > > > > > would be a small, but still a > > > > change in the > > > > > > > > > RSM > > > > > > > > > > > > contract: > > > > > > > > > > > > > > > > > throw > > > > > > > > > > > > > > > > > > > an > > > > > > > > > > > > > > > > > > > > > > > > exception instead of returning > > an > > > > empty > > > > > > > > > > > > InputStream. > > > > > > > > > > > > > > > Maybe > > > > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > > > > > be > > > > > > > > > > > > > > > > > > > > > > > > included right away to save the > > > > migration > > > > > > > > > later? > > > > > > > > > > > > What do > > > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > > > > think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > > > > > Ivan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Sep 8, 2023, at 02:52, > > > > Satish Duggana > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Jose, > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for looking into this > > > > issue and > > > > > > > > > > > resolving > > > > > > > > > > > > it > > > > > > > > > > > > > > > with > > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > > > quick > > > > > > > > > > > > > > > > > > > > > fix. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 7 Sept 2023 at 21:40, > > > > José Armando > > > > > > > > > > > García > > > > > > > > > > > > > > > Sancio > > > > > > > > > > > > > > > > > > > > > > > > > <jsan...@confluent.io.invalid > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 6, 2023 at > > 4:58 PM > > > > Satish > > > > > > > > > > > Duggana < > > > > > > > > > > > > > > > > > > > > > > > > satish.dugg...@gmail.com> > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Greg, > > > > > > > > > > > > > > > > > > > > > > > > > > > It seems > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-14273 > > > > > > > > > > > > > > > > > > > > has > > > > > > > > > > > > > > > > > > > > > > > been > > > > > > > > > > > > > > > > > > > > > > > > > > > there in 3.5.x too. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I also agree that it > > should be > > > > a blocker > > > > > > > > > for > > > > > > > > > > > > 3.6.0. > > > > > > > > > > > > > > > It > > > > > > > > > > > > > > > > > should > > > > > > > > > > > > > > > > > > > > > have > > > > > > > > > > > > > > > > > > > > > > > > > > been a blocker for those > > > > previous > > > > > > > > > releases. I > > > > > > > > > > > > didn't > > > > > > > > > > > > > > > > fix > > > > > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > > > > > > because, > > > > > > > > > > > > > > > > > > > > > > > > > > unfortunately, I wasn't > > aware > > > > of the > > > > > > > > > issue > > > > > > > > > > > and > > > > > > > > > > > > jira. > > > > > > > > > > > > > > > > > > > > > > > > > > I'll create a PR with a > > fix in > > > > case the > > > > > > > > > > > > original > > > > > > > > > > > > > > > author > > > > > > > > > > > > > > > > > > > doesn't > > > > > > > > > > > > > > > > > > > > > > > > respond in time. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Satish, do you agree? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > -José > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > -David > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
