Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-14 Thread Manikumar
Hi Randall and Konstantine, Thanks for the explanation. Since it block users upgrading from older Connect versions to 2.4 release, we will consider the JIRA as blocker for 2.4.0. Thanks On Wed, Nov 13, 2019 at 11:58 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > Hi Manikumar, >

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-13 Thread Randall Hauch
Ismael, Konstantine replied to this thread just before your last email, and he happened to provided more justification and reasoning as to why we think this should be a blocker -- most importantly because it will cause failures for anyone upgrading from 2.2.x and using Connect's DLQ. Please let m

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-13 Thread Ismael Juma
Randall, Generally, if it's not a regression, it needs strong justification to be considered a blocker. The fact that it's been there for a whole release cycle gives some credence that it's not an obvious blocker. If you think it's a blocker, you can suggest so and provide a bit more detail on the

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-13 Thread Konstantine Karantasis
Hi Manikumar, thanks for your quick response on the Connect issue. I'd like to help shedding some more light here. The issue is clearly a regression. It may block users upgrading from older Connect versions to the most recent one. Unfortunately it was introduced in an area of the configuration th

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-13 Thread Randall Hauch
Hi, Manikumar. Why does the fact that "this is an existing issue" imply it should not be treated as a blocker? Clearly, some users will experience a problem when upgrading to AK 2.4, and the fact that we missed this problem in AK 2.3 IMO should not, by itself, result in this issue being considered

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-12 Thread Sophie Blee-Goldman
Hi Manikumar, We have another potential blocker in 2.4, which affects Streams members that fall out of the group and may cause them to die upon rejoin. The ticket is here https://issues.apache.org/jira/browse/KAFKA-9178 The fix is very straightforward, and we have a small PR out for it here http

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-12 Thread Manikumar
Hi Chris, Since this is existing issue, I am hesitant to include as blocker for 2.4 release. I am in the process of publishing RC0 for 2.4. We will try to include KAFKA-9046, if there any other blockers, which requires roll out of new RC. Thanks, On Wed, Nov 13, 2019 at 5:46 AM John Roesler wr

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-12 Thread John Roesler
Hi again Manikumar, Thanks for looking at the PR. It's not easy to pin this particular bug down to a specific commit, but I've ported my test to the 2.3 branch. It looks like this bug pre-dates 2.4, so it's not a regression. Therefore, after reflection, we don't need to block the 2.4 release on h

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-12 Thread Chris Egerton
Hi Manikumar, I'd like to propose the following issue as a blocker: https://issues.apache.org/jira/browse/KAFKA-9046 with a fix PR available at: https://github.com/apache/kafka/pull/7525 Essentially, if a Connect cluster is configured to use the DLQ feature against a secured Kafka cluster, it wi

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-12 Thread John Roesler
Hi Manikumar, There is a new bug report that we think is a blocker: https://issues.apache.org/jira/browse/KAFKA-9169 https://github.com/apache/kafka/pull/7681 See also the dev@ thread entitled "Why standby tasks read from the StandbyTasks::checkpointedOffsets in assignStandbyPartitions()" for co

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-11 Thread Manikumar
Hi all, I am trying to get stable system test runs on 2.4 branch. While analyzing the ReassignPartitionsTest failure, Rajini identified a issue, which can cause a successful produce to be failed. Thanks to Rajini for identifying the issue. https://issues.apache.org/jira/browse/KAFKA-9171 https://

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-04 Thread Manikumar
Hi all, The underlying issue of KAFKA-8677 can cause data loss in consumers. I have included KAFKA-8677 as blocker to 2.4 release. Thanks to Guozhang for identifying the issue. https://issues.apache.org/jira/browse/KAFKA-8677 PR: https://github.com/apache/kafka/pull/7613 Thanks On Fri, Nov 1, 2

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-01 Thread Manikumar
Hi All, We still have couple of blockers to close. PRs available for both the blockers. https://issues.apache.org/jira/browse/KAFKA-8972 https://issues.apache.org/jira/browse/KAFKA-9080 Thanks, On Fri, Oct 25, 2019 at 10:48 PM Manikumar wrote: > Hi all, > > Quick update on the 2.4 release.

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-25 Thread Manikumar
Hi all, Quick update on the 2.4 release. We still have one blocker to close. I will create the first RC after closing the blocker. https://issues.apache.org/jira/browse/KAFKA-8972 Thank you! On Fri, Oct 18, 2019 at 12:51 AM Matthias J. Sax wrote: > Just FYI: > > There was also https://issues.

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-17 Thread Matthias J. Sax
Just FYI: There was also https://issues.apache.org/jira/browse/KAFKA-9058 that I just merged. -Matthias On 10/17/19 7:59 AM, Manikumar wrote: > Hi all, > > The code freeze deadline has now passed and at this point only blockers > will be allowed. > We have three blockers for 2.4.0. I will move

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-17 Thread Manikumar
Hi all, The code freeze deadline has now passed and at this point only blockers will be allowed. We have three blockers for 2.4.0. I will move out most of the JIRAs that aren't currently being worked on. If you think any of the other JIRAs are critical to include in 2.4.0, please update the fix ve

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-08 Thread Manikumar
Thanks Bruno. We will mark KIP-471 as complete. On Tue, Oct 8, 2019 at 2:39 PM Bruno Cadonna wrote: > Hi Manikumar, > > It is technically true that KIP-471 is not completed, but the only > aspect that is not there are merely two metrics that I could not add > due to the RocksDB version currently

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-08 Thread Bruno Cadonna
Hi Manikumar, It is technically true that KIP-471 is not completed, but the only aspect that is not there are merely two metrics that I could not add due to the RocksDB version currently used in Streams. Adding those two metrics once the RocksDB version will have been increased, will be a minor ef

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-07 Thread Ismael Juma
Thanks Manikumar. To be clear, we should be focusing on stabilization from now until the code freeze. It's OK to merge small PRs that complete KIPs that have been partially merged or that are effectively fixes, but features should generally target trunk and the next release. Ismael On Mon, Oct 7,

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-07 Thread Manikumar
Hi all, I have moved couple of accepted KIPs without a PR to the next release. We still have quite a few KIPs with PRs that are being reviewed, but haven't yet been merged. I have left all of these in assuming these PRs are ready and not risky to merge. Please update your assigned KIPs/JIRAs, if

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-10-02 Thread Manikumar
Hi all, Let's extend the feature freeze deadline to Friday to merge the current work in-progress PRs. Please ensure that all major features have been merged and that any minor features have PRs by EOD Friday. We will be cutting the 2.4 release branch early on Monday morning. Thanks, Manikumar On

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-26 Thread Manikumar
Hi Gwen, As suggested by Ismael, If required, we can extend the feature freeze to end of the week. Thanks, On Thu, Sep 26, 2019 at 7:32 AM Ismael Juma wrote: > Hi Gwen, > > Good question. > > I think we should stick to the schedule. As usual, some features will slip > to the next release and t

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-26 Thread Manikumar
Hi all, I have included all the approved KIPs for 2.4.0 release. Thanks, On Thu, Sep 26, 2019 at 2:28 AM Randall Hauch wrote: > Hi, Manikumar. > > Thanks for acting as release manager. Can we please add the following KIPs > that have all been approved with PRs either merged or in progress:

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-25 Thread Ismael Juma
Hi Gwen, Good question. I think we should stick to the schedule. As usual, some features will slip to the next release and that's ok, they'll get a bit more time to stabilize. In the past, we have allowed 1-2 days over the feature freeze date (i.e. end of the week instead of Wednesday) for featur

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-25 Thread Gwen Shapira
Hi Mani and team, The feature freeze deadline is right next to Kafka Summit, making it a bit stressful for the committers and community members with PRs in flight. What do you think about delaying the feature freeze and code freeze by a week, to allow our community to enjoy Kafka Summit? Gwen On

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-25 Thread Randall Hauch
Hi, Manikumar. Thanks for acting as release manager. Can we please add the following KIPs that have all been approved with PRs either merged or in progress: - https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect - https:/

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-25 Thread Konstantine Karantasis
Hi Manikumar, let's add one more on the list that was voted on time and is straightforward: KIP-521: https://cwiki.apache.org/confluence/display/KAFKA/KIP-521%3A+Enable+redirection+of+Connect%27s+log4j+messages+to+a+file+by+default with jira ticket: https://issues.apache.org/jira/browse/KAFKA-560

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-25 Thread Chris Egerton
Hi Manikumar, Would it be possible to add KIP-507 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints) to the 2.4 release, as it has recently been accepted? Cheers, Chris On Tue, Sep 24, 2019 at 9:33 AM Kevin Lu wrote: > Hi Manikumar, > > Pl

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-24 Thread Kevin Lu
Hi Manikumar, Please add KIP-517 as it is now accepted too https://cwiki.apache.org/confluence/display/KAFKA/KIP-517%3A+Add+consumer+metrics+to+observe+user+poll+behavior Thanks. Regards, Kevin On Tue, Sep 24, 2019 at 7:25 AM Manikumar wrote: > Hi All, > > Just a reminder that tomorrow (Sep 2

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-24 Thread Manikumar
Hi All, Just a reminder that tomorrow (Sep 25, 2019) is the last date for KIP acceptance for 2.4.0 release. Thanks, On Tue, Sep 24, 2019 at 7:05 PM Manikumar wrote: > Hi Rajini, > > Added KIP-525 to the release plan for tracking. > > Thanks, > > > On Tue, Sep 24, 2019 at 4:36 PM Rajini Sivaram

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-24 Thread Manikumar
Hi Rajini, Added KIP-525 to the release plan for tracking. Thanks, On Tue, Sep 24, 2019 at 4:36 PM Rajini Sivaram wrote: > Hi Manikumar, > > Can we add KIP-525 ( > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response > ) > to

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-24 Thread Rajini Sivaram
Hi Manikumar, Can we add KIP-525 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-525+-+Return+topic+metadata+and+configs+in+CreateTopics+response) to 2.4.0 release? It has been accepted and is a minor KIP. Thank you, Rajini On Fri, Sep 20, 2019 at 6:14 AM Manikumar wrote: > Hi, > > A

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-19 Thread Manikumar
Hi, Added KIP-401 to the wiki page for tracking. Thanks, On Fri, Sep 20, 2019 at 7:55 AM Paul Whalen wrote: > Manikumar, > > KIP-401 was accepted a few weeks ago and there is a PR pending review, can > it be included in the release as well? > > Thanks, > Paul > > On Mon, Sep 16, 2019 at 6:14

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-19 Thread Paul Whalen
Manikumar, KIP-401 was accepted a few weeks ago and there is a PR pending review, can it be included in the release as well? Thanks, Paul On Mon, Sep 16, 2019 at 6:14 AM Manikumar wrote: > Hi All, > > Just a reminder that any new/pending KIP must pass vote by next Wednesday > (Sep 25, 2019) to

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-16 Thread Manikumar
Hi All, Just a reminder that any new/pending KIP must pass vote by next Wednesday (Sep 25, 2019) to be included in Apache Kafka 2.4.0 release. Also keep in mind that deadline for feature freeze is Oct 2, 2019. In order to be included in the release, major features/KIPs need to be merged and minor

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-16 Thread Manikumar
Hi Mickael, Yes, we can include. Added KIP-396 to the wiki page for tracking. Thanks, Manikumar On Mon, Sep 16, 2019 at 3:36 PM Mickael Maison wrote: > Hi Manikumar, > > Can we also include KIP-396? > (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484 > ) > It has been

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-16 Thread Mickael Maison
Hi Manikumar, Can we also include KIP-396? (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484) It has been accepted and the PR is ready for review: https://github.com/apache/kafka/pull/7296 Thanks On Mon, Sep 16, 2019 at 10:56 AM Manikumar wrote: > > Hi Viktor, > > Yes,

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-16 Thread Manikumar
Hi Viktor, Yes, we can include KIP-434. Thanks, On Mon, Sep 16, 2019 at 3:09 PM Viktor Somogyi-Vass wrote: > Hi Manikumar, > > Can we please also include KIP-434? > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+Cleaner+Count+Metrics > It has been ac

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-16 Thread Viktor Somogyi-Vass
Hi Manikumar, Can we please also include KIP-434? https://cwiki.apache.org/confluence/display/KAFKA/KIP-434%3A+Add+Replica+Fetcher+and+Log+Cleaner+Count+Metrics It has been accepted and there is already a pull request under review. Thanks, Viktor On Fri, Sep 6, 2019 at 9:59 AM Manikumar wrote:

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-06 Thread Manikumar
Hi David, Yes, we can include KIP-511. KIP must be accepted by KIP Freeze date (Sep 25, 2019 ) Thanks, On Fri, Sep 6, 2019 at 12:53 PM David Jacot wrote: > Hi Manikumar, > > Could we add KIP-511 to the plan? I think it will make it. > > Thanks, > David > > On Tue, Aug 27, 2019 at 5:32 PM Man

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-06 Thread David Jacot
Hi Manikumar, Could we add KIP-511 to the plan? I think it will make it. Thanks, David On Tue, Aug 27, 2019 at 5:32 PM Manikumar wrote: > Hi all, > > I put together a draft release plan with Oct 2019 as the release month and > a list of KIPs that have already been voted: > > https://cwiki.apac

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-08-27 Thread Manikumar
Hi all, I put together a draft release plan with Oct 2019 as the release month and a list of KIPs that have already been voted: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=125307901 Here are the dates: 1) KIP Freeze: Sep 25, 2019 (A KIP must be accepted by this date in ord

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-08-12 Thread Ismael Juma
Thanks for volunteering Manikumar. +1 Ismael On Mon, Aug 12, 2019 at 7:54 AM Manikumar wrote: > Hi all, > > I would like to volunteer to be the release manager for our next time-based > feature release (v2.4.0). > > If that sounds good, I'll post the release plan over the next few days. > > Tha

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-08-12 Thread Colin McCabe
+1. Thanks, Manikumar. Colin On Mon, Aug 12, 2019, at 08:25, Matthias J. Sax wrote: > Thanks Manikumar! SGTM. > > > On 8/12/19 7:54 AM, Manikumar wrote: > > Hi all, > > > > I would like to volunteer to be the release manager for our next time-based > > feature release (v2.4.0). > > > > If tha

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-08-12 Thread Matthias J. Sax
Thanks Manikumar! SGTM. On 8/12/19 7:54 AM, Manikumar wrote: > Hi all, > > I would like to volunteer to be the release manager for our next time-based > feature release (v2.4.0). > > If that sounds good, I'll post the release plan over the next few days. > > Thanks, > Manikumar > signature

[DISCUSS] Apache Kafka 2.4.0 release

2019-08-12 Thread Manikumar
Hi all, I would like to volunteer to be the release manager for our next time-based feature release (v2.4.0). If that sounds good, I'll post the release plan over the next few days. Thanks, Manikumar