[VOTE] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-08-16 Thread Vahid S Hashemian
I would like to start a vote on KIP-325 which aims at adding a beginning offset column to consumer group command describe output. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-325%3A+Extend+Consumer+Group+Command+to+Show+Beginning+Offsets Discussion thread: https://www.mail-arc

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-08-16 Thread Vahid S Hashemian
who provided feedback and / or voted on this KIP. --Vahid From: "Vahid S Hashemian" To: dev Date: 08/07/2018 11:17 AM Subject:[VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer Hi all, I'd like to start a vote on KIP-289 to modify the defa

Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-08-15 Thread Vahid S Hashemian
better fit. If there is no additional comment/feedback I'll start a vote soon. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 08/07/2018 01:58 PM Subject:Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-08-13 Thread Vahid S Hashemian
oses. cheers, Colin On Tue, Aug 7, 2018, at 11:14, Vahid S Hashemian wrote: > Hi all, > > I'd like to start a vote on KIP-289 to modify the default group id of > KafkaConsumer. > The KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-289%3A+Improve+the+defaul

Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-08-07 Thread Vahid S Hashemian
=87296412 ** Less processing overhead compared to Option 1 ** Partition size can be provided via kafka-topics tool For further info, please see the recent discussions on the thread. Thanks! --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 07/03/2018 09:11

[VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-08-07 Thread Vahid S Hashemian
Hi all, I'd like to start a vote on KIP-289 to modify the default group id of KafkaConsumer. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-289%3A+Improve+the+default+group+id+behavior+in+KafkaConsumer The discussion thread: https://www.mail-archive.com/dev@kafka.apache.org/msg8

Re: [VOTE] KIP-341: Update Sticky Assignor's User Data Protocol

2018-08-06 Thread Vahid S Hashemian
The KIP passes with 3 binding and 2 non-binding +1 votes, and no -1 or 0 votes. Binding +1s * Jason Gustafson * Dong Lin * Rajini Sivaram Non-binding +1s * Ted Yu * Mike Freyberger Thank you for providing feedback or vote on this KIP. --Vahid From: "Vahid S Hashemian" To:

Re: ConsumerGroupCommand tool improvement?

2018-08-06 Thread Vahid S Hashemian
l rest my case. Thanks! --Vahid From: Colin McCabe To: us...@kafka.apache.org Date: 08/06/2018 12:44 PM Subject:Re: ConsumerGroupCommand tool improvement? On Mon, Aug 6, 2018, at 12:04, Vahid S Hashemian wrote: > Hi Colin, > > Thanks for considering the idea and sh

Re: ConsumerGroupCommand tool improvement?

2018-08-06 Thread Vahid S Hashemian
d? best, Colin On Thu, Aug 2, 2018, at 14:23, Vahid S Hashemian wrote: > Hi all, > > A requirement has been raised by a colleague and I wanted to see if there > is any interest in the community in adding the functionality to Apache > Kafka. > > ConsumerGroupComma

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-08-03 Thread Vahid S Hashemian
The KIP has been updated. If it looks good and there are no further comments I'll start a vote early next week. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 08/03/2018 08:19 AM Subject:Re: [DISCUSS] KIP-289: Improve the default gro

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-08-03 Thread Vahid S Hashemian
default group.id. > > Thanks, > Jason > > On Thu, Aug 2, 2018 at 11:55 AM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > Hi Jason, Colin, > > > > Thanks for your feedback. > > > > If we are going to keep supporting the empty

ConsumerGroupCommand tool improvement?

2018-08-02 Thread Vahid S Hashemian
Hi all, A requirement has been raised by a colleague and I wanted to see if there is any interest in the community in adding the functionality to Apache Kafka. ConsumerGroupCommand tool in describe ('--describe' or '--describe --offsets') mode currently lists all topics the group has consumed

[VOTE] KIP-341: Update Sticky Assignor's User Data Protocol

2018-08-02 Thread Vahid S Hashemian
Hi everyone, I believe the feedback on this KIP has been addressed so far. So I'd like to start a vote. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-341%3A+Update+Sticky+Assignor%27s+User+Data+Protocol Discussion thread: https://www.mail-archive.com/dev@kafka.apache.org/msg897

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-08-02 Thread Vahid S Hashemian
of the ACL problem. I guess we could just deprecate the use on the > client. What do you think? > > Thanks, > Jason > > On Wed, Aug 1, 2018 at 3:19 PM, Vahid S Hashemian > wrote: > > > Thanks Jason for responding to Colin's concerns. > > >

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

2018-08-01 Thread Vahid S Hashemian
If there is no additional feedback on this KIP I'll start a vote soon. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 07/18/2018 04:47 PM Subject:Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol The KIP is u

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-08-01 Thread Vahid S Hashemian
hid, > > > > I was thinking we'd only use the old API version if we had to. That is, > > only if the user has explicitly configured "" as the group.id. > Otherwise, > > we'd just use the new one. Another option is to just drop support in t

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

2018-07-30 Thread Vahid S Hashemian
Such a good news on a Monday morning ... Thank you Rajini for driving the release! --Vahid From: Mickael Maison To: Users Cc: dev , annou...@apache.org, kafka-clients Date: 07/30/2018 04:37 AM Subject:Re: [ANNOUNCE] Apache Kafka 2.0.0 Released Great news! Thanks for

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-07-26 Thread Vahid S Hashemian
or changes like this. -Jason On Wed, Jul 25, 2018 at 12:49 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Jason, > > Thanks for additional clarification. > > So the next version of the OffsetCommit API will return an > INVALID_GROUP_ID error for empty group ids;

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-07-25 Thread Vahid S Hashemian
support and only use the latest OffsetCommit version. Does that make sense? Thanks, Jason On Tue, Jul 24, 2018 at 12:36 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Jason, > > Thanks for clarifying. > > So if we are going to continue supporting the empty group

Re: [VOTE] 2.0.0 RC3

2018-07-24 Thread Vahid S Hashemian
+1 (non-binding) Built from source and ran quickstart successfully with both Java 8 and Java 9 on Ubuntu. Thanks Rajini! --Vahid From: Rajini Sivaram To: dev , Users , kafka-clients Date: 07/24/2018 08:33 AM Subject:[VOTE] 2.0.0 RC3 Hello Kafka users, developers and cli

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-07-24 Thread Vahid S Hashemian
23, 2018 at 5:14 PM, Stanislav Kozlovski wrote: > Hey Vahid, > > No I don't see an issue with it. I believe it to be the best approach. > > Best, > Stanisav > > On Mon, Jul 23, 2018 at 12:41 PM Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > >

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-07-23 Thread Vahid S Hashemian
re using both the default group.id and the default > > enable.auto.commit. Perhaps one option is to disable enable.auto.commit > > automatically if no group.id is specified? I am not sure if there are > any > > drawbacks, but my feeling is that implicit use of the empty grou

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-07-23 Thread Vahid S Hashemian
ny > > drawbacks, but my feeling is that implicit use of the empty group.id to > > commit offsets is more likely to be causing users unexpected problems > than > > actually providing a useful capability. > > > > Thoughts? > > > > Thanks, > > Jason

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-07-23 Thread Vahid S Hashemian
. Thoughts? Thanks, Jason On Mon, May 28, 2018 at 9:50 AM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Viktor, > > Thanks for sharing your opinion. > So you're in favor of disallowing the empty ("") group id altogether (even > for fetching

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

2018-07-18 Thread Vahid S Hashemian
The KIP is updated to follow the suggestion of using consumer group generation. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 07/17/2018 02:32 PM Subject:Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol Hi Jason,

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

2018-07-17 Thread Vahid S Hashemian
d of the schema, but I guess we're hoping this is the last change ;-) Thanks, Jason On Fri, Jul 13, 2018 at 10:07 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Mike, > > > > Thanks a lot for reviewing the KIP and sharing your feedback. > > I agree that s

Re: [VOTE] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

2018-07-16 Thread Vahid S Hashemian
+1 (non-binding) Thanks! --Vahid From: Chia-Ping Tsai To: Date: 07/16/2018 01:11 AM Subject:[VOTE] KIP-338 Support to exclude the internal topics in kafka-topics.sh command hi folks, The discussion[1] of KIP-338[2] did not get any objection for last 6 days so it is time

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

2018-07-13 Thread Vahid S Hashemian
or, which also requires some kind of epoch/generation marker to be protected against zombies. So, I’d be in favor of a generic solution here that other assignors can leverage. Best, Mike Freyberger > On Jul 13, 2018, at 6:15 PM, Vahid S Hashemian wrote: > > Hi all, > > I create a

[DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

2018-07-13 Thread Vahid S Hashemian
Hi all, I create a short KIP to address an issue in Sticky Assignor assignment logic: https://cwiki.apache.org/confluence/display/KAFKA/KIP-341%3A+Update+Sticky+Assignor%27s+User+Data+Protocol Please take a look and share your feedback / comments. In particular, there is a Generation Marker sec

Re: [VOTE] 2.0.0 RC2

2018-07-11 Thread Vahid S Hashemian
+1 (non-binding) Built executables from source and ran quickstart (Ubuntu / Java 8) Thanks! --Vahid From: Brett Rann To: dev@kafka.apache.org Cc: Users , kafka-clients Date: 07/10/2018 09:53 PM Subject:Re: [VOTE] 2.0.0 RC2 +1 (non binding) rolling upgrade of tiny sha

Re: [VOTE] KIP-264: Add a consumer metric to record raw fetch size

2018-07-10 Thread Vahid S Hashemian
Bump! From: "Vahid S Hashemian" To: dev Date: 05/25/2018 10:51 AM Subject:[VOTE] KIP-264: Add a consumer metric to record raw fetch size In the absence of additional feedback on this KIP I'd like to start a vote. To summarize, the KIP simply proposes to

Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups API

2018-07-09 Thread Vahid S Hashemian
KIP-231 is approved with 3 "+1" binding votes by Ewen, Jason, and Harsha. Thanks to everyone who provided feedback and/or voted. Regards. --Vahid From: Vahid S Hashemian/Silicon Valley/IBM To: dev Date: 12/19/2017 11:30 AM Subject:[VOTE] KIP-231: Improve the Requi

Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups API

2018-07-06 Thread Vahid S Hashemian
Bump! From: "Vahid S Hashemian" To: dev Date: 06/01/2018 10:21 AM Subject:Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups API I'm bumping this vote thread up as the KIP requires only one binding +1 to pass. The KIP is very similar in nature

Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-07-03 Thread Vahid S Hashemian
lf solution. I'd probably suggest leaving this out or just reporting the absolute difference even if a topic is compacted. -Jason On Thu, Jun 28, 2018 at 1:05 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi James, > > > > Thanks for the feedback. I updated

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

2018-07-03 Thread Vahid S Hashemian
+1 (non-binding) --Vahid From: Gwen Shapira To: dev Date: 07/03/2018 08:49 AM Subject:Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled. +1 On Tue, Jul 3, 2018 at 8:24 AM, Manikumar wrote: > Manikumar > Fri, Jun 29, 7:59 PM (4

Re: [VOTE] 2.0.0 RC1

2018-07-02 Thread Vahid S Hashemian
+1 (non-binding) Built from source and ran quickstart successfully on Ubuntu (with Java 8). Minor: It seems this doc update PR is not included in the RC: https://github.com/apache/kafka/pull/5280 Guozhang seems to have wanted to cherry-pick it to 2.0. Thanks Rajini! --Vahid From: Rajini S

Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-06-28 Thread Vahid S Hashemian
from my iPhone > On Jun 26, 2018, at 11:23 AM, Vahid S Hashemian wrote: > > Hi everyone, > > I have created a trivial KIP to improve the offset reporting of the > consumer group command: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-325%3A+Extend+Consumer+

Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-06-26 Thread Vahid S Hashemian
still > be valuable. If not difficult, leaving this empty for compacted topics to > avoid confusion will also be nice. > > On Tue, Jun 26, 2018 at 4:29 PM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > Hi Gwen, > > > > Thanks for the fe

Re: [DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-06-26 Thread Vahid S Hashemian
ommand to Show Beginning Offsets Small suggestion: you can also add a "partition size" column - difference between log-end and log-start. We've had users ask for this. On Tue, Jun 26, 2018 at 2:34 PM, Gwen Shapira wrote: > This will be useful! Thank you :) > > On Tue, Jun

[DISCUSS] KIP-325: Extend Consumer Group Command to Show Beginning Offsets

2018-06-26 Thread Vahid S Hashemian
Hi everyone, I have created a trivial KIP to improve the offset reporting of the consumer group command: https://cwiki.apache.org/confluence/display/KAFKA/KIP-325%3A+Extend+Consumer+Group+Command+to+Show+Beginning+Offsets Looking forward to your feedback! Thanks. --Vahid

Re: [VOTE] 0.11.0.3 RC0

2018-06-22 Thread Vahid S Hashemian
0.11.0-jdk7/385/ System tests: https://jenkins.confluent.io/job/system-test-kafka/job/0.11.0/217/ /** Thanks, -Matthias [attachment "signature.asc" deleted by Vahid S Hashemian/Silicon Valley/IBM]

Re: [VOTE] 2.0.0 RC0

2018-06-22 Thread Vahid S Hashemian
+1 (non-binding) Built from source and ran quickstart successfully on Ubuntu (with Java 8 and Java 9). Thanks Rajini! --Vahid

Re: [VOTE] 1.1.1 RC1

2018-06-22 Thread Vahid S Hashemian
+1 (non-binding) Built from source and ran quickstart successfully on Ubuntu (with Java 8). Thanks Dong! --Vahid From: Dong Lin To: dev@kafka.apache.org, us...@kafka.apache.org, kafka-clie...@googlegroups.com Date: 06/22/2018 10:10 AM Subject:[VOTE] 1.1.1 RC1 Hello Kafka u

Re: [VOTE] 1.0.2 RC0

2018-06-22 Thread Vahid S Hashemian
1/ System tests: https://jenkins.confluent.io/job/system-test-kafka/job/1.0/217/ /** Thanks, -Matthias [attachment "signature.asc" deleted by Vahid S Hashemian/Silicon Valley/IBM]

Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups API

2018-06-01 Thread Vahid S Hashemian
APIs' minimum required permissions more consistent. Thanks. --Vahid From: Vahid S Hashemian/Silicon Valley/IBM To: dev Date: 12/19/2017 11:30 AM Subject:[VOTE] KIP-231: Improve the Required ACL of ListGroups API I believe the concerns on this KIP have been addres

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-05-28 Thread Vahid S Hashemian
reak altogether but ofc that's just my 2 cents :). Let's gather other's input as well. Cheers, Viktor On Fri, May 25, 2018 at 5:43 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Victor, > > Thanks for reviewing the KIP. > > Yes, to minimize th

[VOTE] KIP-264: Add a consumer metric to record raw fetch size

2018-05-25 Thread Vahid S Hashemian
In the absence of additional feedback on this KIP I'd like to start a vote. To summarize, the KIP simply proposes to add a consumer metric to track the size of raw (uncompressed) fetched messages. The KIP can be found here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-264%3A+Add+a+cons

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-05-25 Thread Vahid S Hashemian
2? Can't we fail with an empty group id even in "fetch" or "fetch_offset"? What was the reason for deciding to fail at "offset_commit"? Was it because of upgrade compatibility reasons? Thanks, Viktor On Thu, May 24, 2018 at 12:06 AM, Ted Yu wrote: > Looks goo

Re: [DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-05-23 Thread Vahid S Hashemian
bq. If they attempt an offset commit they will receive an error. Can you outline what specific error would be encountered ? Thanks On Fri, Apr 27, 2018 at 2:17 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi all, > > I have drafted a proposal for improving

[DISCUSS] KIP-289: Improve the default group id behavior in KafkaConsumer

2018-04-27 Thread Vahid S Hashemian
Hi all, I have drafted a proposal for improving the behavior of KafkaConsumer when using the default group id: https://cwiki.apache.org/confluence/display/KAFKA/KIP-289%3A+Improve+the+default+group+id+behavior+in+KafkaConsumer The proposal based on the issue and suggestion reported in KAFKA-6774

Re: [DISCUSS] KIP-264: Add a consumer metric to record raw fetch size

2018-04-26 Thread Vahid S Hashemian
If there is no additional feedback on this KIP, I'll start a vote soon. --Vahid From: "Vahid S Hashemian" To: dev Date: 02/27/2018 12:44 PM Subject:[DISCUSS] KIP-264: Add a consumer metric to record raw fetch size Hi all, I started a short, and hopefully

Re: [VOTE] KIP-277 - Fine Grained ACL for CreateTopics API

2018-04-26 Thread Vahid S Hashemian
+1 (non-binding) Thanks Edo. --Vahid From: Rajini Sivaram To: dev Date: 04/26/2018 12:28 AM Subject:Re: [VOTE] KIP-277 - Fine Grained ACL for CreateTopics API Hi Edo. Thanks for the KIP. +1 (binding) On Thu, Apr 26, 2018 at 2:38 AM, Stephane Maarek < steph...@simplemachin

Re: [VOTE] Kafka 2.0.0 in June 2018

2018-04-19 Thread Vahid S Hashemian
+1 (non-binding). Thanks Ismael. --Vahid From: Jorge Esteban Quilcate Otoya To: dev@kafka.apache.org Date: 04/19/2018 07:32 AM Subject:Re: [VOTE] Kafka 2.0.0 in June 2018 +1 (non binding), thanks Ismael! El jue., 19 abr. 2018 a las 13:01, Manikumar () escribió: > +1 (non-

Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API

2018-04-17 Thread Vahid S Hashemian
is the preferred way to go for backward compatibility. Though we'd like to plan a deprecation for the Create Cluster check, if wildcard support in ACLs will be added in the future. thoughts ? -- Edoardo Comar IBM Message Hub IBM UK Ltd, Hursley Park,

Re: [VOTE] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-04-09 Thread Vahid S Hashemian
@kafka.apache.org/msg86478.html Thanks. --Vahid From: Vahid S Hashemian/Silicon Valley/IBM To: dev Date: 03/28/2018 10:34 AM Subject:[VOTE] KIP-211: Revise Expiration Semantics of Consumer Group Offsets Hi all, As I believe the feedback and suggestions on this KIP have been

Re: Kafka KIP meeting on Apr. 9 at 9:00am PDT

2018-04-05 Thread Vahid S Hashemian
Hi Jun, I used to receive these invites, but didn't get this one. Please send me an invite. Thanks. Regards, --Vahid From: Jun Rao To: dev Date: 04/05/2018 10:25 AM Subject:Kafka KIP meeting on Apr. 9 at 9:00am PDT Hi, Everyone, We plan to have a Kafka KIP meeting this co

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-04-05 Thread Vahid S Hashemian
own partition assignors, too, with other names. -Matthias On 3/26/18 1:44 PM, Vahid S Hashemian wrote: > Hi all, > > Thanks for the feedback on this KIP so far. > > If there is no additional feedback, I'll start a vote on Wed. > > Thanks. > --Vahid > > [attachment "signature.asc" deleted by Vahid S Hashemian/Silicon Valley/IBM]

Re: [DISCUSS] KIP-277 - Fine Grained ACL for CreateTopics API

2018-04-04 Thread Vahid S Hashemian
Hi Edo, Mickael, The intent of this KIP seems to be rather similar to KIP-231 (Improve the Required ACL of ListGroups API). The feedback I received on that KIP was to allow for backward compatibility, and, as a result, the Describe(Cluster) ACL was preserved; and a Describe(Group) ACL was intro

Re: Kafka Authorizer interface review

2018-04-03 Thread Vahid S Hashemian
Hi Mickael, Thanks for detailed description on these authorization issues. I agree they need to be reviewed and fixed in the areas you specified, or even at a higher that simplifies their maintenance as the matrix is expanded or needs to be modified. FYI, KIP-231 also attempts at addressing the

Re: [ANNOUNCE] New Committer: Dong Lin

2018-03-28 Thread Vahid S Hashemian
Congratulations Dong! Well deserved. From: Bill Bejeck To: dev@kafka.apache.org Date: 03/28/2018 11:04 AM Subject:Re: [ANNOUNCE] New Committer: Dong Lin Congrats Dong! On Wed, Mar 28, 2018 at 1:58 PM, Ted Yu wrote: > Congratulations, Dong. > > On Wed, Mar 28, 2018 at 10:58

[VOTE] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-03-28 Thread Vahid S Hashemian
Hi all, As I believe the feedback and suggestions on this KIP have been addressed so far, I'd like to start a vote. The KIP can be found at https://cwiki.apache.org/confluence/display/KAFKA/KIP-211%3A+Revise+Expiration+Semantics+of+Consumer+Group+Offsets Thanks in advance for voting :) --Vahid

Re: [kafka-clients] Re: [VOTE] 1.1.0 RC4

2018-03-27 Thread Vahid S Hashemian
Hi Rajini, +1 (non-binding) Built from source on Linux and Windows (Java 8), and tested quickstart on both platforms. Connect quickstart on Windows is not working as per my note on RC3: https://www.mail-archive.com/dev@kafka.apache.org/msg86138.html No other issues detected. Thanks! --Vahid

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-03-26 Thread Vahid S Hashemian
Hi all, Thanks for the feedback on this KIP so far. If there is no additional feedback, I'll start a vote on Wed. Thanks. --Vahid

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-03-19 Thread Vahid S Hashemian
coordinator change and the group is unloaded from the coordinator cache. Thanks, Jason On Tue, Mar 6, 2018 at 4:41 PM, Vahid S Hashemian wrote: > Hi Jason, > > Thanks a lot for your clarification and feedback. > Your statements below all seem reasonable to me. > > I have updat

Re: Subject: [VOTE] 1.1.0 RC3

2018-03-16 Thread Vahid S Hashemian
Hi Damian, Thanks for running the release. I tried building from source and running the quick start on Linux & Windows with both Java 8 & 9. Here's the result: +-+-+-+ | | Linux | Windows | + +-+-+ |

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-03-06 Thread Vahid S Hashemian
apturing the subscription change? Yes, I think that can work. So we would expire offsets for a consumer group individually if they have reached the retention time and the group is not empty, but is no longer subscribed to them. Is that right? Thanks, Jason On Fri, Mar 2, 2018 at 3:36 PM, Va

Re: [VOTE] KIP-186: Increase offsets retention default to 7 days

2018-03-06 Thread Vahid S Hashemian
+1 (non-binding) Thanks Ewen. --Vahid From: Ewen Cheslack-Postava To: dev@kafka.apache.org Date: 03/05/2018 11:35 AM Subject:[VOTE] KIP-186: Increase offsets retention default to 7 days I'd like to kick off voting for KIP-186: https://urldefense.proofpoint.com/v2/url?u=htt

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-03-02 Thread Vahid S Hashemian
mon this use case is and whether it needs to be solved as part of this KIP. Thanks, Jason On Thu, Feb 1, 2018 at 12:40 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Thanks James for sharing that scenario. > > I agree it makes sense to be able to remove offsets

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-03-01 Thread Vahid S Hashemian
n the update is ready for review. Apologies for the confusion. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 03/01/2018 11:43 AM Subject:Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets Hi Jason, Thanks for your f

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-03-01 Thread Vahid S Hashemian
are always Empty. One additional note: I have seen recently a case where the offset cache caused an OOM on the broker. I looked into it and found that most of the cache was used for storing console consumer offsets. I know you had a patch before which turned off auto-commit when the gr

Re: [DISCUSS] KIP-264: Add a consumer metric to record raw fetch size

2018-02-27 Thread Vahid S Hashemian
e. Can you add description for that ? Looks good overall. On Tue, Feb 27, 2018 at 12:42 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi all, > > I started a short, and hopefully trivial, KIP to add a KafkaConsumer > metric: > https://urldefense.proofpoint.com/v

[DISCUSS] KIP-264: Add a consumer metric to record raw fetch size

2018-02-27 Thread Vahid S Hashemian
Hi all, I started a short, and hopefully trivial, KIP to add a KafkaConsumer metric: https://cwiki.apache.org/confluence/display/KAFKA/KIP-264%3A+Add+a+consumer+metric+to+record+raw+fetch+size Your feedback and suggestions are welcome! Thanks. --Vahid

Re: [VOTE] 1.1.0 RC0

2018-02-26 Thread Vahid S Hashemian
+1 (non-binding) Built the source and ran quickstart (including streams) successfully on Ubuntu (with both Java 8 and Java 9). I understand the Windows platform is not officially supported, but I ran the same on Windows 10, and except for Step 7 (Connect) everything else worked fine. There ar

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-02-23 Thread Vahid S Hashemian
ase is and whether it needs to be solved as part of this KIP. Thanks, Jason On Thu, Feb 1, 2018 at 12:40 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Thanks James for sharing that scenario. > > I agree it makes sense to be able to remove offsets for the topics t

Re: [VOTE] 1.0.1 RC2

2018-02-23 Thread Vahid S Hashemian
+1 Built source and ran quickstarts successfully on Ubuntu and Windows 64bit. Thanks. --Vahid From: Damian Guy To: dev@kafka.apache.org Date: 02/23/2018 01:42 AM Subject:Re: [VOTE] 1.0.1 RC2 +1 Built src and ran tests Ran streams quickstart On Thu, 22 Feb 2018 at 21:32 T

Re: Eclipse or Intellij workspace setup

2018-02-08 Thread Vahid S Hashemian
These articles may be useful: - https://developer.ibm.com/opentech/2016/06/06/how-to-develop-for-apache-kafka-using-eclipse/ - https://developer.ibm.com/opentech/2017/11/02/how-to-develop-for-apache-kafka-using-intellij-idea/ I personally prefer IntelliJ. Hope that helps. --Vahid From: R

Bay Area Apache Kafka Meetup - Morning of Feb 20

2018-02-07 Thread Vahid S Hashemian
Kafka users and developers, The next *Bay Area Apache Kafka Meetup* is on the *morning of Feb 20* and is hosted by *Index Developer Conference* at Moscone West in San Francisco . Meetup Info: https://www.meetup.com/KafkaBayArea/events/247433783/ Registration Link: https://ibm.co/2n742Jn (required

Re: [VOTE] 1.0.1 RC0

2018-02-07 Thread Vahid S Hashemian
Hi Ewen, +1 Building from source and running the quickstart were successful on Ubuntu and Windows 10. Thanks for running the release. --Vahid From: Ewen Cheslack-Postava To: dev@kafka.apache.org, us...@kafka.apache.org, kafka-clie...@googlegroups.com Date: 02/05/2018 07:49 PM Subje

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-02-01 Thread Vahid S Hashemian
> Plus it's more wire efficient because consumers no longer need to commit > offsets for partitions that have no new messages just to keep those offsets > alive. > > On Fri, Jan 12, 2018 at 10:21 AM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > >> T

Re: [DISCUSS] KIP-229: DeleteGroups API

2018-01-26 Thread Vahid S Hashemian
e know. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 01/12/2018 10:30 AM Subject:Re: [DISCUSS] KIP-229: DeleteGroups API If there are no additional comments/feedback on this KIP, I'll start a vote on Monday. Thanks. --Vahid From:

Re: 1.1 KIPs

2018-01-24 Thread Vahid S Hashemian
Hi Damian, Could you please add KIP-229 to the list? It was approved earlier this week https://www.mail-archive.com/dev@kafka.apache.org/msg84851.html Thanks for running the release. --Vahid From: Damian Guy To: dev@kafka.apache.org Date: 01/24/2018 01:20 PM Subject:Re: 1.1

Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups API

2018-01-23 Thread Vahid S Hashemian
s, but I'm not sure why that would be the case. Wouldn't a wildcard group describe permission enable that? That said, it's probably better to keep the old permission for backwards compatibility. -Jason On Mon, Jan 22, 2018 at 11:07 AM, Vahid S Hashemian < vahidhashem...@us.ibm

Re: [VOTE] KIP-229: DeleteGroups API

2018-01-23 Thread Vahid S Hashemian
Thanks to those who provided feedback and/or voted on this KIP. The KIP passes with the following vote count: * 3 binding +1's by Rajini, Guozhang, Jason * 5 non-binding +1's by Jeff, Ted, Edo, Mickael, Colin Regards. --Vahid From: "Vahid S Hashemian" To: dev@k

Re: [VOTE] KIP-231: Improve the Required ACL of ListGroups API

2018-01-22 Thread Vahid S Hashemian
n On Tue, Dec 19, 2017 at 11:30 AM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > I believe the concerns on this KIP have been addressed so far. > Therefore, I'd like to start a vote. > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confl

Re: [VOTE] KIP-229: DeleteGroups API

2018-01-22 Thread Vahid S Hashemian
Bumping this thread as this KIP requires only one more binding +1 to make it to 1.1.0. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 01/16/2018 10:41 AM Subject:Re: [VOTE] KIP-229: DeleteGroups API Thanks Colin. Your suggestion is fai

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

2018-01-17 Thread Vahid S Hashemian
Great news! Congratulations Rajini! --Vahid From: Gwen Shapira To: "dev@kafka.apache.org" , Users Date: 01/17/2018 10:49 AM Subject:[ANNOUNCE] New Kafka PMC Member: Rajini Sivaram Dear Kafka Developers, Users and Fans, Rajini Sivaram became a committer in April 2017. Sin

Re: [VOTE] KIP-229: DeleteGroups API

2018-01-16 Thread Vahid S Hashemian
t;> IBM UK Ltd, Hursley Park, SO21 2JN > > >> > > >> > > >> > > >> From: Ted Yu > > >> To: dev@kafka.apache.org > > >> Date: 15/01/2018 20:33 > > >> Subject:Re: [VOTE] KIP-229: DeleteGroups

Re: [VOTE] KIP-229: DeleteGroups API

2018-01-16 Thread Vahid S Hashemian
>> > >> From: Ted Yu > >> To: dev@kafka.apache.org > >> Date: 15/01/2018 20:33 > >> Subject:Re: [VOTE] KIP-229: DeleteGroups API > >> > >> > >> > >> +1 > >> > >> On Mon, Jan 15, 2018 at 1

[VOTE] KIP-229: DeleteGroups API

2018-01-15 Thread Vahid S Hashemian
Happy Monday, I believe the concerns on this KIP have been addressed in the current version of the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-229%3A+DeleteGroups+API So I'd like to start a vote. Thanks. --Vahid

Re: [DISCUSS] KIP-229: DeleteGroups API

2018-01-12 Thread Vahid S Hashemian
If there are no additional comments/feedback on this KIP, I'll start a vote on Monday. Thanks. --Vahid From: "Vahid S Hashemian" To: dev Date: 11/28/2017 04:03 PM Subject:[DISCUSS] KIP-229: DeleteGroups API Hi everyone, I started KIP-229 and proposed a

Re: [DISCUSS] KIP-211: Revise Expiration Semantics of Consumer Group Offsets

2018-01-12 Thread Vahid S Hashemian
retention setting proves to be problematic in the future. It was suggested earlier in the discussion thread that the KIP should propose a per-group retention config to circumvent this risk. I look forward to hearing your thoughts. Thanks! --Vahid From: "Vahid S Hashemian" To:

Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API

2018-01-03 Thread Vahid S Hashemian
f the ConsumerGroup permissions are weird, but this KIP brings the describe ACLs into better alignment with everything else and makes things more functional for clients with more locked down permissions. -Ewen On Fri, Dec 15, 2017 at 12:57 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> w

[VOTE] KIP-231: Improve the Required ACL of ListGroups API

2017-12-19 Thread Vahid S Hashemian
I believe the concerns on this KIP have been addressed so far. Therefore, I'd like to start a vote. https://cwiki.apache.org/confluence/display/KAFKA/KIP-231%3A+Improve+the+Required+ACL+of+ListGroups+API Thanks. --Vahid

Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public

2017-12-18 Thread Vahid S Hashemian
+1 Thanks for the KIP. --Vahid From: Ted Yu To: dev@kafka.apache.org Date: 12/18/2017 02:45 PM Subject:Re: [VOTE] KIP-243: Make ProducerConfig and ConsumerConfig constructors public +1 nit: via "copy and past" an 'e' is missing at the end. On Mon, Dec 18, 2017 at 2:38 PM

Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API

2017-12-15 Thread Vahid S Hashemian
If there are no other feedback or suggestion on this KIP, I'll start a vote early next week. Thanks. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 11/29/2017 03:18 PM Subject:Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups AP

Re: [DISCUSS] KIP-229: DeleteGroups API

2017-12-12 Thread Vahid S Hashemian
The KIP is updated based on the feedback so far. Apologies to Dong for the typo in his name in my earlier message! --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 12/12/2017 02:37 PM Subject:Re: [DISCUSS] KIP-229: DeleteGroups API Than

Re: [DISCUSS] KIP-229: DeleteGroups API

2017-12-12 Thread Vahid S Hashemian
I think this is the one Dong was referring to. I thought it might be a little premature since we don't have any other group APIs supported. That said, there is a separate proposal to expose list/describe functionality, so if that gets in, we could add support for deletion as well. -Jason On

Re: [DISCUSS] KIP-229: DeleteGroups API

2017-12-12 Thread Vahid S Hashemian
Hi Jason, Thanks for reviewing the KIP. Please see my responses inline. I'll make the necessary updates to the KIP and send another note once that's done. --Vahid From: Jason Gustafson To: dev@kafka.apache.org Date: 12/11/2017 03:43 PM Subject:Re: [DISCUSS] KIP-229: DeleteGro

Re: [DISCUSS] KIP-229: DeleteGroups API

2017-12-11 Thread Vahid S Hashemian
If there are no additional feedback on this KIP, I'll start a vote in a couple of days. Thanks. --Vahid From: Vahid S Hashemian/Silicon Valley/IBM To: dev@kafka.apache.org Date: 11/29/2017 11:15 AM Subject:Re: [DISCUSS] KIP-229: DeleteGroups API Hi Dong, Thanks a lo

Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API

2017-12-05 Thread Vahid S Hashemian
Describe Group access is in place) an empty list is returned without any authorization error. --Vahid From: "Vahid S Hashemian" To: dev@kafka.apache.org Date: 12/05/2017 02:42 PM Subject:Re: [DISCUSS] KIP-231: Improve the Required ACL of ListGroups API Hi Dong,

  1   2   3   >