Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-16 Thread Mickael Maison
Closing this vote as we are well over 72 hours :) The vote has passed with +6 binding votes (Guozhang, Vahid, Bill, Harsha, Colin and Jason) and +7 non binding votes (Gabor, Jungtaek, Ryanne, Andrew, Eno, Edo and Patrik). Thanks to everyone that reviewed and helped improve this proposal. On Thu,

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-15 Thread Guozhang Wang
+1 (binding). Thanks! Guozhang On Wed, Aug 14, 2019 at 5:18 PM Vahid Hashemian wrote: > +1 (binding) > > Thanks Michael for the suggestion of simplifying offset > retrieval/alteration operations. > > --Vahid > > On Wed, Aug 14, 2019 at 4:42 PM Bill Bejeck wrote: > > > Thanks for the KIP Mick

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Vahid Hashemian
+1 (binding) Thanks Michael for the suggestion of simplifying offset retrieval/alteration operations. --Vahid On Wed, Aug 14, 2019 at 4:42 PM Bill Bejeck wrote: > Thanks for the KIP Mickael, looks very useful. > +1 (binding) > > -Bill > > On Wed, Aug 14, 2019 at 6:14 PM Harsha Chintalapani >

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Bill Bejeck
Thanks for the KIP Mickael, looks very useful. +1 (binding) -Bill On Wed, Aug 14, 2019 at 6:14 PM Harsha Chintalapani wrote: > Thanks for the KIP Mickael. LGTM +1 (binding). > -Harsha > > > On Wed, Aug 14, 2019 at 1:10 PM, Colin McCabe wrote: > > > Thanks, Mickael. +1 (binding) > > > > best, >

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Harsha Chintalapani
Thanks for the KIP Mickael. LGTM +1 (binding). -Harsha On Wed, Aug 14, 2019 at 1:10 PM, Colin McCabe wrote: > Thanks, Mickael. +1 (binding) > > best, > Colin > > On Wed, Aug 14, 2019, at 12:07, Gabor Somogyi wrote: > > +1 (non-binding) > I've read it through in depth and as Jungtaek said Spark

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Colin McCabe
Thanks, Mickael. +1 (binding) best, Colin On Wed, Aug 14, 2019, at 12:07, Gabor Somogyi wrote: > +1 (non-binding) > I've read it through in depth and as Jungtaek said Spark can make good use > of it. > > On Wed, 14 Aug 2019, 17:06 Jungtaek Lim, wrote: > > > +1 (non-binding) > > > > I found i

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Gabor Somogyi
+1 (non-binding) I've read it through in depth and as Jungtaek said Spark can make good use of it. On Wed, 14 Aug 2019, 17:06 Jungtaek Lim, wrote: > +1 (non-binding) > > I found it very useful for Spark's case. (Discussion on KIP-505 described > it.) > > Thanks for driving the effort! > > 2019년

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Jungtaek Lim
+1 (non-binding) I found it very useful for Spark's case. (Discussion on KIP-505 described it.) Thanks for driving the effort! 2019년 8월 14일 (수) 오후 8:49, Mickael Maison 님이 작성: > Hi Guozhang, > > Thanks for taking a look. > > 1. Right, I updated the titles of the code blocks > > 2. Yes that's a g

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Mickael Maison
Hi Guozhang, Thanks for taking a look. 1. Right, I updated the titles of the code blocks 2. Yes that's a good idea. I've updated the KIP Thank you On Wed, Aug 14, 2019 at 11:05 AM Mickael Maison wrote: > > Hi Colin, > > Thanks for raising these 2 valid points. I've updated the KIP accordingly

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-14 Thread Mickael Maison
Hi Colin, Thanks for raising these 2 valid points. I've updated the KIP accordingly. On Tue, Aug 13, 2019 at 9:50 PM Guozhang Wang wrote: > > Hi Mickael, > > Thanks for the KIP! > > Just some minor comments. > > 1. Java class names are stale, e.g. "CommitOffsetsOptions.java" should be > "AlterOf

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-13 Thread Guozhang Wang
Hi Mickael, Thanks for the KIP! Just some minor comments. 1. Java class names are stale, e.g. "CommitOffsetsOptions.java" should be "AlterOffsetsOptions". 2. I'd suggest we change the future structure of "AlterOffsetsResult" to *KafkaFuture>>* This is because we will have a hierarchy of two-l

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-13 Thread Colin McCabe
Hi Mickael, Considering that KIP-496, which adds a way of deleting consumer offsets from AdminClient, looks like it is going to get in, this seems like functionality we should definitely have. For alterConsumerGroupOffsets, is the intention to ignore partitions that are not specified in the ma

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-09 Thread Mickael Maison
Hi Jason, 4. Yes, I've added that in the KIP 5. I've updated the KIP to use `alterConsumerGroupOffsets` so it's consistent with other APIs Thank you for the feedback On Wed, Aug 7, 2019 at 7:08 PM Jason Gustafson wrote: > > Thanks Mickael, +1 from me. > > Just a couple more comments: > > > Yes

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-07 Thread Jason Gustafson
Thanks Mickael, +1 from me. Just a couple more comments: > Yes mapping it to a friendlier error would be nice but I'm not sure if we can do that because UNKNOWN_MEMBER_ID is also returned if the group is dead. I think the logic was chanced recently to return `COORDINATOR_NOT_AVAILABLE` if the gr

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-07 Thread Mickael Maison
Hi Jason, Thanks for the feedback 1. Yes listOffsets() should be able to retrieve earliest, latest and by-timestamp offsets. I agree, it's better to avoid exposing magic values. I've updated the KIP to use an OffsetSpec object as suggested. 2. Yes let's expose the leader epoch in ListOffsetsResu

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-06 Thread Jason Gustafson
Thanks for the KIP. This makes sense to me. Just a couple small comments: 1. Can the listOffsets API be used to get the start and end offsets? In the consumer, we use separate APIs for this: `beginningOffsets` and `endOffsets` to avoid the need for sentinels. An alternative would be to introduce a

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-06 Thread Mickael Maison
Hi Colin, Thank you for taking a look! I agree, being able to set consumer group offsets via the AdminClient would be really useful, hence I created this KIP. With the total absence of binding votes, I guessed I needed to make some changes. Do you mean you preferred the previous naming (commitCon

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-05 Thread Colin McCabe
I think it would be useful to have this in AdminClient. Especially if we implement KIP-496: Administrative API to delete consumer offsets. It would be odd to have a way to delete consumer offsets in AdminClient, but not to create them. What do you think? best, Colin On Sun, Aug 4, 2019, at

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-08-04 Thread Mickael Maison
Hi, In an attempt to unblock this KIP, I've made some adjustments: I've renamed the commitConsumerGroupOffsets() methods to resetConsumerGroupOffsets() to reduce confusion. That should better highlight the differences with the regular commit() operation from the Consumer API. I've also added some

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-03-25 Thread Mickael Maison
Bumping this thread once again Ismael, have I answered your questions? While this has received a few non-binding +1s, no committers have voted yet. If you have concerns or questions, please let me know. Thanks On Mon, Feb 11, 2019 at 11:51 AM Mickael Maison wrote: > > Bumping this thread as it'

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-02-11 Thread Mickael Maison
Bumping this thread as it's been a couple of weeks. On Tue, Jan 22, 2019 at 2:26 PM Mickael Maison wrote: > > Thanks Ismael for the feedback. I think your point has 2 parts: > - Having the reset functionality in the AdminClient: > The fact we have a command line tool illustrate that this operatio

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-22 Thread Mickael Maison
Thanks Ismael for the feedback. I think your point has 2 parts: - Having the reset functionality in the AdminClient: The fact we have a command line tool illustrate that this operation is relatively common. I seems valuable to be able to perform this operation directly via a proper API in addition

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-21 Thread Ismael Juma
The KIP doesn't discuss the option of using KafkaConsumer directly as far as I can tell. We have tried to avoid having the same functionality in multiple clients so it would be good to explain why this is necessary here (not saying it isn't). Ismael On Mon, Jan 21, 2019, 10:29 AM Mickael Maison

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-21 Thread Mickael Maison
Thanks Ryanne for the feedback, all suggestions sounded good, I've updated the KIP accordingly. On Mon, Jan 21, 2019 at 3:43 PM Ryanne Dolan wrote: > > +1 (non-binding) > > But I suggest: > > - drop "get" from getOffset, getTimestamp. > > - add to the motivation section why this is better than co

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-21 Thread Ryanne Dolan
+1 (non-binding) But I suggest: - drop "get" from getOffset, getTimestamp. - add to the motivation section why this is better than constructing a KafkaConsumer and using seek(), commit() etc. - add some rejected alternatives. Ryanne On Mon, Jan 21, 2019, 7:57 AM Dongjin Lee We have +4 non-

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-21 Thread Dongjin Lee
We have +4 non-binding for this vote. Is there any committer who is interested in this issue? Thanks, Dongjin On Mon, Jan 21, 2019 at 10:33 PM Andrew Schofield wrote: > +1 (non-binding). Thanks for the KIP. > > On 21/01/2019, 12:45, "Eno Thereska" wrote: > > +1 (non binding). Thanks. > >

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-21 Thread Andrew Schofield
+1 (non-binding). Thanks for the KIP. On 21/01/2019, 12:45, "Eno Thereska" wrote: +1 (non binding). Thanks. On Mon, Jan 21, 2019 at 12:30 PM Mickael Maison wrote: > Bumping this thread. Considering this KIP is relatively straigh > forward, can we get some votes or

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-21 Thread Eno Thereska
+1 (non binding). Thanks. On Mon, Jan 21, 2019 at 12:30 PM Mickael Maison wrote: > Bumping this thread. Considering this KIP is relatively straigh > forward, can we get some votes or feedback if you think it's not? > Thanks > > On Tue, Jan 8, 2019 at 5:40 PM Edoardo Comar wrote: > > > > +1 (non

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-21 Thread Mickael Maison
Bumping this thread. Considering this KIP is relatively straigh forward, can we get some votes or feedback if you think it's not? Thanks On Tue, Jan 8, 2019 at 5:40 PM Edoardo Comar wrote: > > +1 (non-binding) > Thanks Mickael! > > On Tue, 8 Jan 2019 at 17:39, Patrik Kleindl wrote: > > > +1 (non

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-08 Thread Edoardo Comar
+1 (non-binding) Thanks Mickael! On Tue, 8 Jan 2019 at 17:39, Patrik Kleindl wrote: > +1 (non-binding) > Thanks, sounds very helpful > Best regards > Patrik > > > Am 08.01.2019 um 18:10 schrieb Mickael Maison >: > > > > Hi all, > > > > I'd like to start the vote on KIP-396: > > > https://cwiki.

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

2019-01-08 Thread Patrik Kleindl
+1 (non-binding) Thanks, sounds very helpful Best regards Patrik > Am 08.01.2019 um 18:10 schrieb Mickael Maison : > > Hi all, > > I'd like to start the vote on KIP-396: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97551484 > > Thanks