Re: [DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

2018-04-17 Thread John Roesler
Hi all, I'm closing this discussion and marking KIP-288 as discarded. Please pick up the discussion on the KIP-266 thread. Thanks, -John On Tue, Apr 17, 2018 at 2:07 PM, John Roesler wrote: > Ah, I see that this KIP is a sub-kip of KIP-266. Sorry about that; I've > sent a message to the discus

Re: [DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

2018-04-17 Thread John Roesler
Ah, I see that this KIP is a sub-kip of KIP-266. Sorry about that; I've sent a message to the discussion thread of KIP-266 to work this out. Thanks, -John On Tue, Apr 17, 2018 at 1:29 PM, John Roesler wrote: > Thanks, Richard, I'll check it out. > > On Tue, Apr 17, 2018 at 1:19 PM, Richard Yu

Re: [DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

2018-04-17 Thread John Roesler
Hi Ted, Thanks for the review. During phase #2, the consumer continuously polls until it either gets some responses or spends the whole timeout. If it reaches the timeout, it doesn't mean that the operation "timed out" in an exceptional way, it just means we determined that there are no records a

Re: [DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

2018-04-17 Thread John Roesler
Thanks, Richard, I'll check it out. On Tue, Apr 17, 2018 at 1:19 PM, Richard Yu wrote: > Hi John, > > bq. #1 (wait for metadata) is infinite. > > Some of what you stated in this KIP has already been previously discussed > in a older KIP. (KIP-266) > Just for your reference. > > Thanks, Richard >

Re: [DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

2018-04-17 Thread Ted Yu
bq. This method will continue to return an empty collection if it doesn't complete phase #2 within the timeout. If phase #1 times out, poll will throw a TimeoutException. It seems response from the new poll method depends on which phase it is in prior to the timeout. Wouldn't it be cleaner if Time

Re: [DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

2018-04-17 Thread Richard Yu
Hi John, bq. #1 (wait for metadata) is infinite. Some of what you stated in this KIP has already been previously discussed in a older KIP. (KIP-266) Just for your reference. Thanks, Richard On Tue, Apr 17, 2018 at 11:08 AM, John Roesler wrote: > Hello all, > > I am proposing KIP-288 to improv

[DISCUSS] KIP-288: Consumer poll timeout change and new waitForAssignment method

2018-04-17 Thread John Roesler
Hello all, I am proposing KIP-288 to improve the semantics of Consumer.poll() w.r.t. the timeout parameter and to add a new method for blocking for assignment only. Please find the details here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-288%3A+Consumer.poll%28%29+timeout+semantic+chan