RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

well enough. --CH -Original Message- From: Colin McCabe Sent: Tuesday, January 29, 2019 4:00 PM To: dev@kafka.apache.org Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer On Mon, Jan 28, 2019, at 06:57, ChienHsing Wu wrote

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

y responses below: > > -Original Message- > From: ChienHsing Wu > Sent: Monday, January 21, 2019 12:47 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumption Across Partitions in KafkaConsumer > > Hi all,

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

in KafkaConsumer Hi all, not sure what to do next as weeks have gone by, guys. --CH -Original Message- From: ChienHsing Wu Sent: Monday, January 14, 2019 9:13 AM To: dev@kafka.apache.org Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Hi all, not sure what to do next as weeks have gone by, guys. --CH -Original Message- From: ChienHsing Wu Sent: Monday, January 14, 2019 9:13 AM To: dev@kafka.apache.org Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi, I

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi guys, I am not sure what to do next in this KIP process. Could anyone please help/advise me on what to do next? Thanks, CH -Original Message- From: ChienHsing Wu Sent: Wednesday, January 02

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi ChienHsing Wu, Maybe I'm misunderstanding something, but I'm not sure I see the need for a KIP here. You can just set max.partition.fetch.bytes to a very small value. That will

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

nyone please help/advise me on what to do next? > > Thanks! > > CH > > -Original Message- > From: ChienHsing Wu > Sent: Wednesday, December 12, 2018 1:05 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumptio

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

> > Thanks! > > CH > > -Original Message- > From: ChienHsing Wu > Sent: Wednesday, December 12, 2018 1:05 PM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message > Consumption Across Partitions in KafkaConsumer >

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Good to know that, Thanks! Nonetheless, that introduces additional complexity at the client side for a common expectation to more or less receives records in a fair fashion. CH -Original

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

@kafka.apache.org Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi ChienHsing, We are actually working on buffering the already fetched data for paused topicPartitions, so ideally it should not have any effect on performance. Associated jira

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

e: A, A, A, B, B, B. > > > > Ideally, it should alternate between A and B. I was proposing to move > > to the next one fetch for the next poll call based on the order in the > > completed fetch queue, so the order becomes A, B, A, B, A, B. The > > implementation parses

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

the completed fetch earlier and have a list to manage them. CH -Original Message- From: Mayuresh Gharat Sent: Tuesday, December 11, 2018 6:58 PM To: dev@kafka.apache.org Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

inal Message- > From: Mayuresh Gharat > Sent: Tuesday, December 11, 2018 1:21 PM > To: dev@kafka.apache.org > Subject: Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption > Across Partitions in KafkaConsumer > > Hi ChienHsing, > > Thanks for the KIP. >

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

VVbwp7TLA&s=Ptfb85HFvz0TqKSju21-_uV-U_0_HHNlnNf0kT > tRlgk&e=> > would be very valuable. > > > > Thanks, ChienHsing > > > > -Original Message----- > From: ChienHsing Wu > Sent: Tuesday, December 04, 2018 11:43 AM > To: dev@kafka.apache.org > Su

Re: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

+Fair+Message+Consumption+Across+Partitions+in+KafkaConsumer> > would be very valuable. > > > > Thanks, ChienHsing > > > > -Original Message- > From: ChienHsing Wu > Sent: Tuesday, December 04, 2018 11:43 AM > To: dev@kafka.apache.org > Subject: RE: [EXTERNAL

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

E: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi, Any comments/updates? I am not sure the next steps if no one has any further comments. Thanks, CH -Original Message- From: ChienHsing Wu mailto:chien...@opentext.com>&

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

-Original Message- From: Matt Farmer Sent: Monday, November 19, 2018 9:32 PM To: dev@kafka.apache.org Subject: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi there, Thanks for the KIP. We’ve run into issues with this at Mailchimp so something to

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Farmer Sent: Monday, November 19, 2018 9:32 PM To: dev@kafka.apache.org Subject: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi there, Thanks for the KIP. We’ve run into issues with this at Mailchimp so something to address consuming behavior

Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

> > From: ChienHsing Wu > Sent: Friday, November 09, 2018 1:10 PM > To: dev@kafka.apache.org > Subject: RE: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions > in KafkaConsumer > > Just to check: Will anyone review this? It's been silent for a week... > Thanks,

RE: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Hi, could anyone please review this KIP? Thanks, ChienHsing From: ChienHsing Wu Sent: Friday, November 09, 2018 1:10 PM To: dev@kafka.apache.org Subject: RE: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Just to check: Will anyone review this? It's been s

RE: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Just to check: Will anyone review this? It's been silent for a week... Thanks, ChienHsing From: ChienHsing Wu Sent: Monday, November 05, 2018 4:18 PM To: 'dev@kafka.apache.org' Subject: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Hi I just p

[DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Hi I just put together the KIP page as requested. This email is to start the discussion thread. KIP: KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer Pull