Re: [DISCUSS] KIP-447: Producer scalability for exactly once semantics

[Jason Gustafson]

Yeah, my reasoning is that the group metadata is only relevant to the
subscription API. So it makes sense to only expose it to the rebalance
listener.

One option we could consider is bring back the `initTransactions` overload.
Then usage looks something like this:

consumer.subscribe(topics, new RebalanceListener() {
  void onGroupJoined(GroupMetadata metadata) {
    producer.initTransactions(metadata);
  }
}

That seems pretty clean. What do you think?

-Jason

On Tue, Aug 13, 2019 at 6:07 PM Boyang Chen <reluctanthero...@gmail.com>
wrote:

> Hey Guozhang,
>
> thanks for the suggestion. Could you elaborate more on why defining a
> direct consumer API would be easier? The benefit of reusing consumer
> rebalance listener is to consolidate the entry point of consumer internal
> states. Compared with letting consumer generate a deep-copy of metadata
> every time we call #sendOffsetsToTransactions, using a callback seems
> reducing unnecessary updates towards the metadata. WDYT?
>
> Boyang
>
> On Tue, Aug 13, 2019 at 2:14 PM Guozhang Wang <wangg...@gmail.com> wrote:
>
> > Hi Boyang, Jason,
> >
> > If we are going to expose the generation id / group.instance id etc
> anyways
> > I think its slightly better to just add a new API on KafkaConsumer
> > returning the ConsumerGroupMetadata (option 3) than passing it in on an
> > additional callback of ConsumerRebalanceListener.
> > It feels easier to leverage, than requiring users to pass in the
> listener.
> >
> > Guozhang
> >
> > On Mon, Aug 12, 2019 at 3:41 PM Boyang Chen <reluctanthero...@gmail.com>
> > wrote:
> >
> > > Thanks Jason, the intuition behind defining a separate callback
> function
> > is
> > > that, with KIP-429 we no longer guarantee to call
> OnPartitionsAssigned()
> > or
> > > OnPartitionsRevoked() with each rebalance. Our requirement is to be
> > > up-to-date with group metadata such as generation information, so
> > callback
> > > like onGroupJoined() would make more sense as it should be invoked
> after
> > > every successful rebalance.
> > >
> > > Best,
> > > Boyang
> > >
> > > On Mon, Aug 12, 2019 at 2:02 PM Jason Gustafson <ja...@confluent.io>
> > > wrote:
> > >
> > > > Hey Boyang,
> > > >
> > > > I favor option 4 as well. It's a little more cumbersome than 3 for
> this
> > > use
> > > > case, but it seems like a cleaner separation of concerns. The
> rebalance
> > > > listener is already concerned with events affecting the assignment
> > > > lifecycle and group membership. I think the only thing I'm wondering
> is
> > > > whether it should be a separate callback as you've suggested, or if
> it
> > > > would make sense to overload `onPartitionsAssigned`. If it's
> separate,
> > > > maybe a name like `onGroupJoined` would be clearer?
> > > >
> > > > Thanks,
> > > > Jason
> > > >
> > > >
> > > >
> > > > On Thu, Aug 8, 2019 at 10:59 PM Boyang Chen <
> > reluctanthero...@gmail.com>
> > > > wrote:
> > > >
> > > > > Thank you Jason. We had some offline discussion on properly keeping
> > > group
> > > > > metadata up to date, and here are some of our options brainstormed:
> > > > > 1. Let the caller of `sendOffsetsToTransaction(offset, metadata)`
> > > > maintain
> > > > > the ever-changing group metadata. This could be done on stream
> side,
> > > but
> > > > > for non-stream EOS the sample code will become complicated as the
> > user
> > > > > needs to implement the partition assignor interface to get the
> update
> > > > from
> > > > > `onAssignment`
> > > > >
> > > > > 2. Get a new API on producer like `refreshGroupMetadata(metadata)`.
> > > This
> > > > is
> > > > > similar to option 1 except that now in the partition assignor
> > callback
> > > we
> > > > > could straightly pass in the producer instance, which simplifies
> the
> > > > > non-stream EOS, however this new API seems weird to define on
> > producer.
> > > > >
> > > > > 3. Make an accessing interface to group metadata, or just expose
> the
> > > > group
> > > > > metadata through a consumer API like `consumer.GroupMetadata()`.
> This
> > > is
> > > > > the old way which avoids the users’ effort to implement partition
> > > > assignor
> > > > > directly.
> > > > >
> > > > > 4. Expose the group metadata through rebalance listener, which is a
> > > more
> > > > > well-known and adopted callback interface. We could do sth like
> > > > > `onGroupMetadataUpdated(ConsumerGroupMetadata metadata)`
> > > > >
> > > > > To simplify the code logic, we believe option 3 & 4 are better
> > > solutions,
> > > > > and of which I slightly prefer option 4 as it is the most clean
> > > solution
> > > > > with less intrusion to both consumer and producer APIs.
> > > > >
> > > > > WDYT?
> > > > >
> > > > > Boyang
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Aug 7, 2019 at 9:20 AM Jason Gustafson <ja...@confluent.io
> >
> > > > wrote:
> > > > >
> > > > > > Hi Boyang,
> > > > > >
> > > > > > > We already persist member.id, instance.id and generation.id in
> > the
> > > > > > offset
> > > > > > topic, what extra fields we need to store?
> > > > > >
> > > > > > Yeah, you're right. I was a little confused and thought this
> > > > information
> > > > > > was needed by the transaction coordinator.
> > > > > >
> > > > > > > This should be easily done on the stream side as we have
> > > > > > StreamsPartitionAssignor to reflect metadata changes upon
> > > > > #onAssignment(),
> > > > > > but non-stream user has to code the callback by hand, do you
> think
> > > the
> > > > > > convenience we sacrifice here worth the simplification benefit?
> > > > > >
> > > > > > Either way, you need a reference to the consumer. I was mostly
> just
> > > > > > thinking it would be better to reduce the integration point to
> its
> > > > > minimum.
> > > > > > Have you thought through the implications of needing to keep
> > around a
> > > > > > reference to the consumer in the producer? What if it gets
> closed?
> > It
> > > > > seems
> > > > > > better not to have to think about these cases.
> > > > > >
> > > > > > -Jason
> > > > > >
> > > > > > On Tue, Aug 6, 2019 at 9:53 PM Boyang Chen <
> > > reluctanthero...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Thank you for the suggestions Jason. And a side note for
> > Guozhang,
> > > I
> > > > > > > updated the KIP to reflect the dependency on 447.
> > > > > > >
> > > > > > > On Tue, Aug 6, 2019 at 11:35 AM Jason Gustafson <
> > > ja...@confluent.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Boyang, thanks for the updates. I have a few more
> comments:
> > > > > > > >
> > > > > > > > 1. We are adding some new fields to TxnOffsetCommit to
> support
> > > > > > > group-based
> > > > > > > > fencing. Do we need these fields to be persisted in the
> offsets
> > > > topic
> > > > > > to
> > > > > > > > ensure that the fencing still works after a coordinator
> > failover?
> > > > > > > >
> > > > > > > > We already persist member.id, instance.id and generation.id
> in
> > > the
> > > > > > > offset
> > > > > > > topic, what extra fields we need to store?
> > > > > > >
> > > > > > >
> > > > > > > > 2. Since you are proposing a new `groupMetadata` API, have
> you
> > > > > > considered
> > > > > > > > whether we still need the `initTransactions` overload?
> Another
> > > way
> > > > > > would
> > > > > > > be
> > > > > > > > to pass it through the `sendOffsetsToTransaction` API:
> > > > > > > >
> > > > > > > > void sendOffsetsToTransaction(Map<TopicPartition,
> > > > OffsetAndMetadata>
> > > > > > > > offsets, GroupMetadata groupMetadata) throws
> > > > > > > > ProducerFencedException, IllegalGenerationException;
> > > > > > > >
> > > > > > > > This seems a little more consistent with the current API and
> > > avoids
> > > > > the
> > > > > > > > direct dependence on the Consumer in the producer.
> > > > > > > >
> > > > > > > > Note that although we avoid one dependency to consumer,
> > producer
> > > > > needs
> > > > > > to
> > > > > > > periodically update
> > > > > > > its group metadata, or in this case the caller of
> > > > > > > *sendOffsetsToTransaction(Map<TopicPartition,
> > > > > > > OffsetAndMetadata>*
> > > > > > > *offsets, GroupMetadata groupMetadata) *is responsible for
> > getting
> > > > the
> > > > > > > latest value of group metadata.
> > > > > > > This should be easily done on the stream side as we have
> > > > > > > StreamsPartitionAssignor to reflect metadata changes upon
> > > > > > #onAssignment(),
> > > > > > > but non-stream user has to code the callback by hand, do you
> > think
> > > > the
> > > > > > > convenience we sacrifice here worth the simplification benefit?
> > > > > > >
> > > > > > >
> > > > > > > > 3. Can you clarify the behavior of the clients when the
> brokers
> > > do
> > > > > not
> > > > > > > > support the latest API versions? This is both for the new
> > > > > > TxnOffsetCommit
> > > > > > > > and the OffsetFetch APIs. I guess the high level idea in
> > streams
> > > is
> > > > > to
> > > > > > > > detect broker support before instantiating the producer and
> > > > > consumer. I
> > > > > > > > think that's reasonable, but we might need some approach for
> > > > > > non-streams
> > > > > > > > use cases. One option I was considering is enforcing the
> latest
> > > > > version
> > > > > > > > through the new `sendOffsetsToTransaction` API. Basically
> when
> > > you
> > > > > use
> > > > > > > the
> > > > > > > > new API, we require support for the latest TxnOffsetCommit
> > > version.
> > > > > > This
> > > > > > > > puts some burden on users, but it avoids breaking correctness
> > > > > > assumptions
> > > > > > > > when the new APIs are in use. What do you think?
> > > > > > > >
> > > > > > > Yes, I think we haven't covered this case, so the plan is to
> > crash
> > > > the
> > > > > > > non-stream application when the job is using new sendOffsets
> API.
> > > > > > >
> > > > > > > >
> > > > > > > > -Jason
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Aug 5, 2019 at 6:06 PM Boyang Chen <
> > > > > reluctanthero...@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Yep, Guozhang I think that would be best as passing in an
> > > entire
> > > > > > > consumer
> > > > > > > > > instance is indeed cumbersome.
> > > > > > > > >
> > > > > > > > > Just saw you updated KIP-429, I will follow-up to change
> 447
> > as
> > > > > well.
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Boyang
> > > > > > > > >
> > > > > > > > > On Mon, Aug 5, 2019 at 3:18 PM Guozhang Wang <
> > > wangg...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > okay I think I understand your concerns about
> > > > > ConsumerGroupMetadata
> > > > > > > > now:
> > > > > > > > > if
> > > > > > > > > > we still want to only call initTxns once, then we should
> > > allow
> > > > > the
> > > > > > > > > whatever
> > > > > > > > > > passed-in parameter to reflect the latest value of
> > generation
> > > > id
> > > > > > > > whenever
> > > > > > > > > > sending the offset fetch request.
> > > > > > > > > >
> > > > > > > > > > Whereas the current ConsumerGroupMetadata is a static
> > object.
> > > > > > > > > >
> > > > > > > > > > Maybe we can consider having an extended class of
> > > > > > > ConsumerGroupMetadata
> > > > > > > > > > whose values are updated from the consumer's rebalance
> > > > callback?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Guozhang
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Mon, Aug 5, 2019 at 9:26 AM Boyang Chen <
> > > > > > > reluctanthero...@gmail.com
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Thank you Guozhang for the reply! I'm curious whether
> > > KIP-429
> > > > > has
> > > > > > > > > > reflected
> > > > > > > > > > > the latest change on ConsumerGroupMetadata? Also
> > regarding
> > > > > > question
> > > > > > > > > one,
> > > > > > > > > > > the group metadata needs to be accessed via callback,
> > does
> > > > that
> > > > > > > mean
> > > > > > > > we
> > > > > > > > > > > need a separate producer API such like
> > > > > > > > > > > "producer.refreshMetadata(groupMetadata)" to be able to
> > > > access
> > > > > it
> > > > > > > > > instead
> > > > > > > > > > > of passing in the consumer instance?
> > > > > > > > > > >
> > > > > > > > > > > Boyang
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Aug 2, 2019 at 4:36 PM Guozhang Wang <
> > > > > wangg...@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Thanks Boyang,
> > > > > > > > > > > >
> > > > > > > > > > > > I've made another pass on KIP-447 as well as
> > > > > > > > > > > > https://github.com/apache/kafka/pull/7078, and have
> > some
> > > > > minor
> > > > > > > > > > comments
> > > > > > > > > > > > about the proposed API:
> > > > > > > > > > > >
> > > > > > > > > > > > 1. it seems instead of needing the whole
> KafkaConsumer
> > > > > object,
> > > > > > > > you'd
> > > > > > > > > > only
> > > > > > > > > > > > need the "ConsumerGroupMetadata", in that case can we
> > > just
> > > > > pass
> > > > > > > in
> > > > > > > > > that
> > > > > > > > > > > > object into the initTxns call?
> > > > > > > > > > > >
> > > > > > > > > > > > 2. the current trunk already has a public class named
> > > > > > > > > > > > (ConsumerGroupMetadata)
> > > > > > > > > > > > under o.a.k.clients.consumer created by KIP-429. If
> we
> > > want
> > > > > to
> > > > > > > just
> > > > > > > > > use
> > > > > > > > > > > > that then maybe it makes less sense to declare a base
> > > > > > > GroupMetadata
> > > > > > > > > as
> > > > > > > > > > we
> > > > > > > > > > > > are already leaking such information on the assignor
> > > > anyways.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Guozhang
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Jul 30, 2019 at 1:55 PM Boyang Chen <
> > > > > > > > > > reluctanthero...@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Thank you Guozhang for the reply. We will consider
> > the
> > > > > > > interface
> > > > > > > > > > change
> > > > > > > > > > > > > from 429 as a backup plan for 447.
> > > > > > > > > > > > >
> > > > > > > > > > > > > And bumping this thread for more discussion.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, Jul 22, 2019 at 6:28 PM Guozhang Wang <
> > > > > > > > wangg...@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > On Sat, Jul 20, 2019 at 9:50 AM Boyang Chen <
> > > > > > > > > > > > reluctanthero...@gmail.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thank you Guozhang for the suggestion! I would
> > > > normally
> > > > > > > > prefer
> > > > > > > > > > > > naming a
> > > > > > > > > > > > > > > flag corresponding to its functionality. Seems
> to
> > > me
> > > > > > > > > > > > `isolation_level`
> > > > > > > > > > > > > > > makes us another hop on information track.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Fair enough, let's use a separate flag name
> then
> > :)
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > As for the generation.id exposure, I'm fine
> > > > leveraging
> > > > > > the
> > > > > > > > new
> > > > > > > > > > API
> > > > > > > > > > > > > from
> > > > > > > > > > > > > > > 429, but however is that design finalized yet,
> > and
> > > > > > whether
> > > > > > > > the
> > > > > > > > > > API
> > > > > > > > > > > > will
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > added on the generic Consumer<K, V> interface?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The current PartitionAssignor is inside
> > `internals`
> > > > > > package
> > > > > > > > and
> > > > > > > > > > in
> > > > > > > > > > > > > > KIP-429
> > > > > > > > > > > > > > we are going to create a new interface out of
> > > > `internals`
> > > > > > to
> > > > > > > > > really
> > > > > > > > > > > > make
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > public APIs, and as part of that we are
> refactoring
> > > > some
> > > > > of
> > > > > > > its
> > > > > > > > > > > method
> > > > > > > > > > > > > > signatures. I just feel some of the newly
> > introduced
> > > > > > classes
> > > > > > > > can
> > > > > > > > > be
> > > > > > > > > > > > > reused
> > > > > > > > > > > > > > in your KIP as well, i.e. just for code
> > succinctness,
> > > > but
> > > > > > no
> > > > > > > > > > > semantical
> > > > > > > > > > > > > > indications.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Boyang
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Fri, Jul 19, 2019 at 3:57 PM Guozhang Wang <
> > > > > > > > > > wangg...@gmail.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Boyang, thanks for the updated proposal!
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 3.a. As Jason mentioned, with EOS enabled we
> > > still
> > > > > need
> > > > > > > to
> > > > > > > > > > > augment
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > offset fetch request with a boolean to
> indicate
> > > > "give
> > > > > > me
> > > > > > > an
> > > > > > > > > > > > retriable
> > > > > > > > > > > > > > > error
> > > > > > > > > > > > > > > > code if there's pending offset, rather than
> > > sending
> > > > > me
> > > > > > > the
> > > > > > > > > > > > committed
> > > > > > > > > > > > > > > offset
> > > > > > > > > > > > > > > > immediately". Personally I still feel it is
> > okay
> > > to
> > > > > > > > > piggy-back
> > > > > > > > > > on
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > > ISOLATION_LEVEL boolean, but I'm also fine
> with
> > > > > another
> > > > > > > > > > > > > > > `await_transaction`
> > > > > > > > > > > > > > > > boolean if you feel strongly about it.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 10. About the exposure of generation id,
> there
> > > may
> > > > be
> > > > > > > some
> > > > > > > > > > > > > refactoring
> > > > > > > > > > > > > > > work
> > > > > > > > > > > > > > > > coming from KIP-429 that can benefit KIP-447
> as
> > > > well
> > > > > > > since
> > > > > > > > we
> > > > > > > > > > are
> > > > > > > > > > > > > > > wrapping
> > > > > > > > > > > > > > > > the consumer subscription / assignment data
> in
> > > new
> > > > > > > classes.
> > > > > > > > > > Note
> > > > > > > > > > > > that
> > > > > > > > > > > > > > > > current proposal does not `generationId`
> since
> > > with
> > > > > the
> > > > > > > > > > > cooperative
> > > > > > > > > > > > > > > sticky
> > > > > > > > > > > > > > > > assignor we think it is not necessary for
> > > > > correctness,
> > > > > > > but
> > > > > > > > > also
> > > > > > > > > > > if
> > > > > > > > > > > > we
> > > > > > > > > > > > > > > agree
> > > > > > > > > > > > > > > > it is okay to expose it we can potentially
> > > include
> > > > it
> > > > > > in
> > > > > > > > > > > > > > > > `ConsumerAssignmentData` as well.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Thu, Jul 18, 2019 at 3:55 PM Boyang Chen <
> > > > > > > > > > > > > > reluctanthero...@gmail.com>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Thank you Jason for the ideas.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Mon, Jul 15, 2019 at 5:28 PM Jason
> > > Gustafson <
> > > > > > > > > > > > > ja...@confluent.io>
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Hi Boyang,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Thanks for the updates. A few comments
> > below:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 1. The KIP mentions that `
> > > > transaction.timeout.ms
> > > > > `
> > > > > > > > should
> > > > > > > > > > be
> > > > > > > > > > > > > > reduced
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > 10s.
> > > > > > > > > > > > > > > > > > I think this makes sense for Kafka
> Streams
> > > > which
> > > > > is
> > > > > > > > tied
> > > > > > > > > to
> > > > > > > > > > > the
> > > > > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > > > > group semantics and uses a default 10s
> > > session
> > > > > > > timeout.
> > > > > > > > > > > > However,
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > seems a
> > > > > > > > > > > > > > > > > > bit dangerous to make this change for the
> > > > > producer
> > > > > > > > > > generally.
> > > > > > > > > > > > > Could
> > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > just
> > > > > > > > > > > > > > > > > > change it for streams?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > That sounds good to me.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 2. The new `initTransactions` API takes a
> > > > > > `Consumer`
> > > > > > > > > > > instance.
> > > > > > > > > > > > I
> > > > > > > > > > > > > > > think
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > idea is to basically put in a backdoor to
> > > give
> > > > > the
> > > > > > > > > producer
> > > > > > > > > > > > > access
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > group generationId. It's not clear to me
> > how
> > > > this
> > > > > > > would
> > > > > > > > > > work
> > > > > > > > > > > > > given
> > > > > > > > > > > > > > > > > package
> > > > > > > > > > > > > > > > > > restrictions. I wonder if it would be
> > better
> > > to
> > > > > > just
> > > > > > > > > expose
> > > > > > > > > > > the
> > > > > > > > > > > > > > state
> > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > need from the consumer. I know we have
> been
> > > > > > reluctant
> > > > > > > > to
> > > > > > > > > do
> > > > > > > > > > > > this
> > > > > > > > > > > > > so
> > > > > > > > > > > > > > > far
> > > > > > > > > > > > > > > > > > because we treat the generationId as an
> > > > > > > implementation
> > > > > > > > > > > detail.
> > > > > > > > > > > > > > > > However, I
> > > > > > > > > > > > > > > > > > think we might just bite the bullet and
> > > expose
> > > > it
> > > > > > > > rather
> > > > > > > > > > than
> > > > > > > > > > > > > > coming
> > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > with a messy hack. Concepts such as
> > memberIds
> > > > > have
> > > > > > > > > already
> > > > > > > > > > > been
> > > > > > > > > > > > > > > exposed
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > the AdminClient, so maybe it is not too
> > bad.
> > > > > > > > > Alternatively,
> > > > > > > > > > > we
> > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > use
> > > > > > > > > > > > > > > > > an
> > > > > > > > > > > > > > > > > > opaque type. For example:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > // public
> > > > > > > > > > > > > > > > > > interface GroupMetadata {}
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > // private
> > > > > > > > > > > > > > > > > > interface ConsumerGroupMetadata {
> > > > > > > > > > > > > > > > > >   final int generationId;
> > > > > > > > > > > > > > > > > >   final String memberId;
> > > > > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > // Consumer API
> > > > > > > > > > > > > > > > > > public GroupMetadata groupMetadata();
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I am probably leaning toward just
> exposing
> > > the
> > > > > > state
> > > > > > > we
> > > > > > > > > > need.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Yes, also to mention that Kafka Streams
> use
> > > > > generic
> > > > > > > > > > Cosnumer
> > > > > > > > > > > > API
> > > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > > > doesn't have rich
> > > > > > > > > > > > > > > > > states like a full `KafkaConsumer`. The
> hack
> > > will
> > > > > not
> > > > > > > > work
> > > > > > > > > as
> > > > > > > > > > > > > > expected.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Instead, just exposing the consumer
> > > > generation.id
> > > > > > > seems
> > > > > > > > a
> > > > > > > > > > way
> > > > > > > > > > > > > easier
> > > > > > > > > > > > > > > > work.
> > > > > > > > > > > > > > > > > We could consolidate
> > > > > > > > > > > > > > > > > the API and make it
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > 3. Given that we are already providing a
> way
> > to
> > > > > > > propagate
> > > > > > > > > > group
> > > > > > > > > > > > > state
> > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > the consumer to the producer, I wonder if
> > we
> > > > may
> > > > > as
> > > > > > > > well
> > > > > > > > > > > > include
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > memberId and groupInstanceId. This would
> > make
> > > > the
> > > > > > > > > > validation
> > > > > > > > > > > we
> > > > > > > > > > > > > do
> > > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > TxnOffsetCommit consistent with
> > OffsetCommit.
> > > > If
> > > > > > for
> > > > > > > no
> > > > > > > > > > other
> > > > > > > > > > > > > > > benefit,
> > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > least this may help with debugging.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Yes, we could put them into the
> GroupMetadata
> > > > > struct.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 4. I like the addition of isolation_level
> > to
> > > > the
> > > > > > > offset
> > > > > > > > > > > fetch.
> > > > > > > > > > > > At
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > same
> > > > > > > > > > > > > > > > > > time, its behavior is a bit inconsistent
> > with
> > > > how
> > > > > > it
> > > > > > > is
> > > > > > > > > > used
> > > > > > > > > > > in
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > consumer generally. There is no reason
> for
> > > the
> > > > > > group
> > > > > > > > > > > > coordinator
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > ever
> > > > > > > > > > > > > > > > > > expose aborted data, so this is mostly
> > about
> > > > > > awaiting
> > > > > > > > > > pending
> > > > > > > > > > > > > > offset
> > > > > > > > > > > > > > > > > > commits, not reading uncommitted data.
> > > Perhaps
> > > > > > > instead
> > > > > > > > of
> > > > > > > > > > > > calling
> > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > "isolation level," it should be more like
> > > > > > > > > > > > > > "await_pending_transaction"
> > > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > > something like that?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Also, just to be clear, the consumer
> would
> > > > treat
> > > > > > this
> > > > > > > > as
> > > > > > > > > an
> > > > > > > > > > > > > > optional
> > > > > > > > > > > > > > > > > field,
> > > > > > > > > > > > > > > > > > right? So if the broker does not support
> > the
> > > > > latest
> > > > > > > > > > > OffsetFetch
> > > > > > > > > > > > > > API,
> > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > would silently revert to reading the old
> > > data.
> > > > > > > > Basically
> > > > > > > > > it
> > > > > > > > > > > > would
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > the streams version probing logic to
> ensure
> > > > that
> > > > > > the
> > > > > > > > > > > > expectation
> > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > API fits with the usage of `
> > transctional.id
> > > `.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Sounds like a better naming to me, while
> I
> > > > think
> > > > > it
> > > > > > > > could
> > > > > > > > > > be
> > > > > > > > > > > > > > > shortened
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > `await_transaction`.
> > > > > > > > > > > > > > > > > I think the field should be optional, too.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > Jason
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > On Mon, Jul 8, 2019 at 3:19 PM Boyang
> Chen
> > <
> > > > > > > > > > > > > > > reluctanthero...@gmail.com
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Hey Guozhang,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > I will correct my statement from last
> > > email.
> > > > I
> > > > > > > don't
> > > > > > > > > > think
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > read_committed (3.a) is necessary to be
> > > added
> > > > > to
> > > > > > > the
> > > > > > > > > > > > > OffsetFetch
> > > > > > > > > > > > > > > > > request,
> > > > > > > > > > > > > > > > > > > as if we are using EOS application, the
> > > > > > underlying
> > > > > > > > > > > consumers
> > > > > > > > > > > > > > within
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > group should always back off when there
> > is
> > > > > > pending
> > > > > > > > > > offsets.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Let me know if you think this is
> correct.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > On Tue, Jul 2, 2019 at 3:21 PM Boyang
> > Chen
> > > <
> > > > > > > > > > > > > > > > reluctanthero...@gmail.com
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Thank you Guozhang for the questions,
> > > > inline
> > > > > > > > answers
> > > > > > > > > > are
> > > > > > > > > > > > > below.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > On Tue, Jul 2, 2019 at 3:14 PM Boyang
> > > Chen
> > > > <
> > > > > > > > > > > > > > > > > reluctanthero...@gmail.com
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >> Hey all,
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >> I have done a fundamental polish of
> > > > KIP-447
> > > > > > > > > > > > > > > > > > > >> <
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > >> written a design doc
> > > > > > > > > > > > > > > > > > > >> <
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1LhzHGeX7_Lay4xvrEXxfciuDWATjpUXQhrEIkph9qRE/edit#
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > depicting
> > > > > > > > > > > > > > > > > > > >> internal changes. We stripped off
> many
> > > > > > > > > implementation
> > > > > > > > > > > > > details
> > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > KIP,
> > > > > > > > > > > > > > > > > > > >> and simplified the public changes
> by a
> > > > lot.
> > > > > > For
> > > > > > > > > > > reviewers,
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > highly
> > > > > > > > > > > > > > > > > > > >> recommended to fully understand EOS
> > > design
> > > > > in
> > > > > > > > KIP-98
> > > > > > > > > > and
> > > > > > > > > > > > > read
> > > > > > > > > > > > > > > its
> > > > > > > > > > > > > > > > > > > >> corresponding design doc
> > > > > > > > > > > > > > > > > > > >> <
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/11Jqy_GjUGtdXJK94XGsEIK7CP1SnQGdp2eF0wSw9ra8/edit
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > if
> > > > > > > > > > > > > > > > > > > >> you haven't done so already.
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >> Let me know if you found anything
> > > > confusing
> > > > > > > around
> > > > > > > > > the
> > > > > > > > > > > KIP
> > > > > > > > > > > > > or
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > design.
> > > > > > > > > > > > > > > > > > > >> Would be happy to discuss in depth.
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >> Best,
> > > > > > > > > > > > > > > > > > > >> Boyang
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >> On Wed, Jun 26, 2019 at 11:00 AM
> > > Guozhang
> > > > > > Wang <
> > > > > > > > > > > > > > > > wangg...@gmail.com>
> > > > > > > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >>> 2. The reason we did not expose
> > > > > > generation.id
> > > > > > > > from
> > > > > > > > > > > > > > > KafkaConsumer
> > > > > > > > > > > > > > > > > > > public
> > > > > > > > > > > > > > > > > > > >>> APIs directly is to abstract this
> > > notion
> > > > > from
> > > > > > > > users
> > > > > > > > > > > > (since
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > an
> > > > > > > > > > > > > > > > > > > >>> implementation detail of the
> > rebalance
> > > > > > protocol
> > > > > > > > > > itself,
> > > > > > > > > > > > > e.g.
> > > > > > > > > > > > > > if
> > > > > > > > > > > > > > > > > user
> > > > > > > > > > > > > > > > > > > >>> calls
> > > > > > > > > > > > > > > > > > > >>> consumer.assign() they do not need
> to
> > > > > invoke
> > > > > > > > > > > > > > > ConsumerCoordinator
> > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > no
> > > > > > > > > > > > > > > > > > > >>> need to be aware of generation.id
> at
> > > > all).
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> On the other hand, with the current
> > > > > proposal
> > > > > > > the
> > > > > > > > > > > > > > > txn.coordiantor
> > > > > > > > > > > > > > > > > did
> > > > > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > >>> know about the latest generation
> from
> > > the
> > > > > > > > > > > source-of-truth
> > > > > > > > > > > > > > > > > > > >>> group.coordinator; instead, it will
> > > only
> > > > > bump
> > > > > > > up
> > > > > > > > > the
> > > > > > > > > > > > > > generation
> > > > > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> producer's InitProducerIdRequest
> > only.
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> The key here is that
> > GroupCoordinator,
> > > > when
> > > > > > > > > handling
> > > > > > > > > > > > > > > > > > > >>> `InitProducerIdRequest
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >> In the new design, we just pass the
> > > entire
> > > > > > > > consumer
> > > > > > > > > > > > instance
> > > > > > > > > > > > > > > into
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > producer through
> > > > > > > > > > > > > > > > > > > > #initTransaction, so no public API
> will
> > > be
> > > > > > > created.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >> 3. I agree that if we rely on the
> > group
> > > > > > > > coordinator
> > > > > > > > > to
> > > > > > > > > > > > block
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > returning
> > > > > > > > > > > > > > > > > > > >>> offset-fetch-response if
> > read-committed
> > > > is
> > > > > > > > enabled,
> > > > > > > > > > > then
> > > > > > > > > > > > we
> > > > > > > > > > > > > > do
> > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > need
> > > > > > > > > > > > > > > > > > > >>> to
> > > > > > > > > > > > > > > > > > > >>> store partition assignment on txn
> > > > > coordinator
> > > > > > > and
> > > > > > > > > > > > therefore
> > > > > > > > > > > > > > > it's
> > > > > > > > > > > > > > > > > > better
> > > > > > > > > > > > > > > > > > > >>> to
> > > > > > > > > > > > > > > > > > > >>> still decouple them. For that case
> we
> > > > still
> > > > > > > need
> > > > > > > > to
> > > > > > > > > > > > update
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > KIP
> > > > > > > > > > > > > > > > > > wiki
> > > > > > > > > > > > > > > > > > > >>> page that includes:
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> 3.a. Augment OffsetFetchRequest
> with
> > > the
> > > > > > > > > > > ISOLATION_LEVEL
> > > > > > > > > > > > as
> > > > > > > > > > > > > > > well.
> > > > > > > > > > > > > > > > > > > >>> 3.b. Add new error code in
> > > > > > OffsetFetchResponse
> > > > > > > to
> > > > > > > > > let
> > > > > > > > > > > > > client
> > > > > > > > > > > > > > > > > backoff
> > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > >>> retry if there are pending txns
> > > including
> > > > > the
> > > > > > > > > > > interested
> > > > > > > > > > > > > > > > > partitions.
> > > > > > > > > > > > > > > > > > > >>> 3.c. Also in the worst case we
> would
> > > let
> > > > > the
> > > > > > > > client
> > > > > > > > > > be
> > > > > > > > > > > > > > blocked
> > > > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> txn.timeout period, and for that
> > > > rationale
> > > > > we
> > > > > > > may
> > > > > > > > > > need
> > > > > > > > > > > to
> > > > > > > > > > > > > > > > consider
> > > > > > > > > > > > > > > > > > > >>> reducing
> > > > > > > > > > > > > > > > > > > >>> our default txn.timeout value as
> > well.
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> Addressed 3.b and 3.c, will do 3.a.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >> 4. According to Colin it seems we do
> > not
> > > > > need
> > > > > > to
> > > > > > > > > > create
> > > > > > > > > > > > > > another
> > > > > > > > > > > > > > > > KIP
> > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > >>> can just complete it as part of
> > > KIP-117 /
> > > > > > > > > KAFKA-5214;
> > > > > > > > > > > and
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > need
> > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > do
> > > > > > > > > > > > > > > > > > > >>> some cleanup to have
> BrokerApiVersion
> > > > > exposed
> > > > > > > > from
> > > > > > > > > > > > > > AdminClient
> > > > > > > > > > > > > > > > > > (@Colin
> > > > > > > > > > > > > > > > > > > >>> please let use know if you have any
> > > > > concerns
> > > > > > > > > exposing
> > > > > > > > > > > > it).
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >> I think we no longer need to rely on
> > api
> > > > > > version
> > > > > > > > for
> > > > > > > > > > > > > > > > initialization,
> > > > > > > > > > > > > > > > > > > > since we will be using the
> upgrade.from
> > > > > config
> > > > > > > > > anyway.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >>> Guozhang
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> On Tue, Jun 25, 2019 at 6:43 PM
> Jason
> > > > > > > Gustafson <
> > > > > > > > > > > > > > > > > ja...@confluent.io>
> > > > > > > > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> > For reference, we have
> > > > > > > BrokerApiVersionCommand
> > > > > > > > > > > already
> > > > > > > > > > > > > as a
> > > > > > > > > > > > > > > > > public
> > > > > > > > > > > > > > > > > > > >>> > interface. We have a bit of tech
> > debt
> > > > at
> > > > > > the
> > > > > > > > > moment
> > > > > > > > > > > > > because
> > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > uses a
> > > > > > > > > > > > > > > > > > > >>> > custom AdminClient. It would be
> > nice
> > > to
> > > > > > clean
> > > > > > > > > that
> > > > > > > > > > > up.
> > > > > > > > > > > > In
> > > > > > > > > > > > > > > > > general,
> > > > > > > > > > > > > > > > > > I
> > > > > > > > > > > > > > > > > > > >>> think
> > > > > > > > > > > > > > > > > > > >>> > it is reasonable to expose from
> > > > > > AdminClient.
> > > > > > > It
> > > > > > > > > can
> > > > > > > > > > > be
> > > > > > > > > > > > > used
> > > > > > > > > > > > > > > by
> > > > > > > > > > > > > > > > > > > >>> management
> > > > > > > > > > > > > > > > > > > >>> > tools to inspect running Kafka
> > > versions
> > > > > for
> > > > > > > > > > example.
> > > > > > > > > > > > > > > > > > > >>> >
> > > > > > > > > > > > > > > > > > > >>> > -Jason
> > > > > > > > > > > > > > > > > > > >>> >
> > > > > > > > > > > > > > > > > > > >>> > On Tue, Jun 25, 2019 at 4:37 PM
> > > Boyang
> > > > > > Chen <
> > > > > > > > > > > > > > > > > > > >>> reluctanthero...@gmail.com>
> > > > > > > > > > > > > > > > > > > >>> > wrote:
> > > > > > > > > > > > > > > > > > > >>> >
> > > > > > > > > > > > > > > > > > > >>> > > Thank you for the context
> Colin.
> > > The
> > > > > > > groupId
> > > > > > > > > was
> > > > > > > > > > > > > indeed a
> > > > > > > > > > > > > > > > > > > copy-paste
> > > > > > > > > > > > > > > > > > > >>> > error.
> > > > > > > > > > > > > > > > > > > >>> > > Our use case here for 447 is
> > > (Quoted
> > > > > from
> > > > > > > > > > > Guozhang):
> > > > > > > > > > > > > > > > > > > >>> > > '''
> > > > > > > > > > > > > > > > > > > >>> > > I think if we can do something
> > else
> > > > to
> > > > > > > > > > > > > > > > > > > >>> > > avoid this config though, for
> > > example
> > > > > we
> > > > > > > can
> > > > > > > > > use
> > > > > > > > > > > the
> > > > > > > > > > > > > > > embedded
> > > > > > > > > > > > > > > > > > > >>> AdminClient
> > > > > > > > > > > > > > > > > > > >>> > > to send the APIVersion request
> > upon
> > > > > > > starting
> > > > > > > > > up,
> > > > > > > > > > > and
> > > > > > > > > > > > > > based
> > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> > returned
> > > > > > > > > > > > > > > > > > > >>> > > value decides whether to go to
> > the
> > > > old
> > > > > > code
> > > > > > > > > path
> > > > > > > > > > or
> > > > > > > > > > > > the
> > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > > > > > behavior.
> > > > > > > > > > > > > > > > > > > >>> > > '''
> > > > > > > > > > > > > > > > > > > >>> > > The benefit we get is to avoid
> > > > adding a
> > > > > > new
> > > > > > > > > > > > > configuration
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > make a
> > > > > > > > > > > > > > > > > > > >>> > > decision simply base on broker
> > > > version.
> > > > > > If
> > > > > > > > you
> > > > > > > > > > have
> > > > > > > > > > > > > > > concerns
> > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > >>> > exposing
> > > > > > > > > > > > > > > > > > > >>> > > ApiVersion for client, we could
> > > > > > > > > > > > > > > > > > > >>> > > try to think of alternative
> > > solutions
> > > > > > too.
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> > > Boyang
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> > > On Tue, Jun 25, 2019 at 4:20 PM
> > > Colin
> > > > > > > McCabe
> > > > > > > > <
> > > > > > > > > > > > > > > > > cmcc...@apache.org
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> > > > kafka.api.ApiVersion is an
> > > internal
> > > > > > > class,
> > > > > > > > > not
> > > > > > > > > > > > > suitable
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > exposing
> > > > > > > > > > > > > > > > > > > >>> > > > through AdminClient.  That
> > class
> > > is
> > > > > not
> > > > > > > > even
> > > > > > > > > > > > > accessible
> > > > > > > > > > > > > > > > > without
> > > > > > > > > > > > > > > > > > > >>> having
> > > > > > > > > > > > > > > > > > > >>> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > broker jars on your
> CLASSPATH.
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > Another question is, what is
> > the
> > > > > > groupId
> > > > > > > > > > > parameter
> > > > > > > > > > > > > > doing
> > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> call?
> > > > > > > > > > > > > > > > > > > >>> > > The
> > > > > > > > > > > > > > > > > > > >>> > > > API versions are the same no
> > > matter
> > > > > > what
> > > > > > > > > > consumer
> > > > > > > > > > > > > group
> > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > use,
> > > > > > > > > > > > > > > > > > > >>> right?
> > > > > > > > > > > > > > > > > > > >>> > > > Perhaps this was a copy and
> > paste
> > > > > > error?
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > This is not the first time we
> > > have
> > > > > > > > discussed
> > > > > > > > > > > > having a
> > > > > > > > > > > > > > > > method
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > >>> > > > AdminClient to retrieve API
> > > version
> > > > > > > > > > information.
> > > > > > > > > > > > In
> > > > > > > > > > > > > > > fact,
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> original
> > > > > > > > > > > > > > > > > > > >>> > > KIP
> > > > > > > > > > > > > > > > > > > >>> > > > which created
> KafkaAdminClient
> > > > > > specified
> > > > > > > an
> > > > > > > > > API
> > > > > > > > > > > for
> > > > > > > > > > > > > > > > fetching
> > > > > > > > > > > > > > > > > > > >>> version
> > > > > > > > > > > > > > > > > > > >>> > > > information.  It was called
> > > > > apiVersions
> > > > > > > and
> > > > > > > > > it
> > > > > > > > > > is
> > > > > > > > > > > > > still
> > > > > > > > > > > > > > > > there
> > > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > >>> the
> > > > > > > > > > > > > > > > > > > >>> > > wiki.
> > > > > > > > > > > > > > > > > > > >>> > > > See
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> >
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+admin+operations
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > However, this API wasn't
> ready
> > in
> > > > > time
> > > > > > > for
> > > > > > > > > > 0.11.0
> > > > > > > > > > > > so
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > shipped
> > > > > > > > > > > > > > > > > > > >>> without
> > > > > > > > > > > > > > > > > > > >>> > > > it.  There was a JIRA to
> > > implement
> > > > it
> > > > > > for
> > > > > > > > > later
> > > > > > > > > > > > > > versions,
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-5214
> > > > > > > > > > > ,
> > > > > > > > > > > > > as
> > > > > > > > > > > > > > > well
> > > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > >>> PR,
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > https://github.com/apache/kafka/pull/3012
> > > > > > > > .
> > > > > > > > > > > > > However,
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > started
> > > > > > > > > > > > > > > > > > > >>> to
> > > > > > > > > > > > > > > > > > > >>> > > > rethink whether this
> > AdminClient
> > > > > > function
> > > > > > > > was
> > > > > > > > > > > even
> > > > > > > > > > > > > > > > necessary.
> > > > > > > > > > > > > > > > > > > >>> Most of
> > > > > > > > > > > > > > > > > > > >>> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > use-cases we could think of
> > > seemed
> > > > > like
> > > > > > > > > > horrible
> > > > > > > > > > > > > hacks.
> > > > > > > > > > > > > > > So
> > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > has
> > > > > > > > > > > > > > > > > > > >>> > never
> > > > > > > > > > > > > > > > > > > >>> > > > really been implemented
> (yet?).
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > best,
> > > > > > > > > > > > > > > > > > > >>> > > > Colin
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > On Tue, Jun 25, 2019, at
> 15:46,
> > > > > Boyang
> > > > > > > Chen
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > Actually, after a second
> > > > thought, I
> > > > > > > think
> > > > > > > > > it
> > > > > > > > > > > > > actually
> > > > > > > > > > > > > > > > makes
> > > > > > > > > > > > > > > > > > > >>> sense to
> > > > > > > > > > > > > > > > > > > >>> > > > > support auto upgrade
> through
> > > > admin
> > > > > > > client
> > > > > > > > > to
> > > > > > > > > > > help
> > > > > > > > > > > > > use
> > > > > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > api
> > > > > > > > > > > > > > > > > > > >>> version
> > > > > > > > > > > > > > > > > > > >>> > > > > from
> > > > > > > > > > > > > > > > > > > >>> > > > > broker.
> > > > > > > > > > > > > > > > > > > >>> > > > > A draft KIP is here:
> > > > > > > > > > > > > > > > > > > >>> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> >
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-483%3A++Add+Broker+Version+API+in+Admin+Client
> > > > > > > > > > > > > > > > > > > >>> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > Boyang
> > > > > > > > > > > > > > > > > > > >>> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > On Tue, Jun 25, 2019 at
> 2:57
> > PM
> > > > > > Boyang
> > > > > > > > > Chen <
> > > > > > > > > > > > > > > > > > > >>> > > reluctanthero...@gmail.com>
> > > > > > > > > > > > > > > > > > > >>> > > > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > > Thank you Guozhang, some
> of
> > > my
> > > > > > > > > > understandings
> > > > > > > > > > > > are
> > > > > > > > > > > > > > > > inline
> > > > > > > > > > > > > > > > > > > below.
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > > On Tue, Jun 25, 2019 at
> > 11:05
> > > > AM
> > > > > > > Jason
> > > > > > > > > > > > Gustafson
> > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > >>> > ja...@confluent.io
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > I think co-locating
> does
> > > > have
> > > > > > some
> > > > > > > > > > merits
> > > > > > > > > > > > > here,
> > > > > > > > > > > > > > > i.e.
> > > > > > > > > > > > > > > > > > > >>> letting the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > ConsumerCoordinator
> > which
> > > > has
> > > > > > the
> > > > > > > > > > > > > > source-of-truth
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > >>> assignment
> > > > > > > > > > > > > > > > > > > >>> > to
> > > > > > > > > > > > > > > > > > > >>> > > > act
> > > > > > > > > > > > > > > > > > > >>> > > > > >> as
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > the TxnCoordinator as
> > > well;
> > > > > but
> > > > > > I
> > > > > > > > > agree
> > > > > > > > > > > > > there's
> > > > > > > > > > > > > > > also
> > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > >>> cons
> > > > > > > > > > > > > > > > > > > >>> > of
> > > > > > > > > > > > > > > > > > > >>> > > > > >> coupling
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > them together. I'm
> > still a
> > > > bit
> > > > > > > > > inclining
> > > > > > > > > > > > > towards
> > > > > > > > > > > > > > > > > > > colocation
> > > > > > > > > > > > > > > > > > > >>> but
> > > > > > > > > > > > > > > > > > > >>> > if
> > > > > > > > > > > > > > > > > > > >>> > > > there
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > are good rationales
> not
> > to
> > > > do
> > > > > > so I
> > > > > > > > can
> > > > > > > > > > be
> > > > > > > > > > > > > > > convinced
> > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > well.
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> The good rationale is
> that
> > > we
> > > > > have
> > > > > > > no
> > > > > > > > > > > > mechanism
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > colocate
> > > > > > > > > > > > > > > > > > > >>> > > > partitions ;).
> > > > > > > > > > > > > > > > > > > >>> > > > > >> Are you suggesting we
> > store
> > > > the
> > > > > > > group
> > > > > > > > > and
> > > > > > > > > > > > > > > transaction
> > > > > > > > > > > > > > > > > > state
> > > > > > > > > > > > > > > > > > > >>> in the
> > > > > > > > > > > > > > > > > > > >>> > > > same
> > > > > > > > > > > > > > > > > > > >>> > > > > >> log? Can you be more
> > > concrete
> > > > > > about
> > > > > > > > the
> > > > > > > > > > > > benefit?
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> -Jason
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> On Tue, Jun 25, 2019 at
> > > 10:51
> > > > AM
> > > > > > > > > Guozhang
> > > > > > > > > > > > Wang <
> > > > > > > > > > > > > > > > > > > >>> > wangg...@gmail.com>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > Hi Boyang,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > 1. One advantage of
> > retry
> > > > > > against
> > > > > > > > > > on-hold
> > > > > > > > > > > is
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > >>> not
> > > > > > > > > > > > > > > > > > > >>> > > > tie-up a
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > handler thread (of
> > course
> > > > the
> > > > > > > latter
> > > > > > > > > > could
> > > > > > > > > > > > do
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > same
> > > > > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > > > >>> that
> > > > > > > > > > > > > > > > > > > >>> > > > involves
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > using a purgatory
> which
> > is
> > > > > more
> > > > > > > > > > > > complicated),
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > also
> > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > >>> > less
> > > > > > > > > > > > > > > > > > > >>> > > > > >> likely to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > violate request
> timeout.
> > > So
> > > > I
> > > > > > > think
> > > > > > > > > > there
> > > > > > > > > > > > are
> > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > >>> rationales to
> > > > > > > > > > > > > > > > > > > >>> > > > prefer
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > retries.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >  That sounds fair to me,
> > also
> > > > we
> > > > > > are
> > > > > > > > > > avoiding
> > > > > > > > > > > > > usage
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > another
> > > > > > > > > > > > > > > > > > > >>> > > > purgatory
> > > > > > > > > > > > > > > > > > > >>> > > > > > instance. Usually for one
> > > > > back-off
> > > > > > > > > > > > > > > > > > > >>> > > > > > we are only delaying 50ms
> > > > during
> > > > > > > > startup
> > > > > > > > > > > which
> > > > > > > > > > > > is
> > > > > > > > > > > > > > > > trivial
> > > > > > > > > > > > > > > > > > > cost.
> > > > > > > > > > > > > > > > > > > >>> > This
> > > > > > > > > > > > > > > > > > > >>> > > > > > behavior shouldn't be
> > > changed.
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > > > 2. Regarding
> > > > > > > > > "ConsumerRebalanceListener":
> > > > > > > > > > > > both
> > > > > > > > > > > > > > > > > > > >>> > > > ConsumerRebalanceListener
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > and PartitionAssignors
> > are
> > > > > > > > > > > user-customizable
> > > > > > > > > > > > > > > > modules,
> > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > >>> only
> > > > > > > > > > > > > > > > > > > >>> > > > > >> difference
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > is that the former is
> > > > > specified
> > > > > > > via
> > > > > > > > > code
> > > > > > > > > > > and
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > latter
> > > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > >>> > > > specified via
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > config.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > Regarding Jason's
> > proposal
> > > > of
> > > > > > > > > > > > > > ConsumerAssignment,
> > > > > > > > > > > > > > > > one
> > > > > > > > > > > > > > > > > > > thing
> > > > > > > > > > > > > > > > > > > >>> to
> > > > > > > > > > > > > > > > > > > >>> > > note
> > > > > > > > > > > > > > > > > > > >>> > > > > >> though
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > with KIP-429 the
> > > > > > > onPartitionAssigned
> > > > > > > > > may
> > > > > > > > > > > not
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > called
> > > > > > > > > > > > > > > > > > if
> > > > > > > > > > > > > > > > > > > >>> the
> > > > > > > > > > > > > > > > > > > >>> > > > assignment
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > does not change,
> whereas
> > > > > > > > onAssignment
> > > > > > > > > > > would
> > > > > > > > > > > > > > always
> > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > >>> called at
> > > > > > > > > > > > > > > > > > > >>> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > end
> > > > > > > > > > > > > > > > > > > >>> > > > > >> of
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > sync-group response.
> My
> > > > > proposed
> > > > > > > > > > semantics
> > > > > > > > > > > > is
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > `RebalanceListener#onPartitionsXXX`
> > > > > > > > > are
> > > > > > > > > > > used
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > > >>> notifications
> > > > > > > > > > > > > > > > > > > >>> > to
> > > > > > > > > > > > > > > > > > > >>> > > > user,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> and
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > hence if there's no
> > > changes
> > > > > > these
> > > > > > > > will
> > > > > > > > > > not
> > > > > > > > > > > > be
> > > > > > > > > > > > > > > > called,
> > > > > > > > > > > > > > > > > > > >>> whereas
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > `PartitionAssignor` is
> > > used
> > > > > for
> > > > > > > > > assignor
> > > > > > > > > > > > > logic,
> > > > > > > > > > > > > > > > whose
> > > > > > > > > > > > > > > > > > > >>> callback
> > > > > > > > > > > > > > > > > > > >>> > > would
> > > > > > > > > > > > > > > > > > > >>> > > > > >> always
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > be called no matter if
> > the
> > > > > > > > partitions
> > > > > > > > > > have
> > > > > > > > > > > > > > changed
> > > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > > not.
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> I think a third option
> is
> > to
> > > > > > > > gracefully
> > > > > > > > > > > expose
> > > > > > > > > > > > > > > > > generation
> > > > > > > > > > > > > > > > > > id
> > > > > > > > > > > > > > > > > > > >>> as
> > > > > > > > > > > > > > > > > > > >>> > part
> > > > > > > > > > > > > > > > > > > >>> > > > of
> > > > > > > > > > > > > > > > > > > >>> > > > > > consumer API, so that we
> > > don't
> > > > > need
> > > > > > > to
> > > > > > > > > > > > > > > > > > > >>> > > > > > bother overloading
> various
> > > > > > callbacks.
> > > > > > > > Of
> > > > > > > > > > > > course,
> > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > builds
> > > > > > > > > > > > > > > > > > > >>> upon
> > > > > > > > > > > > > > > > > > > >>> > the
> > > > > > > > > > > > > > > > > > > >>> > > > > > assumption that topic
> > > > partitions
> > > > > > > > > > > > > > > > > > > >>> > > > > > will not be included in
> new
> > > > > > > > > initTransaction
> > > > > > > > > > > > API.
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > > > 3. I feel it is a bit
> > > awkward
> > > > > to
> > > > > > > let
> > > > > > > > > the
> > > > > > > > > > > > > > > > TxnCoordinator
> > > > > > > > > > > > > > > > > > > >>> keeping
> > > > > > > > > > > > > > > > > > > >>> > > > partition
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > assignments since it
> is
> > > sort
> > > > > of
> > > > > > > > taking
> > > > > > > > > > > over
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > job
> > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > ConsumerCoordinator,
> and
> > > may
> > > > > > > likely
> > > > > > > > > > cause
> > > > > > > > > > > a
> > > > > > > > > > > > > > > > > split-brain
> > > > > > > > > > > > > > > > > > > >>> problem
> > > > > > > > > > > > > > > > > > > >>> > as
> > > > > > > > > > > > > > > > > > > >>> > > > two
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > coordinators keep a
> copy
> > > of
> > > > > this
> > > > > > > > > > > assignment
> > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > > may
> > > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > >>> > > different.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > I think co-locating
> does
> > > > have
> > > > > > some
> > > > > > > > > > merits
> > > > > > > > > > > > > here,
> > > > > > > > > > > > > > > i.e.
> > > > > > > > > > > > > > > > > > > >>> letting the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > ConsumerCoordinator
> > which
> > > > has
> > > > > > the
> > > > > > > > > > > > > > source-of-truth
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > >>> assignment
> > > > > > > > > > > > > > > > > > > >>> > to
> > > > > > > > > > > > > > > > > > > >>> > > > act
> > > > > > > > > > > > > > > > > > > >>> > > > > >> as
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > the TxnCoordinator as
> > > well;
> > > > > but
> > > > > > I
> > > > > > > > > agree
> > > > > > > > > > > > > there's
> > > > > > > > > > > > > > > also
> > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > >>> cons
> > > > > > > > > > > > > > > > > > > >>> > of
> > > > > > > > > > > > > > > > > > > >>> > > > > >> coupling
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > them together. I'm
> > still a
> > > > bit
> > > > > > > > > inclining
> > > > > > > > > > > > > towards
> > > > > > > > > > > > > > > > > > > colocation
> > > > > > > > > > > > > > > > > > > >>> but
> > > > > > > > > > > > > > > > > > > >>> > if
> > > > > > > > > > > > > > > > > > > >>> > > > there
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > are good rationales
> not
> > to
> > > > do
> > > > > > so I
> > > > > > > > can
> > > > > > > > > > be
> > > > > > > > > > > > > > > convinced
> > > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > well.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > > The purpose of
> co-location
> > is
> > > > to
> > > > > > let
> > > > > > > > txn
> > > > > > > > > > > > > > coordinator
> > > > > > > > > > > > > > > > see
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> group
> > > > > > > > > > > > > > > > > > > >>> > > > > > assignment. This priority
> > is
> > > > > > weakened
> > > > > > > > > > > > > > > > > > > >>> > > > > > when we already have
> > defense
> > > on
> > > > > the
> > > > > > > > > > consumer
> > > > > > > > > > > > > offset
> > > > > > > > > > > > > > > > > fetch,
> > > > > > > > > > > > > > > > > > > so I
> > > > > > > > > > > > > > > > > > > >>> > guess
> > > > > > > > > > > > > > > > > > > >>> > > > it's
> > > > > > > > > > > > > > > > > > > >>> > > > > > not super important
> > anymore.
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > 4. I guess I'm
> > preferring
> > > > the
> > > > > > > > > philosophy
> > > > > > > > > > > of
> > > > > > > > > > > > > > "only
> > > > > > > > > > > > > > > > add
> > > > > > > > > > > > > > > > > > > >>> configs if
> > > > > > > > > > > > > > > > > > > >>> > > > > >> there's no
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > other ways", since
> more
> > > and
> > > > > more
> > > > > > > > > configs
> > > > > > > > > > > > would
> > > > > > > > > > > > > > > make
> > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > less
> > > > > > > > > > > > > > > > > > > >>> and
> > > > > > > > > > > > > > > > > > > >>> > > less
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > intuitive out of the
> box
> > > to
> > > > > use.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > I think it's a valid
> > point
> > > > > that
> > > > > > > > checks
> > > > > > > > > > > upon
> > > > > > > > > > > > > > > starting
> > > > > > > > > > > > > > > > > up
> > > > > > > > > > > > > > > > > > > >>> does not
> > > > > > > > > > > > > > > > > > > >>> > > > cope
> > > > > > > > > > > > > > > > > > > >>> > > > > >> with
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > brokers downgrading
> but
> > > even
> > > > > > with
> > > > > > > a
> > > > > > > > > > > config,
> > > > > > > > > > > > > but
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > still
> > > > > > > > > > > > > > > > > > > >>> hard
> > > > > > > > > > > > > > > > > > > >>> > > for
> > > > > > > > > > > > > > > > > > > >>> > > > > >> users
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > to determine when they
> > can
> > > > be
> > > > > > > > ensured
> > > > > > > > > > the
> > > > > > > > > > > > > broker
> > > > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > > > never
> > > > > > > > > > > > > > > > > > > >>> > > > downgrade
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > anymore and hence can
> > > safely
> > > > > > > switch
> > > > > > > > > the
> > > > > > > > > > > > > config.
> > > > > > > > > > > > > > So
> > > > > > > > > > > > > > > > my
> > > > > > > > > > > > > > > > > > > >>> feeling is
> > > > > > > > > > > > > > > > > > > >>> > > > that
> > > > > > > > > > > > > > > > > > > >>> > > > > >> this
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > config would not be
> > > helping
> > > > > too
> > > > > > > much
> > > > > > > > > > > still.
> > > > > > > > > > > > If
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > want
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > >>> be at
> > > > > > > > > > > > > > > > > > > >>> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> safer
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > side, then I'd suggest
> > we
> > > > > modify
> > > > > > > the
> > > > > > > > > > > > > Coordinator
> > > > > > > > > > > > > > > ->
> > > > > > > > > > > > > > > > > > > >>> > NetworkClient
> > > > > > > > > > > > > > > > > > > >>> > > > > >> hierarchy
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > to allow the
> > NetworkClient
> > > > > being
> > > > > > > > able
> > > > > > > > > to
> > > > > > > > > > > > pass
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > APIVersion
> > > > > > > > > > > > > > > > > > > >>> > > > metadata to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > Coordinator, so that
> > > > > Coordinator
> > > > > > > can
> > > > > > > > > > rely
> > > > > > > > > > > on
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > logic
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > >>> > change
> > > > > > > > > > > > > > > > > > > >>> > > > its
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > behavior dynamically.
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > > The stream thread init
> > could
> > > > not
> > > > > be
> > > > > > > > > > supported
> > > > > > > > > > > > by
> > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > client
> > > > > > > > > > > > > > > > > > > >>> > coordinator
> > > > > > > > > > > > > > > > > > > >>> > > > > > behavior change on the
> fly,
> > > > > > > > > > > > > > > > > > > >>> > > > > > we are only losing
> > > > possibilities
> > > > > > > after
> > > > > > > > we
> > > > > > > > > > > > > > > initialized.
> > > > > > > > > > > > > > > > > > (main
> > > > > > > > > > > > > > > > > > > >>> thread
> > > > > > > > > > > > > > > > > > > >>> > > > gets
> > > > > > > > > > > > > > > > > > > >>> > > > > > exit and no thread has
> > global
> > > > > > picture
> > > > > > > > > > > anymore)
> > > > > > > > > > > > > > > > > > > >>> > > > > > If we do want to support
> > auto
> > > > > > version
> > > > > > > > > > > > detection,
> > > > > > > > > > > > > > > admin
> > > > > > > > > > > > > > > > > > client
> > > > > > > > > > > > > > > > > > > >>> > request
> > > > > > > > > > > > > > > > > > > >>> > > > in
> > > > > > > > > > > > > > > > > > > >>> > > > > > this sense shall be
> easier.
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > 5. I do not have a
> > > concrete
> > > > > idea
> > > > > > > > about
> > > > > > > > > > how
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > > impact
> > > > > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > > > >>> Connect
> > > > > > > > > > > > > > > > > > > >>> > > > would
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > make, maybe Randall or
> > > > > > Konstantine
> > > > > > > > can
> > > > > > > > > > > help
> > > > > > > > > > > > > > here?
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > > Sounds good, let's see
> > their
> > > > > > > thoughts.
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > Guozhang
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > On Mon, Jun 24, 2019
> at
> > > > 10:26
> > > > > PM
> > > > > > > > > Boyang
> > > > > > > > > > > > Chen <
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > reluctanthero...@gmail.com>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > Hey Jason,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > thank you for the
> > > proposal
> > > > > > here.
> > > > > > > > > Some
> > > > > > > > > > of
> > > > > > > > > > > > my
> > > > > > > > > > > > > > > > thoughts
> > > > > > > > > > > > > > > > > > > >>> below.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > On Mon, Jun 24, 2019
> > at
> > > > 8:58
> > > > > > PM
> > > > > > > > > Jason
> > > > > > > > > > > > > > Gustafson
> > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > >>> > > > ja...@confluent.io>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > Hi Boyang,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > Thanks for picking
> > > this
> > > > > up!
> > > > > > > > Still
> > > > > > > > > > > > reading
> > > > > > > > > > > > > > > > through
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> > updates,
> > > > > > > > > > > > > > > > > > > >>> > > > but
> > > > > > > > > > > > > > > > > > > >>> > > > > >> here
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > are
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > a couple initial
> > > > comments
> > > > > on
> > > > > > > the
> > > > > > > > > > APIs:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > 1. The
> > > > > `TxnProducerIdentity`
> > > > > > > > class
> > > > > > > > > > is
> > > > > > > > > > > a
> > > > > > > > > > > > > bit
> > > > > > > > > > > > > > > > > > awkward. I
> > > > > > > > > > > > > > > > > > > >>> think
> > > > > > > > > > > > > > > > > > > >>> > > we
> > > > > > > > > > > > > > > > > > > >>> > > > are
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > trying
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > to encapsulate
> state
> > > > from
> > > > > > the
> > > > > > > > > > current
> > > > > > > > > > > > > group
> > > > > > > > > > > > > > > > > > > assignment.
> > > > > > > > > > > > > > > > > > > >>> > Maybe
> > > > > > > > > > > > > > > > > > > >>> > > > > >> something
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > like
> > > > `ConsumerAssignment`
> > > > > > > would
> > > > > > > > be
> > > > > > > > > > > > > clearer?
> > > > > > > > > > > > > > If
> > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > make
> > > > > > > > > > > > > > > > > > > >>> the
> > > > > > > > > > > > > > > > > > > >>> > > usage
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > consistent
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > across the
> consumer
> > > and
> > > > > > > > producer,
> > > > > > > > > > then
> > > > > > > > > > > > we
> > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > avoid
> > > > > > > > > > > > > > > > > > > >>> exposing
> > > > > > > > > > > > > > > > > > > >>> > > > > >> internal
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > state
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > like the
> > generationId.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > For example:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > // Public API
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > interface
> > > > > > ConsumerAssignment {
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> >  Set<TopicPartition>
> > > > > > > > > partittions();
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > }
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > // Not a public
> API
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > class
> > > > > > > InternalConsumerAssignment
> > > > > > > > > > > > > implements
> > > > > > > > > > > > > > > > > > > >>> > > ConsumerAssignment {
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> >  Set<TopicPartition>
> > > > > > > > partittions;
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >   int
> generationId;
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > }
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > Then we can change
> > the
> > > > > > > rebalance
> > > > > > > > > > > > listener
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > something
> > > > > > > > > > > > > > > > > > > >>> like
> > > > > > > > > > > > > > > > > > > >>> > > > this:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > onPartitionsAssigned(ConsumerAssignment
> > > > > > > > > > > > > > > > > assignment)
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > And on the
> producer:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > void
> > > > > initTransactions(String
> > > > > > > > > > groupId,
> > > > > > > > > > > > > > > > > > > ConsumerAssignment
> > > > > > > > > > > > > > > > > > > >>> > > > > >> assignment);
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > 2. Another bit of
> > > > > > awkwardness
> > > > > > > is
> > > > > > > > > the
> > > > > > > > > > > > fact
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > > > >>> to
> > > > > > > > > > > > > > > > > > > >>> > pass
> > > > > > > > > > > > > > > > > > > >>> > > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > groupId
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > through both
> > > > > > > initTransactions()
> > > > > > > > > and
> > > > > > > > > > > > > > > > > > > >>> > > sendOffsetsToTransaction().
> > > > > > > > > > > > > > > > > > > >>> > > > We
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > could
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > consider a config
> > > > instead.
> > > > > > > Maybe
> > > > > > > > > > > > something
> > > > > > > > > > > > > > > like
> > > > > > > > > > > > > > > > `
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> transactional.group.id
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > `?
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > Then we could
> > simplify
> > > > the
> > > > > > > > > producer
> > > > > > > > > > > > APIs,
> > > > > > > > > > > > > > > > > > potentially
> > > > > > > > > > > > > > > > > > > >>> even
> > > > > > > > > > > > > > > > > > > >>> > > > > >> deprecating
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > current
> > > > > > > > sendOffsetsToTransaction.
> > > > > > > > > In
> > > > > > > > > > > > fact,
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > > > > > >>> > usage,
> > > > > > > > > > > > > > > > > > > >>> > > > the `
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > transational.id`
> > > config
> > > > > is
> > > > > > > not
> > > > > > > > > > > needed.
> > > > > > > > > > > > It
> > > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > >>> nice if
> > > > > > > > > > > > > > > > > > > >>> > we
> > > > > > > > > > > > > > > > > > > >>> > > > don't
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > have
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > provide it.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > I like the idea of
> > > > > > combining 1
> > > > > > > > and
> > > > > > > > > > 2.
> > > > > > > > > > > We
> > > > > > > > > > > > > > could
> > > > > > > > > > > > > > > > > > > >>> definitely
> > > > > > > > > > > > > > > > > > > >>> > pass
> > > > > > > > > > > > > > > > > > > >>> > > > in a
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > group.id config
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > so that we could
> avoid
> > > > > > exposing
> > > > > > > > that
> > > > > > > > > > > > > > information
> > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > >>> public
> > > > > > > > > > > > > > > > > > > >>> > > API.
> > > > > > > > > > > > > > > > > > > >>> > > > The
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > question I have
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > is that whether we
> > > should
> > > > > name
> > > > > > > the
> > > > > > > > > > > > interface
> > > > > > > > > > > > > > > > > > > >>> `GroupAssignment`
> > > > > > > > > > > > > > > > > > > >>> > > > > >> instead,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > so
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > that Connect later
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > could also extend on
> > the
> > > > > same
> > > > > > > > > > interface,
> > > > > > > > > > > > > just
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > echo
> > > > > > > > > > > > > > > > > > > >>> > Guozhang's
> > > > > > > > > > > > > > > > > > > >>> > > > point
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > here, Also the base
> > > > > interface
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > is better to be
> > defined
> > > > > empty
> > > > > > > for
> > > > > > > > > easy
> > > > > > > > > > > > > > > extension,
> > > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > > > >>> define an
> > > > > > > > > > > > > > > > > > > >>> > > > > >> abstract
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > type called
> `Resource`
> > > to
> > > > be
> > > > > > > > > shareable
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > later IMHO.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > By the way, I'm a
> > bit
> > > > > > confused
> > > > > > > > > about
> > > > > > > > > > > > > > > discussion
> > > > > > > > > > > > > > > > > > above
> > > > > > > > > > > > > > > > > > > >>> about
> > > > > > > > > > > > > > > > > > > >>> > > > > >> colocating
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > txn and group
> > > > > coordinators.
> > > > > > > That
> > > > > > > > > is
> > > > > > > > > > > not
> > > > > > > > > > > > > > > actually
> > > > > > > > > > > > > > > > > > > >>> necessary,
> > > > > > > > > > > > > > > > > > > >>> > is
> > > > > > > > > > > > > > > > > > > >>> > > > it?
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > Yes, this is not a
> > > > > > requirement
> > > > > > > > for
> > > > > > > > > > > this
> > > > > > > > > > > > > KIP,
> > > > > > > > > > > > > > > > > because
> > > > > > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > > > >>> is
> > > > > > > > > > > > > > > > > > > >>> > > > > >> inherently
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > impossible to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > achieve co-locating
> > > topic
> > > > > > > > partition
> > > > > > > > > > of
> > > > > > > > > > > > > > > > transaction
> > > > > > > > > > > > > > > > > > log
> > > > > > > > > > > > > > > > > > > >>> and
> > > > > > > > > > > > > > > > > > > >>> > > > consumed
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > offset
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > topics.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > Thanks,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > Jason
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > On Mon, Jun 24, 2019
> > at
> > > > > 10:07
> > > > > > AM
> > > > > > > > > > Boyang
> > > > > > > > > > > > > Chen <
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > reluctanthero...@gmail.com
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > Thank you Ismael
> > for
> > > > the
> > > > > > > > > > suggestion.
> > > > > > > > > > > > We
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > attempt
> > > > > > > > > > > > > > > > > > > >>> to
> > > > > > > > > > > > > > > > > > > >>> > > > address
> > > > > > > > > > > > > > > > > > > >>> > > > > >> it by
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > giving more
> > details
> > > to
> > > > > > > > rejected
> > > > > > > > > > > > > > alternative
> > > > > > > > > > > > > > > > > > section.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > Thank you for
> the
> > > > > comment
> > > > > > > > > > Guozhang!
> > > > > > > > > > > > > > Answers
> > > > > > > > > > > > > > > > are
> > > > > > > > > > > > > > > > > > > inline
> > > > > > > > > > > > > > > > > > > >>> > > below.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > On Sun, Jun 23,
> > 2019
> > > > at
> > > > > > 6:33
> > > > > > > > PM
> > > > > > > > > > > > Guozhang
> > > > > > > > > > > > > > > Wang
> > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > >>> > > > wangg...@gmail.com
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > Hello Boyang,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > Thanks for the
> > > KIP,
> > > > I
> > > > > > have
> > > > > > > > > some
> > > > > > > > > > > > > comments
> > > > > > > > > > > > > > > > > below:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > 1. "Once
> > > > transactions
> > > > > > are
> > > > > > > > > > > complete,
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > call
> > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > >>> > return."
> > > > > > > > > > > > > > > > > > > >>> > > > This
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > seems
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > different from
> > the
> > > > > > > existing
> > > > > > > > > > > > behavior,
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > >>> would
> > > > > > > > > > > > > > > > > > > >>> > > > return a
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > retriable
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > CONCURRENT_TRANSACTIONS
> > > > > > > and
> > > > > > > > > let
> > > > > > > > > > > the
> > > > > > > > > > > > > > client
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > >>> retry, is
> > > > > > > > > > > > > > > > > > > >>> > > this
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > intentional?
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > I don’t think it
> > is
> > > > > > > > intentional,
> > > > > > > > > > > and I
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > defer
> > > > > > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > > > > >>> > > > question to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > Jason
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > when he got time
> > to
> > > > > answer
> > > > > > > > since
> > > > > > > > > > > from
> > > > > > > > > > > > > > what I
> > > > > > > > > > > > > > > > > > > >>> understood
> > > > > > > > > > > > > > > > > > > >>> > > retry
> > > > > > > > > > > > > > > > > > > >>> > > > and
> > > > > > > > > > > > > > > > > > > >>> > > > > >> on
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > hold
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > seem both valid
> > > > > > approaches.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > 2. "an
> overload
> > to
> > > > > > > > > > > > > onPartitionsAssigned
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> > consumer's
> > > > > > > > > > > > > > > > > > > >>> > > > > >> rebalance
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > listener
> > > interface":
> > > > > as
> > > > > > > part
> > > > > > > > > of
> > > > > > > > > > > > > KIP-341
> > > > > > > > > > > > > > > > we've
> > > > > > > > > > > > > > > > > > > >>> already
> > > > > > > > > > > > > > > > > > > >>> > add
> > > > > > > > > > > > > > > > > > > >>> > > > this
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > information
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > to the
> > > onAssignment
> > > > > > > > callback.
> > > > > > > > > > > Would
> > > > > > > > > > > > > this
> > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > >>> sufficient?
> > > > > > > > > > > > > > > > > > > >>> > Or
> > > > > > > > > > > > > > > > > > > >>> > > > more
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > generally
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > speaking,
> which
> > > > > > > information
> > > > > > > > > have
> > > > > > > > > > > to
> > > > > > > > > > > > be
> > > > > > > > > > > > > > > > passed
> > > > > > > > > > > > > > > > > > > >>> around in
> > > > > > > > > > > > > > > > > > > >>> > > > > >> rebalance
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > callback
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > while others
> can
> > > be
> > > > > > passed
> > > > > > > > > > around
> > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > >>> PartitionAssignor
> > > > > > > > > > > > > > > > > > > >>> > > > > >> callback? In
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > Streams
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > for example
> both
> > > > > > callbacks
> > > > > > > > are
> > > > > > > > > > > used
> > > > > > > > > > > > > but
> > > > > > > > > > > > > > > most
> > > > > > > > > > > > > > > > > > > >>> critical
> > > > > > > > > > > > > > > > > > > >>> > > > > >> information
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > is
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > passed
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > via
> > onAssignment.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > We still need to
> > > > extend
> > > > > > > > > > > > > > > > > ConsumerRebalanceListener
> > > > > > > > > > > > > > > > > > > >>> because
> > > > > > > > > > > > > > > > > > > >>> > > > it’s the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > interface we
> could
> > > > have
> > > > > > > public
> > > > > > > > > > > access
> > > > > > > > > > > > > to.
> > > > > > > > > > > > > > > The
> > > > > > > > > > > > > > > > > > > >>> > #onAssignment
> > > > > > > > > > > > > > > > > > > >>> > > > call
> > > > > > > > > > > > > > > > > > > >>> > > > > >> is
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > defined
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > on
> > PartitionAssignor
> > > > > level
> > > > > > > > which
> > > > > > > > > > is
> > > > > > > > > > > > not
> > > > > > > > > > > > > > easy
> > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > work
> > > > > > > > > > > > > > > > > > > >>> with
> > > > > > > > > > > > > > > > > > > >>> > > > external
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > producers.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > 3. "We propose
> > to
> > > > use
> > > > > a
> > > > > > > > > separate
> > > > > > > > > > > > > record
> > > > > > > > > > > > > > > type
> > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > >>> order to
> > > > > > > > > > > > > > > > > > > >>> > > > store
> > > > > > > > > > > > > > > > > > > >>> > > > > >> the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > group
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > assignment.":
> > > hmm, I
> > > > > > > thought
> > > > > > > > > > with
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > third
> > > > > > > > > > > > > > > > > > typed
> > > > > > > > > > > > > > > > > > > >>> > > > > >> FindCoordinator,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > same
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > broker that
> act
> > as
> > > > the
> > > > > > > > > consumer
> > > > > > > > > > > > > > > coordinator
> > > > > > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > > > >>> > always
> > > > > > > > > > > > > > > > > > > >>> > > be
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > selected
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > as
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > the txn
> > > coordinator,
> > > > > in
> > > > > > > > which
> > > > > > > > > > case
> > > > > > > > > > > > it
> > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > access
> > > > > > > > > > > > > > > > > > > its
> > > > > > > > > > > > > > > > > > > >>> > local
> > > > > > > > > > > > > > > > > > > >>> > > > cache
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > metadata /
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > offset topic
> to
> > > get
> > > > > this
> > > > > > > > > > > information
> > > > > > > > > > > > > > > > already?
> > > > > > > > > > > > > > > > > We
> > > > > > > > > > > > > > > > > > > >>> just
> > > > > > > > > > > > > > > > > > > >>> > need
> > > > > > > > > > > > > > > > > > > >>> > > > to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> think
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > about
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > how to make
> > these
> > > > two
> > > > > > > > modules
> > > > > > > > > > > > directly
> > > > > > > > > > > > > > > > > exchange
> > > > > > > > > > > > > > > > > > > >>> > > information
> > > > > > > > > > > > > > > > > > > >>> > > > > >> without
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > messing
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > up the code
> > > > hierarchy.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > These two
> > > coordinators
> > > > > > will
> > > > > > > be
> > > > > > > > > on
> > > > > > > > > > > the
> > > > > > > > > > > > > same
> > > > > > > > > > > > > > > > > broker
> > > > > > > > > > > > > > > > > > > only
> > > > > > > > > > > > > > > > > > > >>> > when
> > > > > > > > > > > > > > > > > > > >>> > > > > >> number of
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > partitions for
> > > > > transaction
> > > > > > > > state
> > > > > > > > > > > topic
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > > > > > >>> offset
> > > > > > > > > > > > > > > > > > > >>> > > > topic
> > > > > > > > > > > > > > > > > > > >>> > > > > >> are
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > same. This
> > normally
> > > > > holds
> > > > > > > > true,
> > > > > > > > > > but
> > > > > > > > > > > > I'm
> > > > > > > > > > > > > > > afraid
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > we couldn't make
> > > this
> > > > > > > > > assumption?
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > 4. The config of
> > > > > > > > > > > > > > > > > > "CONSUMER_GROUP_AWARE_TRANSACTION":
> > > > > > > > > > > > > > > > > > > >>> it
> > > > > > > > > > > > > > > > > > > >>> > > seems
> > > > > > > > > > > > > > > > > > > >>> > > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > goal
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > of
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > this config is
> > > just
> > > > to
> > > > > > > avoid
> > > > > > > > > > > > > > old-versioned
> > > > > > > > > > > > > > > > > > broker
> > > > > > > > > > > > > > > > > > > >>> to not
> > > > > > > > > > > > > > > > > > > >>> > > be
> > > > > > > > > > > > > > > > > > > >>> > > > > >> able to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > recognize
> newer
> > > > > > versioned
> > > > > > > > > > client.
> > > > > > > > > > > I
> > > > > > > > > > > > > > think
> > > > > > > > > > > > > > > if
> > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > > > >>> do
> > > > > > > > > > > > > > > > > > > >>> > > > something
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > else
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > avoid this
> > config
> > > > > > though,
> > > > > > > > for
> > > > > > > > > > > > example
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > use
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> > > > embedded
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > AdminClient
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > to send the
> > > > APIVersion
> > > > > > > > request
> > > > > > > > > > > upon
> > > > > > > > > > > > > > > starting
> > > > > > > > > > > > > > > > > up,
> > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > >>> > based
> > > > > > > > > > > > > > > > > > > >>> > > > on
> > > > > > > > > > > > > > > > > > > >>> > > > > >> the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > returned
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > value decides
> > > > whether
> > > > > to
> > > > > > > go
> > > > > > > > to
> > > > > > > > > > the
> > > > > > > > > > > > old
> > > > > > > > > > > > > > > code
> > > > > > > > > > > > > > > > > path
> > > > > > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > > > >>> the
> > > > > > > > > > > > > > > > > > > >>> > > new
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > behavior.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > Admittedly
> > asking
> > > a
> > > > > > random
> > > > > > > > > > broker
> > > > > > > > > > > > > about
> > > > > > > > > > > > > > > > > > APIVersion
> > > > > > > > > > > > > > > > > > > >>> does
> > > > > > > > > > > > > > > > > > > >>> > > not
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > guarantee
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > whole
> cluster's
> > > > > > versions,
> > > > > > > > but
> > > > > > > > > > what
> > > > > > > > > > > > we
> > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > do
> > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > >>> first
> > > > > > > > > > > > > > > > > > > >>> > > 1)
> > > > > > > > > > > > > > > > > > > >>> > > > find
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > coordinator
> (and
> > > if
> > > > > the
> > > > > > > > random
> > > > > > > > > > > > broker
> > > > > > > > > > > > > > does
> > > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > > > > even
> > > > > > > > > > > > > > > > > > > >>> > > > recognize
> > > > > > > > > > > > > > > > > > > >>> > > > > >> the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > new
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > discover type,
> > > fall
> > > > > back
> > > > > > > to
> > > > > > > > > old
> > > > > > > > > > > path
> > > > > > > > > > > > > > > > > directly),
> > > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > >>> then
> > > > > > > > > > > > > > > > > > > >>> > > 2)
> > > > > > > > > > > > > > > > > > > >>> > > > ask
> > > > > > > > > > > > > > > > > > > >>> > > > > >> the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > discovered
> > > > coordinator
> > > > > > > about
> > > > > > > > > its
> > > > > > > > > > > > > > supported
> > > > > > > > > > > > > > > > > > > >>> APIVersion.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > The caveat here
> is
> > > > that
> > > > > we
> > > > > > > > have
> > > > > > > > > to
> > > > > > > > > > > > make
> > > > > > > > > > > > > > sure
> > > > > > > > > > > > > > > > > both
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> > group
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > coordinator
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > and
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > transaction
> > > > coordinator
> > > > > > are
> > > > > > > on
> > > > > > > > > the
> > > > > > > > > > > > > latest
> > > > > > > > > > > > > > > > > version
> > > > > > > > > > > > > > > > > > > >>> during
> > > > > > > > > > > > > > > > > > > >>> > > init
> > > > > > > > > > > > > > > > > > > >>> > > > > >> stage.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > This
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > is potentially
> > > doable
> > > > as
> > > > > > we
> > > > > > > > only
> > > > > > > > > > > need
> > > > > > > > > > > > a
> > > > > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > > > > > > >>> group.id
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > to check that.
> In
> > > the
> > > > > > > > meantime,
> > > > > > > > > a
> > > > > > > > > > > > > > hard-coded
> > > > > > > > > > > > > > > > > > config
> > > > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > >>> > > still a
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > favorable
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > backup in case
> the
> > > > > server
> > > > > > > has
> > > > > > > > > > > > > downgraded,
> > > > > > > > > > > > > > so
> > > > > > > > > > > > > > > > you
> > > > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > >>> want
> > > > > > > > > > > > > > > > > > > >>> > > to
> > > > > > > > > > > > > > > > > > > >>> > > > use
> > > > > > > > > > > > > > > > > > > >>> > > > > >> a
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > new
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > version client
> > > without
> > > > > > > > `consumer
> > > > > > > > > > > > group`
> > > > > > > > > > > > > > > > > > > transactional
> > > > > > > > > > > > > > > > > > > >>> > > support.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > 5. This is a
> meta
> > > > > > question:
> > > > > > > > have
> > > > > > > > > > you
> > > > > > > > > > > > > > > > considered
> > > > > > > > > > > > > > > > > > how
> > > > > > > > > > > > > > > > > > > >>> this
> > > > > > > > > > > > > > > > > > > >>> > can
> > > > > > > > > > > > > > > > > > > >>> > > > be
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > applied
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > Kafka Connect
> as
> > > > well?
> > > > > > For
> > > > > > > > > > > example,
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > source
> > > > > > > > > > > > > > > > > > > >>> > connectors,
> > > > > > > > > > > > > > > > > > > >>> > > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > assignment
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > is not by
> > > > > "partitions",
> > > > > > > but
> > > > > > > > by
> > > > > > > > > > > some
> > > > > > > > > > > > > > other
> > > > > > > > > > > > > > > > sort
> > > > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > >>> > > > "resources"
> > > > > > > > > > > > > > > > > > > >>> > > > > >> based
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > on
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > source
> systems,
> > > how
> > > > > > > KIP-447
> > > > > > > > > > would
> > > > > > > > > > > > > affect
> > > > > > > > > > > > > > > > Kafka
> > > > > > > > > > > > > > > > > > > >>> > Connectors
> > > > > > > > > > > > > > > > > > > >>> > > > that
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > implemented
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > EOS as well?
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > No, it's not
> > > currently
> > > > > > > > included
> > > > > > > > > in
> > > > > > > > > > > the
> > > > > > > > > > > > > > > scope.
> > > > > > > > > > > > > > > > > > Could
> > > > > > > > > > > > > > > > > > > >>> you
> > > > > > > > > > > > > > > > > > > >>> > > point
> > > > > > > > > > > > > > > > > > > >>> > > > me
> > > > > > > > > > > > > > > > > > > >>> > > > > >> to a
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > sample source
> > > > connector
> > > > > > who
> > > > > > > > uses
> > > > > > > > > > > EOS?
> > > > > > > > > > > > > > Could
> > > > > > > > > > > > > > > > > always
> > > > > > > > > > > > > > > > > > > >>> > > piggy-back
> > > > > > > > > > > > > > > > > > > >>> > > > into
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > the
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > TxnProducerIdentity
> > > > > struct
> > > > > > > > with
> > > > > > > > > > more
> > > > > > > > > > > > > > > > information
> > > > > > > > > > > > > > > > > > > such
> > > > > > > > > > > > > > > > > > > >>> as
> > > > > > > > > > > > > > > > > > > >>> > > > tasks. If
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > this is
> something
> > to
> > > > > > support
> > > > > > > > in
> > > > > > > > > > near
> > > > > > > > > > > > > term,
> > > > > > > > > > > > > > > an
> > > > > > > > > > > > > > > > > > > abstract
> > > > > > > > > > > > > > > > > > > >>> > type
> > > > > > > > > > > > > > > > > > > >>> > > > called
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > "Resource" could
> > be
> > > > > > provided
> > > > > > > > and
> > > > > > > > > > let
> > > > > > > > > > > > > topic
> > > > > > > > > > > > > > > > > > partition
> > > > > > > > > > > > > > > > > > > >>> and
> > > > > > > > > > > > > > > > > > > >>> > > > connect
> > > > > > > > > > > > > > > > > > > >>> > > > > >> task
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > implement it.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > Guozhang
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > On Sat, Jun
> 22,
> > > 2019
> > > > > at
> > > > > > > 8:40
> > > > > > > > > PM
> > > > > > > > > > > > Ismael
> > > > > > > > > > > > > > > Juma
> > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > >>> > > > ism...@juma.me.uk>
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > Hi Boyang,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > Thanks for
> the
> > > > KIP.
> > > > > > It's
> > > > > > > > > good
> > > > > > > > > > > that
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > listed a
> > > > > > > > > > > > > > > > > > > >>> number
> > > > > > > > > > > > > > > > > > > >>> > of
> > > > > > > > > > > > > > > > > > > >>> > > > > >> rejected
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > >
> alternatives.
> > It
> > > > > would
> > > > > > > be
> > > > > > > > > > > helpful
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > an
> > > > > > > > > > > > > > > > > > > >>> > explanation
> > > > > > > > > > > > > > > > > > > >>> > > > of
> > > > > > > > > > > > > > > > > > > >>> > > > > >> why
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > they
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > were
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > rejected.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > Ismael
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > On Sat, Jun
> > 22,
> > > > 2019
> > > > > > at
> > > > > > > > 8:31
> > > > > > > > > > PM
> > > > > > > > > > > > > Boyang
> > > > > > > > > > > > > > > > Chen
> > > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > >>> > > > > >> bche...@outlook.com
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > wrote:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > > Hey all,
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > > I would
> like
> > > to
> > > > > > start
> > > > > > > a
> > > > > > > > > > > > discussion
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > > KIP-447:
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> >
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > > this is a
> > work
> > > > > > > > originated
> > > > > > > > > by
> > > > > > > > > > > > Jason
> > > > > > > > > > > > > > > > > Gustafson
> > > > > > > > > > > > > > > > > > > >>> and we
> > > > > > > > > > > > > > > > > > > >>> > > > would
> > > > > > > > > > > > > > > > > > > >>> > > > > >> like
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > to
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > proceed
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > > into
> > > discussion
> > > > > > stage.
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > > Let me
> know
> > > your
> > > > > > > > thoughts,
> > > > > > > > > > > > thanks!
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > > Boyang
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > --
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > > -- Guozhang
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > --
> > > > > > > > > > > > > > > > > > > >>> > > > > >> > -- Guozhang
> > > > > > > > > > > > > > > > > > > >>> > > > > >> >
> > > > > > > > > > > > > > > > > > > >>> > > > > >>
> > > > > > > > > > > > > > > > > > > >>> > > > > >
> > > > > > > > > > > > > > > > > > > >>> > > > >
> > > > > > > > > > > > > > > > > > > >>> > > >
> > > > > > > > > > > > > > > > > > > >>> > >
> > > > > > > > > > > > > > > > > > > >>> >
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> --
> > > > > > > > > > > > > > > > > > > >>> -- Guozhang
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > -- Guozhang
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > -- Guozhang
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > -- Guozhang
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > -- Guozhang
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> > -- Guozhang
> >
>