Hey there,
bumping this thread for any further KIP-590 discussion, since it's been
quiet for a while.
Boyang
On Thu, May 21, 2020 at 10:31 AM Boyang Chen <reluctanthero...@gmail.com>
wrote:
> Thanks David, I agree the wording here is not clear, and the fellow broker
> should just send a new CreateTopicRequest in this case.
>
> In the meantime, we had some offline discussion about the Envelope API as
> well. Although it provides certain privileges like data embedding and
> principal embedding, it creates a security hole by letting a malicious user
> impersonate any forwarding broker, thus pretending to be any admin user.
> Passing the principal around also increases the vulnerability, compared
> with other standard ways such as passing a verified token, but it is
> unfortunately not fully supported with Kafka security.
>
> So for the security concerns, we are abandoning the Envelope approach and
> fallback to just forward the raw admin requests. The authentication will
> happen on the receiving broker side instead of on the controller, so that
> we are able to stripped off the principal fields and only include the
> principal in header as optional field for audit logging purpose.
> Furthermore, we shall propose adding a separate endpoint for
> broker-controller communication which should be recommended to enable
> secure connections so that a malicious client could not pretend to be a
> broker and perform impersonation.
>
> Let me know your thoughts.
>
> Best,
> Boyang
>
> On Tue, May 19, 2020 at 12:17 AM David Jacot <dja...@confluent.io> wrote:
>
>> Hi Boyang,
>>
>> I've got another question regarding the auto topic creation case. The KIP
>> says: "Currently the target broker shall just utilize its own ZK client to
>> create
>> internal topics, which is disallowed in the bridge release. For above
>> scenarios,
>> non-controller broker shall just forward a CreateTopicRequest to the
>> controller
>> instead and let controller take care of the rest, while waiting for the
>> response
>> in the meantime." There will be no request to forward in this case, right?
>> Instead,
>> a CreateTopicsRequest is created and sent to the controller node.
>>
>> When the CreateTopicsRequest is sent as a side effect of the
>> MetadataRequest,
>> it would be good to know the principal and the clientId in the controller
>> (quota,
>> audit, etc.). Do you plan to use the Envelope API for this case as well or
>> to call
>> the regular API directly? Another was to phrase it would be: Shall the
>> internal
>> CreateTopicsRequest be sent with the original metadata (principal,
>> clientId, etc.)
>> of the MetadataRequest or as an admin request?
>>
>> Best,
>> David
>>
>> On Fri, May 8, 2020 at 2:04 AM Guozhang Wang <wangg...@gmail.com> wrote:
>>
>> > Just to adds a bit more FYI here related to the last question from
>> David:
>> > in KIP-595 while implementing the new requests we are also adding a
>> > "KafkaNetworkChannel" which is used for brokers to send vote / fetch
>> > records, so besides the discussion on listeners I think implementation
>> wise
>> > we can also consider consolidating a lot of those into the same
>> call-trace
>> > as well -- of course this is not related to public APIs so maybe just
>> needs
>> > to be coordinated among developments:
>> >
>> > 1. Broker -> Controller: ISR Change, Topic Creation, Admin Redirect
>> > (KIP-497).
>> > 2. Controller -> Broker: LeaderAndISR / MetadataUpdate; though these are
>> > likely going to be deprecated post KIP-500.
>> > 3. Txn Coordinator -> Broker: TxnMarker
>> >
>> >
>> > Guozhang
>> >
>> > On Wed, May 6, 2020 at 8:58 PM Boyang Chen <reluctanthero...@gmail.com>
>> > wrote:
>> >
>> > > Hey David,
>> > >
>> > > thanks for the feedbacks!
>> > >
>> > > On Wed, May 6, 2020 at 2:06 AM David Jacot <dja...@confluent.io>
>> wrote:
>> > >
>> > > > Hi Boyang,
>> > > >
>> > > > While re-reading the KIP, I've got few small questions/comments:
>> > > >
>> > > > 1. When auto topic creation is enabled, brokers will send a
>> > > > CreateTopicRequest
>> > > > to the controller instead of writing to ZK directly. It means that
>> > > > creation of these
>> > > > topics are subject to be rejected with an error if a
>> CreateTopicPolicy
>> > is
>> > > > used. Today,
>> > > > it bypasses the policy entirely. I suppose that clusters allowing
>> auto
>> > > > topic creation
>> > > > don't have a policy in place so it is not a big deal. I suggest to
>> call
>> > > > out explicitly the
>> > > > limitation in the KIP though.
>> > > >
>> > > > That's a good idea, will add to the KIP.
>> > >
>> > >
>> > > > 2. In the same vein as my first point. How do you plan to handle
>> errors
>> > > > when internal
>> > > > topics are created by a broker? Do you plan to retry retryable
>> errors
>> > > > indefinitely?
>> > > >
>> > > > I checked a bit on the admin client handling of the create topic
>> RPC.
>> > It
>> > > seems that
>> > > the only retriable exceptions at the moment are NOT_CONTROLLER and
>> > > REQUEST_TIMEOUT.
>> > > So I guess we just need to retry on these exceptions?
>> > >
>> > >
>> > > > 3. Could you clarify which listener will be used for the internal
>> > > requests?
>> > > > Do you plan
>> > > > to use the control plane listener or perhaps the inter-broker
>> listener?
>> > > >
>> > > > As we discussed in the KIP, currently the internal design for
>> > > broker->controller channel has not been
>> > > done yet, and I feel it makes sense to consolidate redirect RPC and
>> > > internal topic creation RPC to this future channel,
>> > > which are details to be filled in the near future, right now some
>> > > controller refactoring effort is still WIP.
>> > >
>> > >
>> > > > Thanks,
>> > > > David
>> > > >
>> > > > On Mon, May 4, 2020 at 9:37 AM Sönke Liebau
>> > > > <soenke.lie...@opencore.com.invalid> wrote:
>> > > >
>> > > > > Ah, I see, thanks for the clarification!
>> > > > >
>> > > > > Shouldn't be an issue I think. My understanding of KIPs was always
>> > that
>> > > > > they are mostly intended as a place to discuss and agree changes
>> up
>> > > > front,
>> > > > > whereas tracking the actual releases that things go into should be
>> > > > handled
>> > > > > in Jira.
>> > > > > So maybe we just create new jiras for any subsequent work and
>> either
>> > > link
>> > > > > those or make them subtasks (even though this jira is already a
>> > subtask
>> > > > > itself), that should allow us to properly track all releases that
>> > work
>> > > > goes
>> > > > > into.
>> > > > >
>> > > > > Thanks for your work on this!!
>> > > > >
>> > > > > Best,
>> > > > > Sönke
>> > > > >
>> > > > >
>> > > > > On Sat, 2 May 2020 at 00:31, Boyang Chen <
>> reluctanthero...@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Sure thing Sonke, what I suggest is that usual KIPs get
>> accepted to
>> > > go
>> > > > > into
>> > > > > > next release. It could span for a couple of releases because of
>> > > > > engineering
>> > > > > > time, but no change has to be shipped in specific future
>> releases,
>> > > like
>> > > > > the
>> > > > > > backward incompatible change for KafkaPrincipal. But I guess
>> it's
>> > not
>> > > > > > really a blocker, as long as we stated clearly in the KIP how we
>> > are
>> > > > > going
>> > > > > > to roll things out, and let it partially finish in 2.6.
>> > > > > >
>> > > > > > Boyang
>> > > > > >
>> > > > > > On Fri, May 1, 2020 at 2:32 PM Sönke Liebau
>> > > > > > <soenke.lie...@opencore.com.invalid> wrote:
>> > > > > >
>> > > > > > > Hi Boyang,
>> > > > > > >
>> > > > > > > thanks for the update, sounds reasonable to me. Making it a
>> > > breaking
>> > > > > > change
>> > > > > > > is definitely the safer route to go.
>> > > > > > >
>> > > > > > > Just one quick question regarding your mail, I didn't fully
>> > > > understand
>> > > > > > what
>> > > > > > > you mean by "I think this is the first time we need to
>> introduce
>> > a
>> > > > KIP
>> > > > > > > without having it
>> > > > > > > fully accepted in next release." - could you perhaps explain
>> > that
>> > > > some
>> > > > > > > more very briefly?
>> > > > > > >
>> > > > > > > Best regards,
>> > > > > > > Sönke
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Fri, 1 May 2020 at 23:03, Boyang Chen <
>> > > reluctanthero...@gmail.com
>> > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hey Tom,
>> > > > > > > >
>> > > > > > > > thanks for the suggestion. As long as we could correctly
>> > > serialize
>> > > > > the
>> > > > > > > > principal and embed in the Envelope, I think we could still
>> > > > leverage
>> > > > > > the
>> > > > > > > > controller to do the client request authentication. Although
>> > this
>> > > > > pays
>> > > > > > an
>> > > > > > > > extra round trip if the authorization is doomed to fail on
>> the
>> > > > > receiver
>> > > > > > > > side, having a centralized processing unit is more favorable
>> > such
>> > > > as
>> > > > > > > > ensuring the audit log is consistent instead of scattering
>> > > between
>> > > > > > > > forwarder and receiver.
>> > > > > > > >
>> > > > > > > > Boyang
>> > > > > > > >
>> > > > > > > > On Wed, Apr 29, 2020 at 9:50 AM Tom Bentley <
>> > tbent...@redhat.com
>> > > >
>> > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi Boyang,
>> > > > > > > > >
>> > > > > > > > > Thanks for the update. In the EnvelopeRequest handling
>> > section
>> > > of
>> > > > > the
>> > > > > > > KIP
>> > > > > > > > > it might be worth saying explicitly that authorization of
>> the
>> > > > > request
>> > > > > > > > will
>> > > > > > > > > happen as normal. Otherwise what you're proposing makes
>> sense
>> > > to
>> > > > > me.
>> > > > > > > > >
>> > > > > > > > > Thanks again,
>> > > > > > > > >
>> > > > > > > > > Tom
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Wed, Apr 29, 2020 at 5:27 PM Boyang Chen <
>> > > > > > > reluctanthero...@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Thanks for the proposed idea Sonke. I reviewed it and
>> had
>> > > some
>> > > > > > > offline
>> > > > > > > > > > discussion with Colin, Rajini and Mathew.
>> > > > > > > > > >
>> > > > > > > > > > We do need to add serializability to the
>> PrincipalBuilder
>> > > > > > interface,
>> > > > > > > > but
>> > > > > > > > > we
>> > > > > > > > > > should not make any default implementation which could
>> go
>> > > wrong
>> > > > > and
>> > > > > > > > messy
>> > > > > > > > > > up with the security in a production environment if the
>> > user
>> > > > > > neglects
>> > > > > > > > it.
>> > > > > > > > > > Instead we need to make it required and backward
>> > > incompatible.
>> > > > > So I
>> > > > > > > > > > integrated your proposed methods and expand the Envelope
>> > RPC
>> > > > > with a
>> > > > > > > > > couple
>> > > > > > > > > > of more fields for audit log purpose as well.
>> > > > > > > > > >
>> > > > > > > > > > Since the KafkaPrincipal builder serializability is a
>> > binary
>> > > > > > > > incompatible
>> > > > > > > > > > change, I propose (also stated in the KIP) the following
>> > > > > > > implementation
>> > > > > > > > > > plan:
>> > > > > > > > > >
>> > > > > > > > > > 1. For next *2.x* release:
>> > > > > > > > > > 1. Get new admin client forwarding changes
>> > > > > > > > > > 2. Get the Envelope RPC implementation
>> > > > > > > > > > 3. Get the forwarding path working and validate
>> the
>> > > > > function
>> > > > > > > with
>> > > > > > > > > > fake principals in testing environment, without
>> > actual
>> > > > > > > triggering
>> > > > > > > > > in
>> > > > > > > > > > the
>> > > > > > > > > > production system
>> > > > > > > > > > 2. For next *3.0 *release:
>> > > > > > > > > > 1. Introduce serializability to PrincipalBuilder
>> > > > > > > > > > 2. Turn on forwarding path in production and
>> perform
>> > > > > > end-to-end
>> > > > > > > > > > testing
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > I think this is the first time we need to introduce a
>> KIP
>> > > > without
>> > > > > > > > having
>> > > > > > > > > it
>> > > > > > > > > > fully accepted in next release. Let me know if this
>> sounds
>> > > > > > > reasonable.
>> > > > > > > > > >
>> > > > > > > > > > On Fri, Apr 24, 2020 at 1:00 AM Sönke Liebau
>> > > > > > > > > > <soenke.lie...@opencore.com.invalid> wrote:
>> > > > > > > > > >
>> > > > > > > > > > > After thinking on this a little bit, maybe this would
>> be
>> > an
>> > > > > > option:
>> > > > > > > > > > >
>> > > > > > > > > > > add default methods serialize and deserialize to the
>> > > > > > > > > > KafkaPrincipalBuilder
>> > > > > > > > > > > interface, these could be very short:
>> > > > > > > > > > >
>> > > > > > > > > > > default String serialize(KafkaPrincipal principal) {
>> > > > > > > > > > > return principal.toString();
>> > > > > > > > > > > }
>> > > > > > > > > > >
>> > > > > > > > > > > default KafkaPrincipal deserialize(String
>> > principalString)
>> > > {
>> > > > > > > > > > > return
>> > > > SecurityUtils.parseKafkaPrincipal(principalString);
>> > > > > > > > > > > }
>> > > > > > > > > > >
>> > > > > > > > > > > This would mean that all existing implementations of
>> that
>> > > > > > interface
>> > > > > > > > > > > are unaffected, as this code is pretty much what is
>> > > currently
>> > > > > > being
>> > > > > > > > > > > used when their principals need to be serialized.
>> > > > > > > > > > >
>> > > > > > > > > > > But it offers people using custom principals the
>> chance
>> > to
>> > > > > > override
>> > > > > > > > > > > these methods and ensure that all information gets
>> > > serialized
>> > > > > for
>> > > > > > > > > > > delegation tokens or request forwarding.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > Wherever we need to de/serialize principals (for
>> example
>> > in
>> > > > the
>> > > > > > > > > > > DelegationTokenManager [1]) we obtain an instance of
>> the
>> > > > > > configured
>> > > > > > > > > > > PrincipalBuilder class and use that to do the actual
>> > work.
>> > > > > > > > > > >
>> > > > > > > > > > > What do you think?
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > Best regards,
>> > > > > > > > > > >
>> > > > > > > > > > > Sönke
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > [1]
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/kafka/blob/33d06082117d971cdcddd4f01392006b543f3c01/core/src/main/scala/kafka/server/DelegationTokenManager.scala#L122
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Thu, 23 Apr 2020 at 17:42, Boyang Chen <
>> > > > > > > > reluctanthero...@gmail.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Thanks all,
>> > > > > > > > > > > >
>> > > > > > > > > > > > IIUC, the necessity of doing the audit log on the
>> > > > controller
>> > > > > > side
>> > > > > > > > is
>> > > > > > > > > > > > because we need to make sure the authorized resource
>> > > > > > > modifications
>> > > > > > > > > > > > eventually arrive on the target broker side, but is
>> > that
>> > > > > really
>> > > > > > > > > > > necessary?
>> > > > > > > > > > > >
>> > > > > > > > > > > > I'm thinking the possibility of doing the audit log
>> on
>> > > the
>> > > > > > > > forwarding
>> > > > > > > > > > > > broker side, which could simplify the discussion of
>> > > > principal
>> > > > > > > > > > > serialization
>> > > > > > > > > > > > here. The other option I could think of is to
>> serialize
>> > > the
>> > > > > > > entire
>> > > > > > > > > > audit
>> > > > > > > > > > > > log message if we were supposed to approve, and
>> pass it
>> > > as
>> > > > > part
>> > > > > > > of
>> > > > > > > > > the
>> > > > > > > > > > > > Envelope.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Let me know if you think either of these approaches
>> > would
>> > > > > work.
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Thu, Apr 23, 2020 at 7:01 AM Sönke Liebau
>> > > > > > > > > > > > <soenke.lie...@opencore.com.invalid> wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > I agree that this would be useful to have and
>> > shouldn't
>> > > > > > create
>> > > > > > > > > issues
>> > > > > > > > > > > in
>> > > > > > > > > > > > > 99% of all cases. But it would be a breaking
>> change
>> > to
>> > > a
>> > > > > > public
>> > > > > > > > > API.
>> > > > > > > > > > > > > I had a quick look at the two large projects that
>> > come
>> > > to
>> > > > > > mind
>> > > > > > > > > which
>> > > > > > > > > > > > might
>> > > > > > > > > > > > > be affected: Ranger and Sentry - both seem to
>> operate
>> > > > > > directly
>> > > > > > > > with
>> > > > > > > > > > > > > KafkaPrincipal instead of subclassing it. But
>> anybody
>> > > who
>> > > > > > > > > > > > > extended KafkaPrincipal would probably need to
>> update
>> > > > their
>> > > > > > > > code..
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Writing this sparked the thought that this issue
>> > should
>> > > > > also
>> > > > > > > > > concern
>> > > > > > > > > > > > > delegation tokens, as Principals need to be
>> > stored/sent
>> > > > > > around
>> > > > > > > > for
>> > > > > > > > > > > those
>> > > > > > > > > > > > > too.
>> > > > > > > > > > > > > Had a brief look at the code and for Delegation
>> > Tokens
>> > > we
>> > > > > > seem
>> > > > > > > to
>> > > > > > > > > use
>> > > > > > > > > > > > > SecurityUtils#parseKafkaPrincipal[1] which would
>> > ignore
>> > > > any
>> > > > > > > > > > additional
>> > > > > > > > > > > > > information from custom Principals.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > We'll probably want to at least add a note on
>> that to
>> > > the
>> > > > > > docs
>> > > > > > > -
>> > > > > > > > > > unless
>> > > > > > > > > > > > it
>> > > > > > > > > > > > > is there already, I've only looked for about 30
>> > > seconds..
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Best regards,
>> > > > > > > > > > > > > Sönke
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > [1]
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/kafka/blob/e9fcfe4fb7b9ae2f537ce355fe2ab51a58034c64/clients/src/main/java/org/apache/kafka/common/utils/SecurityUtils.java#L52
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Thu, 23 Apr 2020 at 14:35, Colin McCabe <
>> > > > > > cmcc...@apache.org
>> > > > > > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Hmm... Maybe we need to add some way to
>> serialize
>> > and
>> > > > > > > > deserialize
>> > > > > > > > > > > > > > KafkaPrincipal subclasses to/from string. We
>> could
>> > > > add a
>> > > > > > > > method
>> > > > > > > > > to
>> > > > > > > > > > > > > > KafkaPrincipalBuilder#deserialize and a method
>> > > > > > > > > > > > KafkaPrincipal#serialize,
>> > > > > > > > > > > > > I
>> > > > > > > > > > > > > > suppose.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > best,
>> > > > > > > > > > > > > > Colin
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Thu, Apr 23, 2020, at 02:14, Tom Bentley
>> wrote:
>> > > > > > > > > > > > > > > Hi folks,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Colin wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > The final broker knows it can trust the
>> > principal
>> > > > > name
>> > > > > > in
>> > > > > > > > the
>> > > > > > > > > > > > > envelope
>> > > > > > > > > > > > > > > > (since EnvelopeRequest requires
>> CLUSTERACTION
>> > on
>> > > > > > > CLUSTER).
>> > > > > > > > > So
>> > > > > > > > > > it
>> > > > > > > > > > > > can
>> > > > > > > > > > > > > > use
>> > > > > > > > > > > > > > > > that principal name for authorization (given
>> > who
>> > > > you
>> > > > > > are,
>> > > > > > > > > what
>> > > > > > > > > > > can
>> > > > > > > > > > > > > you
>> > > > > > > > > > > > > > > > do?) The forwarded principal name will
>> also be
>> > > > used
>> > > > > > for
>> > > > > > > > > > logging.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > My understanding (and I'm happy to be
>> corrected)
>> > is
>> > > > > that
>> > > > > > a
>> > > > > > > > > custom
>> > > > > > > > > > > > > > > authoriser might rely on the KafkaPrincipal
>> > > instance
>> > > > > > being
>> > > > > > > a
>> > > > > > > > > > > subclass
>> > > > > > > > > > > > > of
>> > > > > > > > > > > > > > > KafkaPrincipal (e.g. the subclass has extra
>> > fields
>> > > > with
>> > > > > > the
>> > > > > > > > > > > > principal's
>> > > > > > > > > > > > > > > "roles"). So you can't construct a
>> KafkaPrinicpal
>> > > on
>> > > > > the
>> > > > > > > > > > controller
>> > > > > > > > > > > > > which
>> > > > > > > > > > > > > > > would be guaranteed to work for arbitrary
>> > > > authorizers.
>> > > > > > You
>> > > > > > > > have
>> > > > > > > > > > to
>> > > > > > > > > > > > > > perform
>> > > > > > > > > > > > > > > authorization on the first broker (rejecting
>> some
>> > > of
>> > > > > the
>> > > > > > > > > batched
>> > > > > > > > > > > > > > requests),
>> > > > > > > > > > > > > > > forward the authorized ones to the controller,
>> > > which
>> > > > > then
>> > > > > > > has
>> > > > > > > > > to
>> > > > > > > > > > > > trust
>> > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > the authorization has been done and make the
>> ZK
>> > > > writes
>> > > > > > > > without
>> > > > > > > > > > > > > > > authorization. Because the controller cannot
>> > invoke
>> > > > the
>> > > > > > > > > > authorizer
>> > > > > > > > > > > > that
>> > > > > > > > > > > > > > > means that the authorizer audit logging (on
>> the
>> > > > > > controller)
>> > > > > > > > > would
>> > > > > > > > > > > not
>> > > > > > > > > > > > > > > include these operations. But they would be in
>> > the
>> > > > > audit
>> > > > > > > > > logging
>> > > > > > > > > > > from
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > original broker, so the information would not
>> be
>> > > > lost.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > There's also a problem with using the
>> constructed
>> > > > > > principal
>> > > > > > > > for
>> > > > > > > > > > > other
>> > > > > > > > > > > > > > > logging (i.e. non authorizer logging) on the
>> > > > > controller:
>> > > > > > > > > There's
>> > > > > > > > > > > > > nothing
>> > > > > > > > > > > > > > > stopping a custom KafkaPrincipal subclass from
>> > > > > overriding
>> > > > > > > > > > > toString()
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > > > return something different from
>> > "${type}:${name}".
>> > > If
>> > > > > > > you're
>> > > > > > > > > > > > building a
>> > > > > > > > > > > > > > > "fake" KafkaPrincipal on the controller from
>> the
>> > > type
>> > > > > and
>> > > > > > > > name
>> > > > > > > > > > then
>> > > > > > > > > > > > its
>> > > > > > > > > > > > > > > toString() would be "wrong". A solution to
>> this
>> > > would
>> > > > > be
>> > > > > > to
>> > > > > > > > > also
>> > > > > > > > > > > > > > serialize
>> > > > > > > > > > > > > > > the toString() into the envelope and have some
>> > > > > > > > > > > ProxiedKafkaPrincipal
>> > > > > > > > > > > > > > class
>> > > > > > > > > > > > > > > which you instantiate on the controller which
>> has
>> > > > > > > overridden
>> > > > > > > > > > > toString
>> > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > return the right value. Obviously you could
>> > > optimize
>> > > > > this
>> > > > > > > > using
>> > > > > > > > > > an
>> > > > > > > > > > > > > > optional
>> > > > > > > > > > > > > > > field so you only serialize the toString() if
>> it
>> > > > > differed
>> > > > > > > > from
>> > > > > > > > > > the
>> > > > > > > > > > > > > value
>> > > > > > > > > > > > > > > you'd get from KafkaPrincipal.toString().
>> Maybe
>> > > > > non-audit
>> > > > > > > > > logging
>> > > > > > > > > > > > using
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > wrong string value of a principal is
>> sufficiently
>> > > > minor
>> > > > > > > that
>> > > > > > > > > this
>> > > > > > > > > > > > isn't
>> > > > > > > > > > > > > > > even worth trying to solve.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Kind regards,
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > Tom
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > On Wed, Apr 22, 2020 at 10:59 PM Sönke Liebau
>> > > > > > > > > > > > > > > <soenke.lie...@opencore.com.invalid> wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Hi Colin,
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > thanks for your summary! Just one question -
>> > and
>> > > I
>> > > > > may
>> > > > > > be
>> > > > > > > > > > missing
>> > > > > > > > > > > > an
>> > > > > > > > > > > > > > > > obvious point here..
>> > > > > > > > > > > > > > > > You write:
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > "The initial broker should do authentication
>> > (who
>> > > > are
>> > > > > > > you?)
>> > > > > > > > > and
>> > > > > > > > > > > > come
>> > > > > > > > > > > > > up
>> > > > > > > > > > > > > > > > with a principal name. Then it creates an
>> > > envelope
>> > > > > > > > request,
>> > > > > > > > > > > which
>> > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > > contain that principal name, and sends it
>> along
>> > > > with
>> > > > > > the
>> > > > > > > > > > > unmodified
>> > > > > > > > > > > > > > > > original request to the final broker.
>> [... ]
>> > > The
>> > > > > > final
>> > > > > > > > > broker
>> > > > > > > > > > > > knows
>> > > > > > > > > > > > > > it
>> > > > > > > > > > > > > > > > can trust the principal name in the envelope
>> > > (since
>> > > > > > > > > > > EnvelopeRequest
>> > > > > > > > > > > > > > > > requires CLUSTERACTION on CLUSTER). So it
>> can
>> > > use
>> > > > > that
>> > > > > > > > > > principal
>> > > > > > > > > > > > > name
>> > > > > > > > > > > > > > for
>> > > > > > > > > > > > > > > > authorization (given who you are, what can
>> you
>> > > > do?) "
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > My understanding is, that you don't want to
>> > > > serialize
>> > > > > > the
>> > > > > > > > > > > Principal
>> > > > > > > > > > > > > > (due to
>> > > > > > > > > > > > > > > > the discussed issues with custom principals)
>> > but
>> > > > > reduce
>> > > > > > > the
>> > > > > > > > > > > > principal
>> > > > > > > > > > > > > > down
>> > > > > > > > > > > > > > > > to a string representation that would be
>> used
>> > for
>> > > > > > logging
>> > > > > > > > and
>> > > > > > > > > > > > > > > > authorization?
>> > > > > > > > > > > > > > > > If that understanding is correct then I
>> don't
>> > > think
>> > > > > we
>> > > > > > > > could
>> > > > > > > > > > use
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > regular Authorizer on the target broker,
>> > because
>> > > > that
>> > > > > > > would
>> > > > > > > > > > need
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > actual
>> > > > > > > > > > > > > > > > principal object to work on.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Also, a thought that just occurred to me, we
>> > > might
>> > > > > > > actually
>> > > > > > > > > > need
>> > > > > > > > > > > to
>> > > > > > > > > > > > > log
>> > > > > > > > > > > > > > > > different principal strings for the case of
>> > > queries
>> > > > > > like
>> > > > > > > > > > > > AlterConfigs
>> > > > > > > > > > > > > > > > (mentioned by Rajini) which may contain
>> > multiple
>> > > > > > > resources.
>> > > > > > > > > > Take
>> > > > > > > > > > > an
>> > > > > > > > > > > > > > LDAP
>> > > > > > > > > > > > > > > > authorizer that grants access based on group
>> > > > > > membership -
>> > > > > > > > the
>> > > > > > > > > > > same
>> > > > > > > > > > > > > > > > alterconfig request may contain resources
>> that
>> > > are
>> > > > > > > > authorized
>> > > > > > > > > > > based
>> > > > > > > > > > > > > on
>> > > > > > > > > > > > > > > > group1 as well as resources authorized
>> based on
>> > > > > > > membership
>> > > > > > > > in
>> > > > > > > > > > > > group 2
>> > > > > > > > > > > > > > ..
>> > > > > > > > > > > > > > > > And in all cases we'd need to log the
>> specific
>> > > > > reason I
>> > > > > > > > > think..
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Basically I think that we might have a hard
>> > time
>> > > > > > properly
>> > > > > > > > > > > > authorizing
>> > > > > > > > > > > > > > and
>> > > > > > > > > > > > > > > > logging without being able to forward the
>> > entire
>> > > > > > > > principal..
>> > > > > > > > > > but
>> > > > > > > > > > > > > > again, I
>> > > > > > > > > > > > > > > > might be heading down an entirely wrong path
>> > here
>> > > > :)
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Best regards,
>> > > > > > > > > > > > > > > > Sönke
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > On Wed, 22 Apr 2020 at 23:13, Guozhang Wang
>> <
>> > > > > > > > > > wangg...@gmail.com>
>> > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > Colin, Boyang: thanks for the updates, I
>> > agree
>> > > > that
>> > > > > > an
>> > > > > > > > > > > > > > EnvelopeRequest
>> > > > > > > > > > > > > > > > > would be a less vulnerable approach than
>> > > optional
>> > > > > > > fields,
>> > > > > > > > > and
>> > > > > > > > > > > I'm
>> > > > > > > > > > > > > > just
>> > > > > > > > > > > > > > > > > wondering if we would keep the
>> > EnvelopeRequest
>> > > > for
>> > > > > a
>> > > > > > > long
>> > > > > > > > > > > time. I
>> > > > > > > > > > > > > was
>> > > > > > > > > > > > > > > > > thinking that, potentially if we require
>> > > clients
>> > > > to
>> > > > > > be
>> > > > > > > on
>> > > > > > > > > > newer
>> > > > > > > > > > > > > > version
>> > > > > > > > > > > > > > > > > when talking to a 3.0+ (assuming 3.0 is
>> the
>> > > > bridge
>> > > > > > > > release)
>> > > > > > > > > > > > > brokers,
>> > > > > > > > > > > > > > then
>> > > > > > > > > > > > > > > > > we do not need to keep this code for too
>> > long,
>> > > > but
>> > > > > I
>> > > > > > > > think
>> > > > > > > > > > that
>> > > > > > > > > > > > > > would be
>> > > > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > > > very hasty compatibility breaking so
>> maybe we
>> > > > > indeed
>> > > > > > > need
>> > > > > > > > > to
>> > > > > > > > > > > keep
>> > > > > > > > > > > > > > this
>> > > > > > > > > > > > > > > > > forwarding mechanism many years.
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > Regarding future use cases, I think the
>> > example
>> > > > > that
>> > > > > > > > Boyang
>> > > > > > > > > > > > > > mentioned may
>> > > > > > > > > > > > > > > > > not be very practical honestly, because
>> when
>> > > > > there's
>> > > > > > a
>> > > > > > > > > > > > connectivity
>> > > > > > > > > > > > > > > > issue,
>> > > > > > > > > > > > > > > > > it is either a network partition between
>> > > > > "controller,
>> > > > > > > A |
>> > > > > > > > > B",
>> > > > > > > > > > > or
>> > > > > > > > > > > > > > > > > "controller | A, B". In other words, if
>> the
>> > > > > > controller
>> > > > > > > > can
>> > > > > > > > > > talk
>> > > > > > > > > > > > to
>> > > > > > > > > > > > > A,
>> > > > > > > > > > > > > > > > then
>> > > > > > > > > > > > > > > > > very likely A would not be able to talk
>> to B
>> > > > > > either...
>> > > > > > > > > > anyways,
>> > > > > > > > > > > > > > since the
>> > > > > > > > > > > > > > > > > forwarding would be there for a
>> sufficiently
>> > > long
>> > > > > > > time, I
>> > > > > > > > > > think
>> > > > > > > > > > > > > > keeping
>> > > > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > additional envelope makes sense.
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > Guozhang
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > On Wed, Apr 22, 2020 at 1:47 PM Boyang
>> Chen <
>> > > > > > > > > > > > > > reluctanthero...@gmail.com>
>> > > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > Thanks Colin for the summary! And
>> Guozhang,
>> > > > > > regarding
>> > > > > > > > the
>> > > > > > > > > > > > future
>> > > > > > > > > > > > > > use
>> > > > > > > > > > > > > > > > > cases,
>> > > > > > > > > > > > > > > > > > consider a scenario where there are
>> > temporary
>> > > > > > > > > connectivity
>> > > > > > > > > > > > issue
>> > > > > > > > > > > > > > > > between
>> > > > > > > > > > > > > > > > > > controller to a fellow broker A, the
>> > > controller
>> > > > > > could
>> > > > > > > > > then
>> > > > > > > > > > > > > leverage
>> > > > > > > > > > > > > > > > > another
>> > > > > > > > > > > > > > > > > > healthy broker B to do a forwarding
>> request
>> > > to
>> > > > A
>> > > > > in
>> > > > > > > > order
>> > > > > > > > > > to
>> > > > > > > > > > > > > > maintain a
>> > > > > > > > > > > > > > > > > > communication.
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > Even for KIP-590 scope, the forwarding
>> > > > mechanism
>> > > > > > > shall
>> > > > > > > > > not
>> > > > > > > > > > be
>> > > > > > > > > > > > > > transit
>> > > > > > > > > > > > > > > > as
>> > > > > > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > > > > do need to support older version of
>> admin
>> > > > clients
>> > > > > > > for a
>> > > > > > > > > > > couple
>> > > > > > > > > > > > of
>> > > > > > > > > > > > > > years
>> > > > > > > > > > > > > > > > > > IIUC.
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > Boyang
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > On Wed, Apr 22, 2020 at 1:29 PM Colin
>> > McCabe
>> > > <
>> > > > > > > > > > > > cmcc...@apache.org
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > Hi all,
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > I guess the way I see this working is
>> > that
>> > > > the
>> > > > > > > > request
>> > > > > > > > > > gets
>> > > > > > > > > > > > > sent
>> > > > > > > > > > > > > > from
>> > > > > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > > > client, to the initial broker, and
>> then
>> > > > > forwarded
>> > > > > > > to
>> > > > > > > > > the
>> > > > > > > > > > > > final
>> > > > > > > > > > > > > > > > broker.
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > The initial broker should do
>> > authentication
>> > > > > (who
>> > > > > > > are
>> > > > > > > > > > you?)
>> > > > > > > > > > > > and
>> > > > > > > > > > > > > > come
>> > > > > > > > > > > > > > > > up
>> > > > > > > > > > > > > > > > > > > with a principal name. Then it
>> creates
>> > an
>> > > > > > envelope
>> > > > > > > > > > > request,
>> > > > > > > > > > > > > > which
>> > > > > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > > > > > contain that principal name, and
>> sends it
>> > > > along
>> > > > > > > with
>> > > > > > > > > the
>> > > > > > > > > > > > > > unmodified
>> > > > > > > > > > > > > > > > > > > original request to the final
>> broker. (I
>> > > > agree
>> > > > > > > with
>> > > > > > > > > Tom
>> > > > > > > > > > > and
>> > > > > > > > > > > > > > Rajini
>> > > > > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > > > > > can't forward the information needed
>> to
>> > do
>> > > > > > > > > authentication
>> > > > > > > > > > > on
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > final
>> > > > > > > > > > > > > > > > > > > broker, but I also think we don't need
>> > to,
>> > > > > since
>> > > > > > we
>> > > > > > > > can
>> > > > > > > > > > do
>> > > > > > > > > > > it
>> > > > > > > > > > > > > on
>> > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > > > initial broker.)
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > The final broker knows it can trust
>> the
>> > > > > principal
>> > > > > > > > name
>> > > > > > > > > in
>> > > > > > > > > > > the
>> > > > > > > > > > > > > > > > envelope
>> > > > > > > > > > > > > > > > > > > (since EnvelopeRequest requires
>> > > CLUSTERACTION
>> > > > > on
>> > > > > > > > > > CLUSTER).
>> > > > > > > > > > > > So
>> > > > > > > > > > > > > > it can
>> > > > > > > > > > > > > > > > > use
>> > > > > > > > > > > > > > > > > > > that principal name for authorization
>> > > (given
>> > > > > who
>> > > > > > > you
>> > > > > > > > > are,
>> > > > > > > > > > > > what
>> > > > > > > > > > > > > > can
>> > > > > > > > > > > > > > > > you
>> > > > > > > > > > > > > > > > > > > do?) The forwarded principal name
>> will
>> > > also
>> > > > be
>> > > > > > > used
>> > > > > > > > > for
>> > > > > > > > > > > > > logging.
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > One question is why we need an
>> > > > EnvelopeRequest.
>> > > > > > > > Well,
>> > > > > > > > > if
>> > > > > > > > > > > we
>> > > > > > > > > > > > > > don't
>> > > > > > > > > > > > > > > > have
>> > > > > > > > > > > > > > > > > > an
>> > > > > > > > > > > > > > > > > > > EnvelopeRequest, we need somewhere
>> else
>> > to
>> > > > put
>> > > > > > the
>> > > > > > > > > > > forwarded
>> > > > > > > > > > > > > > > > principal
>> > > > > > > > > > > > > > > > > > > name. I don't think overriding an
>> > existing
>> > > > > field
>> > > > > > > > (like
>> > > > > > > > > > > > > > clientId) is
>> > > > > > > > > > > > > > > > a
>> > > > > > > > > > > > > > > > > > good
>> > > > > > > > > > > > > > > > > > > option for this. It's messy, and
>> loses
>> > > > > > > information.
>> > > > > > > > > It
>> > > > > > > > > > > also
>> > > > > > > > > > > > > > raises
>> > > > > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > > > question of how the final broker knows
>> > that
>> > > > the
>> > > > > > > > > clientId
>> > > > > > > > > > in
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > received
>> > > > > > > > > > > > > > > > > > > message is not "really" a clientId,
>> but
>> > is
>> > > a
>> > > > > > > > principal
>> > > > > > > > > > > name.
>> > > > > > > > > > > > > > Without
>> > > > > > > > > > > > > > > > > an
>> > > > > > > > > > > > > > > > > > > envelope, there's no way to clearly
>> mark
>> > a
>> > > > > > request
>> > > > > > > as
>> > > > > > > > > > > > > forwarded,
>> > > > > > > > > > > > > > so
>> > > > > > > > > > > > > > > > > > there's
>> > > > > > > > > > > > > > > > > > > no reason for the final broker to
>> treat
>> > > this
>> > > > > > > > > differently
>> > > > > > > > > > > > than a
>> > > > > > > > > > > > > > > > regular
>> > > > > > > > > > > > > > > > > > > clientId (or whatever).
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > We talked about using optional fields
>> to
>> > > > > contain
>> > > > > > > the
>> > > > > > > > > > > > forwarded
>> > > > > > > > > > > > > > > > > principal
>> > > > > > > > > > > > > > > > > > > name, but this is also messy and
>> > > potentially
>> > > > > > > > dangerous.
>> > > > > > > > > > > > Older
>> > > > > > > > > > > > > > > > brokers
>> > > > > > > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > > > > > simply ignore the optional fields,
>> which
>> > > > could
>> > > > > > > result
>> > > > > > > > > in
>> > > > > > > > > > > them
>> > > > > > > > > > > > > > > > executing
>> > > > > > > > > > > > > > > > > > > operations as the wrong principal. Of
>> > > > course,
>> > > > > > this
>> > > > > > > > > would
>> > > > > > > > > > > > > > require a
>> > > > > > > > > > > > > > > > > > > misconfiguration in order to happen,
>> but
>> > it
>> > > > > still
>> > > > > > > > seems
>> > > > > > > > > > > > better
>> > > > > > > > > > > > > > to set
>> > > > > > > > > > > > > > > > > up
>> > > > > > > > > > > > > > > > > > > the system so that this
>> misconfiguration
>> > is
>> > > > > > > detected,
>> > > > > > > > > > > rather
>> > > > > > > > > > > > > than
>> > > > > > > > > > > > > > > > > > silently
>> > > > > > > > > > > > > > > > > > > ignored.
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > It's true that the need for
>> forwarding is
>> > > > > > > "temporary"
>> > > > > > > > > in
>> > > > > > > > > > > some
>> > > > > > > > > > > > > > sense,
>> > > > > > > > > > > > > > > > > > since
>> > > > > > > > > > > > > > > > > > > we only need it for older clients.
>> > > However,
>> > > > we
>> > > > > > > will
>> > > > > > > > > want
>> > > > > > > > > > > to
>> > > > > > > > > > > > > > support
>> > > > > > > > > > > > > > > > > > these
>> > > > > > > > > > > > > > > > > > > older clients for many years to come.
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > I agree that the usefulness of
>> > > > EnvelopeRequest
>> > > > > is
>> > > > > > > > > limited
>> > > > > > > > > > > by
>> > > > > > > > > > > > it
>> > > > > > > > > > > > > > > > being a
>> > > > > > > > > > > > > > > > > > > superuser-only request at the moment.
>> > > > Perhaps
>> > > > > > > there
>> > > > > > > > > are
>> > > > > > > > > > > some
>> > > > > > > > > > > > > > changes
>> > > > > > > > > > > > > > > > > to
>> > > > > > > > > > > > > > > > > > > how custom principals work that would
>> > allow
>> > > > us
>> > > > > to
>> > > > > > > get
>> > > > > > > > > > > around
>> > > > > > > > > > > > > > this in
>> > > > > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > > > future. We should think about that so
>> > that
>> > > > we
>> > > > > > have
>> > > > > > > > > this
>> > > > > > > > > > > > > > > > functionality
>> > > > > > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > > > > > the future if it's needed.
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > best,
>> > > > > > > > > > > > > > > > > > > Colin
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > On Wed, Apr 22, 2020, at 11:21,
>> Guozhang
>> > > Wang
>> > > > > > > wrote:
>> > > > > > > > > > > > > > > > > > > > Hello Gwen,
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > The purpose here is for maintaining
>> > > > > > compatibility
>> > > > > > > > old
>> > > > > > > > > > > > > clients,
>> > > > > > > > > > > > > > who
>> > > > > > > > > > > > > > > > do
>> > > > > > > > > > > > > > > > > > not
>> > > > > > > > > > > > > > > > > > > > have functionality to do re-routing
>> > admin
>> > > > > > > requests
>> > > > > > > > > > > > > themselves.
>> > > > > > > > > > > > > > New
>> > > > > > > > > > > > > > > > > > > clients
>> > > > > > > > > > > > > > > > > > > > can of course do this themselves by
>> > > > detecting
>> > > > > > > who's
>> > > > > > > > > the
>> > > > > > > > > > > > > > controller.
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > Hello Colin / Boyang,
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > Regarding the usage of the envelope,
>> > I'm
>> > > > > > curious
>> > > > > > > > what
>> > > > > > > > > > are
>> > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > potential
>> > > > > > > > > > > > > > > > > > > > future use cases that would require
>> > > request
>> > > > > > > > > forwarding
>> > > > > > > > > > > and
>> > > > > > > > > > > > > > hence
>> > > > > > > > > > > > > > > > > > envelope
>> > > > > > > > > > > > > > > > > > > > would be useful? Originally I
>> thought
>> > > that
>> > > > > the
>> > > > > > > > > > forwarding
>> > > > > > > > > > > > > > mechanism
>> > > > > > > > > > > > > > > > > is
>> > > > > > > > > > > > > > > > > > > only
>> > > > > > > > > > > > > > > > > > > > temporary as we need it for the
>> bridge
>> > > > > release,
>> > > > > > > and
>> > > > > > > > > > > moving
>> > > > > > > > > > > > > > forward
>> > > > > > > > > > > > > > > > we
>> > > > > > > > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > > > > > > get rid of this to simplify the code
>> > > base.
>> > > > If
>> > > > > > we
>> > > > > > > do
>> > > > > > > > > > have
>> > > > > > > > > > > > > valid
>> > > > > > > > > > > > > > use
>> > > > > > > > > > > > > > > > > > cases
>> > > > > > > > > > > > > > > > > > > in
>> > > > > > > > > > > > > > > > > > > > the future which makes us believe
>> that
>> > > > > request
>> > > > > > > > > > forwarding
>> > > > > > > > > > > > > would
>> > > > > > > > > > > > > > > > > > actually
>> > > > > > > > > > > > > > > > > > > be
>> > > > > > > > > > > > > > > > > > > > a permanent feature retained on the
>> > > broker
>> > > > > > side,
>> > > > > > > > I'm
>> > > > > > > > > on
>> > > > > > > > > > > > board
>> > > > > > > > > > > > > > with
>> > > > > > > > > > > > > > > > > > adding
>> > > > > > > > > > > > > > > > > > > > the envelope request protocol.
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > Guozhang
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > On Wed, Apr 22, 2020 at 8:22 AM Gwen
>> > > > Shapira
>> > > > > <
>> > > > > > > > > > > > > > g...@confluent.io>
>> > > > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > Hey Boyang,
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > Sorry if this was already
>> discussed,
>> > > but
>> > > > I
>> > > > > > > didn't
>> > > > > > > > > see
>> > > > > > > > > > > > this
>> > > > > > > > > > > > > as
>> > > > > > > > > > > > > > > > > > rejected
>> > > > > > > > > > > > > > > > > > > > > alternative:
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > Until now, we always did client
>> side
>> > > > > routing
>> > > > > > -
>> > > > > > > > the
>> > > > > > > > > > > client
>> > > > > > > > > > > > > > itself
>> > > > > > > > > > > > > > > > > > found
>> > > > > > > > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > > > > > controller via metadata and
>> directed
>> > > > > requests
>> > > > > > > > > > > > accordingly.
>> > > > > > > > > > > > > > > > Brokers
>> > > > > > > > > > > > > > > > > > that
>> > > > > > > > > > > > > > > > > > > > > were not the controller, rejected
>> > those
>> > > > > > > requests.
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > Why did we decide to move to
>> broker
>> > > side
>> > > > > > > routing?
>> > > > > > > > > Was
>> > > > > > > > > > > the
>> > > > > > > > > > > > > > > > > client-side
>> > > > > > > > > > > > > > > > > > > > > option discussed and rejected
>> > somewhere
>> > > > > and I
>> > > > > > > > > missed
>> > > > > > > > > > > it?
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > Gwen
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > On Fri, Apr 3, 2020, 4:45 PM
>> Boyang
>> > > Chen
>> > > > <
>> > > > > > > > > > > > > > > > > reluctanthero...@gmail.com
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > > Hey all,
>> > > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > > I would like to start off the
>> > > > discussion
>> > > > > > for
>> > > > > > > > > > > KIP-590, a
>> > > > > > > > > > > > > > > > follow-up
>> > > > > > > > > > > > > > > > > > > > > > initiative after KIP-500:
>> > > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-590%3A+Redirect+Zookeeper+Mutation+Protocols+to+The+Controller
>> > > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > > This KIP proposes to migrate
>> > existing
>> > > > > > > Zookeeper
>> > > > > > > > > > > > mutation
>> > > > > > > > > > > > > > paths,
>> > > > > > > > > > > > > > > > > > > including
>> > > > > > > > > > > > > > > > > > > > > > configuration, security and
>> quota
>> > > > > changes,
>> > > > > > to
>> > > > > > > > > > > > > > controller-only
>> > > > > > > > > > > > > > > > by
>> > > > > > > > > > > > > > > > > > > always
>> > > > > > > > > > > > > > > > > > > > > > routing these alterations to the
>> > > > > > controller.
>> > > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > > Let me know your thoughts!
>> > > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > > > Best,
>> > > > > > > > > > > > > > > > > > > > > > Boyang
>> > > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > > > > > > > -- Guozhang
>> > > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > > > > -- Guozhang
>> > > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > > > Sönke Liebau
>> > > > > > > > > > > > > > > > Partner
>> > > > > > > > > > > > > > > > Tel. +49 179 7940878
>> > > > > > > > > > > > > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße
>> 8 -
>> > > > 22880
>> > > > > > > > Wedel -
>> > > > > > > > > > > > Germany
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > --
>> > > > > > > > > > > > > Sönke Liebau
>> > > > > > > > > > > > > Partner
>> > > > > > > > > > > > > Tel. +49 179 7940878
>> > > > > > > > > > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 -
>> 22880
>> > > > > Wedel -
>> > > > > > > > > Germany
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > --
>> > > > > > > > > > > Sönke Liebau
>> > > > > > > > > > > Partner
>> > > > > > > > > > > Tel. +49 179 7940878
>> > > > > > > > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880
>> > > Wedel -
>> > > > > > > Germany
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Sönke Liebau
>> > > > > > > Partner
>> > > > > > > Tel. +49 179 7940878
>> > > > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel -
>> > > Germany
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Sönke Liebau
>> > > > > Partner
>> > > > > Tel. +49 179 7940878
>> > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel -
>> Germany
>> > > > >
>> > > >
>> > >
>> >
>> >
>> > --
>> > -- Guozhang
>> >
>>
>
