Thanks Arjun. I've updated the KIP using your suggestion - just a few
slight changes.
On Fri, May 3, 2019 at 4:48 PM Arjun Satish <arjun.sat...@gmail.com> wrote:
> Maybe we can say something like:
>
> This change can have an indirect impact on resource usage by a Connector.
> For example, systems that were enforcing quotas using a "consumer-[id]"
> client id will now have to update their configs to enforce quota on
> "connector-consumer-[id]". For systems that were not enforcing any
> limitations or using default quotas, there should be no change expected.
>
> Best,
>
> On Fri, May 3, 2019 at 1:38 PM Paul Davidson
> <pdavid...@salesforce.com.invalid> wrote:
>
> > Thanks Arjun. I updated the KIP to mention the impact on quotas. Please
> let
> > me know if you think I need more detail. The paragraph I added was:
> >
> > Since the default client.id values are changing, this will also affect
> any
> > > user that has quotas defined against the current defaults. The current
> > > default client.id values are of the form: consumer-{count} and
> > > producer-{count}.
> >
> >
> > Thanks,
> >
> > Paul
> >
> > On Thu, May 2, 2019 at 5:36 PM Arjun Satish <arjun.sat...@gmail.com>
> > wrote:
> >
> > > Paul,
> > >
> > > You might want to make a note on the KIP regarding the impact on
> quotas.
> > >
> > > Thanks,
> > >
> > > On Thu, May 2, 2019 at 9:48 AM Paul Davidson
> > > <pdavid...@salesforce.com.invalid> wrote:
> > >
> > > > Thanks for the votes everyone! KIP-411 is now accepted with:
> > > >
> > > > +3 binding votes (Randall, Jason, Gwen) , and
> > > > +3 non-binding votes (Ryanne, Arjun, Magesh)
> > > >
> > > > Regards,
> > > >
> > > > Paul
> > > >
> > > > On Wed, May 1, 2019 at 10:07 PM Arjun Satish <arjun.sat...@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Good point, Gwen. We always set a non empty value for client id:
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/kafka/blob/2.2.0/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L668
> > > > > .
> > > > >
> > > > > But more importantly, connect client ids (for consumers, for
> example)
> > > > were
> > > > > already of the form "consumer-[0-9]+", and from now on they will be
> > > > > "connector-consumer-[connector_name]-[0-9]+". So, at least for
> > connect
> > > > > consumers/producers, we would have already been hitting the default
> > > quota
> > > > > limits and nothing changes for them. You can correct me if I'm
> > missing
> > > > > something, but seems like this doesn't *break* backward
> > compatibility?
> > > > >
> > > > > I suppose this change only gives us a better way to manage that
> quota
> > > > > limit.
> > > > >
> > > > > Best,
> > > > >
> > > > > On Wed, May 1, 2019 at 9:16 PM Gwen Shapira <g...@confluent.io>
> > wrote:
> > > > >
> > > > > > I'm confused. Surely the default quota applies on empty client
> IDs
> > > too?
> > > > > > otherwise it will be very difficult to enforce?
> > > > > > So setting the client name will only change something if there's
> > > > already
> > > > > a
> > > > > > quota for that client?
> > > > > >
> > > > > > On the other hand, I fully support switching to
> "easy-to-wildcard"
> > > > > template
> > > > > > for the client id.
> > > > > >
> > > > > > On Wed, May 1, 2019 at 8:50 PM Arjun Satish <
> > arjun.sat...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I just realized that setting the client.id on the will now
> > trigger
> > > > any
> > > > > > > quota restrictions (
> > > > > > > https://kafka.apache.org/documentation/#design_quotasconfig)
> on
> > > the
> > > > > > > broker.
> > > > > > > It seems like this PR will enforce quota policies that will
> > either
> > > > > > require
> > > > > > > admins to set limits for each task (since the chosen format is
> > > > > > > connector-*-id), or fallback to some default value.
> > > > > > >
> > > > > > > Maybe we should mention this in the backward compatibility
> > section
> > > > for
> > > > > > the
> > > > > > > KIP. At the same time, since there is no way atm to turn off
> this
> > > > > > feature,
> > > > > > > should this feature be merged and released in the upcoming
> v2.3?
> > > This
> > > > > is
> > > > > > > something the committers can comment better.
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > >
> > > > > > > On Wed, May 1, 2019 at 5:13 PM Gwen Shapira <g...@confluent.io
> >
> > > > wrote:
> > > > > > >
> > > > > > > > hell yeah!
> > > > > > > > +1
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Apr 5, 2019 at 9:08 AM Paul Davidson
> > > > > > > > <pdavid...@salesforce.com.invalid> wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Since we seem to have agreement in the discussion I would
> > like
> > > to
> > > > > > start
> > > > > > > > the
> > > > > > > > > vote on KIP-411.
> > > > > > > > >
> > > > > > > > > See:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-411%3A+Make+default+Kafka+Connect+worker+task+client+IDs+distinct
> > > > > > > > >
> > > > > > > > > Also see the related PR:
> > > > https://github.com/apache/kafka/pull/6097
> > > > > > > > >
> > > > > > > > > Thanks to everyone who contributed!
> > > > > > > > >
> > > > > > > > > Paul
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > *Gwen Shapira*
> > > > > > > > Product Manager | Confluent
> > > > > > > > 650.450.2760 | @gwenshap
> > > > > > > > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > > > > > > > <http://www.confluent.io/blog>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *Gwen Shapira*
> > > > > > Product Manager | Confluent
> > > > > > 650.450.2760 | @gwenshap
> > > > > > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > > > > > <http://www.confluent.io/blog>
> > > > > >
> > > > >
> > > >
> > >
> >
>
