Hi Adam, Assuming you have a cluster where access to any groups is denied by default, you can allow a principal to only have access to one group.
Something like the following: kafka-acls.sh --add --allow-principal User:CN=my-principal --operation Read --topic 'my-topic' --group 'my-group' With this, *my-principal* will only be able to join the consumer group *my-group*. If the principal uses another group, such as *my-group2*, it will get: org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: my-group2 I hope it helps. Best, David On Thu, Aug 22, 2019 at 4:36 PM Adam Bellemare <adam.bellem...@gmail.com> wrote: > I see the consumer group much like the username in relational database > access credentials. We routinely use the consumer group name as the means > of identifying the consumer for operational things, such as alerting based > on consumer group lag, autoscaling based on lag, and tooling around > manipulating the consumer group offset for a particular service. The > consumer group allows us to know, operationally, which offsets we are > observing or manipulating, and ideally we would like to limit the set of > consumer groups possible. > > In practice, I regularly see people simply append an incrementing integer > to the end of their consumer group (cg, cg1, cg2, cg1234) when they want to > reset their application, INSTEAD of following offset reset or kafka-streams > application reset procedures. Sure, it would be nice to get everyone to > follow recommended procedures, but people do what they CAN do, not what > they're supposed to do. This has significant impact on the brokers and > surrounding tooling (orphaned internal topics with indefinite retention, > falsely-firing lag monitoring, broken auto-scaling), and instead of us > building out layers of supportive tooling to isolate ourselves from it, why > not simply set up ACLs to enforce which consumer groups an application can > and cannot use? > > Does this need a KIP? Or is a bug report simply enough? I am unsure how > compatibility would work, so I am leaning towards a KIP... > > Thanks > Adam > > On Wed, Aug 21, 2019 at 5:59 PM Colin McCabe <cmcc...@apache.org> wrote: > > > I think it's worth considering separating out the permissions needed to > > create a consumer group from the permissions needed to join one. We > > distinguish these permissions for topics, and people generally find it > > useful. We could start checking CREATE on GROUP, perhaps? It might be > > hard to do in a compatible way. > > > > cheers, > > Colin > > > > > > On Wed, Aug 21, 2019, at 12:05, Adam Bellemare wrote: > > > +users mailing list > > > > > > David, > > > > > > I don't think I really understand your email. Are you saying that this > > can > > > already be achieved only using the READ ACL? > > > > > > Thanks > > > Adam > > > > > > > > > > > > On Wed, Aug 21, 2019 at 3:58 AM David Jacot <dja...@confluent.io> > wrote: > > > > > > > Hello, > > > > > > > > It would be better to ask such question on the user mailing list. > > > > > > > > The reason is that the group is created automatically when a consumer > > > > joins it. It is not created explicitly so it can be restricted. > > > > > > > > In your case, you could setup a ACL to authorize the application to > > only > > > > use the group you have defined. It would prevent the application from > > > > creating new groups. (READ Acl on Group resource with a specific > name). > > > > > > > > Best, > > > > David > > > > > > > > On Mon, Aug 19, 2019 at 9:01 PM Adam Bellemare < > > adam.bellem...@gmail.com> > > > > wrote: > > > > > > > > > Hi All > > > > > > > > > > I am looking through the Confluent docs and core Kafka docs and > > don't see > > > > > an ACL for group creation: > > > > > > > https://docs.confluent.io/current/kafka/authorization.html#acl-format > > > > > and > > > > > https://kafka.apache.org/documentation/#security_authz > > > > > > > > > > My scenario is simple: We use the consumer group as the means of > > > > > identifying a single application, including tooling for managing > > > > > application resets, offset management, lag monitoring, etc. We > often > > have > > > > > situations where someone resets their consumer group by appending > an > > > > > incremented integer ("cg" to "cg1"), but it throws the rest of the > > > > > monitoring and management tooling out of whack. > > > > > > > > > > Is there a reason why we do not have ACL-based CREATE restrictions > > to a > > > > > particular consumer group? I am willing to do the work to implement > > this > > > > > and test it out, but I wanted to validate that there isn't a reason > > I am > > > > > missing. > > > > > > > > > > Thanks > > > > > Adam > > > > > > > > > > > > > > >
