Hi Stevo, I am still not very clear on your point yet, I guess I was trying to figure out under which circumstances would users prefer to re-set the group id at an existing consumer rather than creating a new instance. As Jason mentioned, since the new consumer is single threaded it should usually be cheap to construct.
Guozhang On Mon, Jul 20, 2015 at 11:06 AM, Jason Gustafson <[email protected]> wrote: > Hey Stevo, > > The new consumer doesn't have any threads of its own, so I think > construction should be fairly cheap. > > -Jason > > On Sun, Jul 19, 2015 at 2:13 PM, Stevo Slavić <[email protected]> wrote: > > > Hello Guozhang, > > > > It would be enough if consumer group could, besides at construction time, > > be set once only after construction. Have to retest, but high level > > consumer in 0.8.1 used to be very heavy weight object (lots of threads > > started, and it would block and take time to construct it). It's > > understandable, considering all of the high level features it has, and > > since it's supposed to be long living object. What would improve with > this > > change is that construction penalty could be paid upfront, while topic > > subscription and joining consumer group ensemble could be done on first > > use, so that first use does not have to suffer from both init and > > subscription penalties. > > > > It would be nice also if consumer group just as subscription could be > > changed later even, so multiple times throughout lifetime of high level > > consumer instance, to avoid constructing new consumer when instance > purpose > > changes. > > > > After looking more into the HLC API, thought maybe this is not needed, > > since there is "public void subscribe(TopicPartition... partitions)" > which > > does not use consumer group management, but problem is that there is no > > matching explicit commit where one could pass consumer group parameter as > > well, to label for which consumer group should offset(s) be committed. > > > > Seems like new HLC has split personality. Maybe (at least) two APIs could > > have been provided instead of one with such differing behaviors. > > > > Kind regards, > > Stevo Slavic. > > > > On Sun, Jul 19, 2015 at 12:01 AM, Guozhang Wang <[email protected]> > > wrote: > > > > > Hi Stevo, > > > > > > Hmm this is interesting, do you have any use cases in mind that need > > > dynamic group changing? > > > > > > Guozhang > > > > > > On Fri, Jul 17, 2015 at 11:13 PM, Stevo Slavić <[email protected]> > > wrote: > > > > > > > Hello Apache Kafka community, > > > > > > > > In new KafkaConsumer API on trunk, it seems it's only possible to > > define > > > > consumer group id at construction time of KafkaConsumer, through > > property > > > > with group.id key. > > > > > > > > Would it make sense and be possible to support setting/changing > > consumer > > > > group id after construction, but before it's actually used for the > > first > > > > time, similar to how subscription is supported through "public void > > > > subscribe(String... topics)"? > > > > > > > > Maybe this can be done through additional method "public void > > > > subscribe(String consumerGroupId, String... topics)" which would > first > > > set > > > > provided consumer group id in coordinator and then call "public void > > > > subscribe(String... topics)". > > > > > > > > Kind regards, > > > > Stevo Slavic. > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > -- -- Guozhang
