Jason: Thanks for your reply.
I did turn on logging for kafka as well when I ran the test. I do not see any partition assignment for the topic happening. If you don’t see anything obviously wrong in what I am doing, I can file a ticket with Confluent and attach the logs to it. Regards Venkat On 2/19/16, 11:22 AM, "Jason Gustafson" <ja...@confluent.io> wrote: >Hi Venkatesan, > >Autocreation of topics happens when the broker receives a topic metadata >request. That should mean that both topics get created when the consumer >does the initial poll() since that is the first time that topic metadata >would be fetched (fetching topic metadata allows the consumer and producer >to determine the number of partitions in the topic and the current >partition leaders). Are you sure that partitions from both topics have not >been assigned at that time? > >Thanks, >Jason > >On Fri, Feb 19, 2016 at 10:43 AM, Venkatesan Subramanian < >vsubraman...@lendingclub.com> wrote: > >> I am running into a curious problem with auto creation of topics and would >> appreciate any insight into what could be causing it. >> >> Some context first: We are using Confluent 2.0 platform and I have an >> integration test where the following happens: >> >> >> 1. A worker thread is spun up and it creates a kafka consumer that >> subscribes to two topics: Topic A and Topic B. Both of these topics do not >> exist at this time. >> 2. The kafka consumer polls the two topics periodically. >> 3. A kafka producer sends an event into Topic A (and hence creates the >> topic through auto topic creation) >> 4. The kafka consumer created previously is assigned a partition from >> Topic A (no assignment for Topic B since it does not exist yet). >> 5. worker polls and receives the event from Topic A partition. It >> processes the event and produces an event into Topic B (and hence the topic >> B is created at this time) >> 6. I am expected to see a partition from Topic B getting assigned to >> the consumer next time a poll happens but nothing happens. >> >> Note that the same integration test works fine if I pre-create the two >> topics before running the test. >> >> ________________________________ >> DISCLAIMER: The information transmitted is intended only for the person or >> entity to which it is addressed and may contain confidential and/or >> privileged material. Any review, re-transmission, dissemination or other >> use of, or taking of any action in reliance upon this information by >> persons or entities other than the intended recipient is prohibited. If you >> received this in error, please contact the sender and destroy any copies of >> this document and any attachments. >>
