Simple.. Topic auto creation ON. As soon as it encounters producer and consumers request for some topic then it creates that topic automatically.
This is very common question and problem people come across, if they are new. Same happened with me.😄 On Fri, 22 May 2020 at 3:28 PM, Jiamei Xie <jiamei....@arm.com> wrote: > Hi Manna, Luke > I used the default configure files without any modification. The default > value for allow.auto.create.topics is true according to > https://kafka.apache.org/documentation/#brokerconfigs. Thanks for your > reply. > > -----Original Message----- > From: M. Manna <manme...@gmail.com> > Sent: 2020年5月22日 16:50 > To: users@kafka.apache.org > Subject: Re: Is that a bug? > > Hey Xie, > > On Fri, 22 May 2020 at 08:31, Jiamei Xie <jiamei....@arm.com> wrote: > > > Hi > > > > Kill all zookeeper and kafka process. Clear zookeeper and kafka data dir. > > Restart zookeeper and kafka. If there are any active client. Topic > > used by client will be auto-created. > > > > How to reproduce? > > > > > > 1. Start zookeeper and kafka zookeeper and kafka config file. > > > > nohup bin/zookeeper-server-start.sh config/zookeeper.properties & > > > > nohup bin/kafka-server-start.sh config/server.properties & > > > > > > > > 1. Create topic test with 2 partitions > > > > bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic test > > --partitions 2 --replication-factor 1 > > > > > > > > 1. Produce some data to topic test > > > > bin/kafka-producer-perf-test.sh --topic test --num-records 50000000 > > --record-size 100 --throughput=-1 --producer-props > > bootstrap.servers=localhost:9092 > > > > > > > > 1. Kill zookeeper and kafka. ProducerPerformance is still running. > > > > jps > > > > 21072 QuorumPeerMain > > > > 21704 ProducerPerformance > > > > 21230 Kafka > > > > 21854 Jps > > > > kill -9 21072 21230 > > > > > > > > 1. Remove Zookeeper and Kafka data > > > > rm -rf /tmp/zookeeper/ > > > > rm -rf /tmp/kafka-logs/ > > > > > > > > 1. Start zookeeper and kafka > > > > nohup bin/zookeeper-server-start.sh config/zookeeper.properties & > > > > nohup bin/kafka-server-start.sh config/server.properties & > > > > > > > > 1. Check topic and you'll see there is topic named test with > > partition 1. And the ProducerPerformance process continues to run > normally. > > > > bin/kafka-topics.sh --describe --zookeeper localhost:2181 > > > > Topic: test PartitionCount: 1 ReplicationFactor: 1 Configs: > > > > Topic: test Partition: 0 Leader: 0 Replicas: 0 > > Isr: 0 > > > > > > Some output of ProducerPerformance process. > > 1995632 records sent, 399126.4 records/sec (38.06 MB/sec), 378.6 ms > > avg latency, 435.0 ms max latency. > > org.apache.kafka.common.errors.TimeoutException: Expiring 148 > > record(s) for test-1:120000 ms has passed since batch creation > > ............................. > > org.apache.kafka.common.errors.TimeoutException: Expiring 148 > > record(s) for test-1:121774 ms has passed since batch creation > > 1711254 records sent, 342250.8 records/sec (32.64 MB/sec), 2324.5 ms > > avg latency, 123473.0 ms max latency. > > > > > > > > Is that a bug? > > > How have you configured your server.properties and consumer.peoperties? > Have you set auto creation ON? > > Regards, > > > > > > > Best Wishes, > > Jiamei > > > > IMPORTANT NOTICE: The contents of this email and any attachments are > > confidential and may also be privileged. If you are not the intended > > recipient, please notify the sender immediately and do not disclose > > the contents to any other person, use it for any purpose, or store or > > copy the information in any medium. Thank you. > > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. >
