Hi,
I have a single node Kafka broker and 3-node Zookeeper cluster.
When I create a new topic with 24 partitions using the command:
kafka-topics.sh --zookeeper zoo3:2181 --create --topic test_topic
--replication-factor 1 --partitions 24
It results in 481M data in the log-dir!? Actually, every partition has
about 21M data containing the files:
00000000000000000000.index 00000000000000000000.log
00000000000000000000.timeindex leader-epoch-checkpoint
I don't remember such behavior from Kafka! Is it normal that each partition
has 21M data while no message is produced yet?
There is no error in Kafka server logs but it produces the following logs
for each partition:
[2019-02-05 10:10:54,957] INFO [Log partition=test_topic-14, dir=/data/esb]
Loading producer state till offset 0 with message format version 2
(kafka.log.Log)
[2019-02-05 10:10:54,957] INFO [Log partition=test_topic-14, dir=/data/esb]
Completed load of log with 1 segments, log start offset 0 and log end
offset 0 in 1 ms (kafka.log.Log)
[2019-02-05 10:10:54,958] INFO Created log for partition test_topic-14 in
/data/esb with properties {compression.type -> producer,
message.format.version -> 2.1-IV2, file.delete.delay.ms -> 60000,
max.message.bytes -> 1000012, min.compaction.lag.ms -> 0,
message.timestamp.type -> CreateTime, message.downconversion.enable ->
true, min.insync.replicas -> 1, segment.jitter.ms -> 0, preallocate ->
false, min.cleanable.dirty.ratio -> 0.5, index.interval.bytes -> 4096,
unclean.leader.election.enable -> false, retention.bytes -> -1,
delete.retention.ms -> 86400000, cleanup.policy -> [delete], flush.ms ->
9223372036854775807, segment.ms -> 604800000, segment.bytes -> 1073741824,
retention.ms -> 604800000, message.timestamp.difference.max.ms ->
9223372036854775807, segment.index.bytes -> 10485760, flush.messages ->
9223372036854775807}. (kafka.log.LogManager)
[2019-02-05 10:10:54,958] INFO [Partition test_topic-14 broker=0] No
checkpointed highwatermark is found for partition test_topic-14
(kafka.cluster.Partition)
[2019-02-05 10:10:54,958] INFO Replica loaded for partition test_topic-14
with initial high watermark 0 (kafka.cluster.Replica)
[2019-02-05 10:10:54,958] INFO [Partition test_topic-14 broker=0]
test_topic-14 starts at Leader Epoch 0 from offset 0. Previous Leader Epoch
was: -1 (kafka.cluster.Partition)
