Sorry sent several times, ignore please this thread.
On 2020/04/16 15:49:16, Ilya R <mrgreen...@gmail.com> wrote: > Hello, everyone > > I'm researching problem when reading a topic from the beginning max speed per > partition/per consumer not more than 80MB/s. But near the end of the topic > speed rise up to 250-300MB/s unexpectedly, not only on the last log segment > on several latest log segments. I've already change network buffers in a > linux, change socket settings on server/client side, but there are not > results. For testing I'm using kafka-console-consimer.sh script from standart > installation. Kafka version is 2.3.0. > > Does anyone else encountered this? > > Server settings > broker.id=1 > log.message.timestamp.type=LogAppendTime > num.network.threads=24 > num.io.threads=10 > offset.metadata.max.bytes=8096 > offsets.load.buffer.size=10485760 > offsets.topic.num.partitions=1 > offsets.topic.replication.factor=1 > socket.request.max.bytes=4194304 > socket.send.buffer.bytes=4194304 > leader.imbalance.check.interval.seconds = 30 > log.dir=/local/kafka/data > log.dirs=/local/kafka/data > zookeeper.connect=localhost:2181 > message.max.bytes = 50000000 > log.segment.bytes = 104857600 > > Topic settings: > compression.type uncompressed > segment.bytes 104857600 > max.message.bytes 50000000 > min.compaction.lag.ms 43200000 > message.timestamp.type LogAppendTime > min.cleanable.dirty.ratio 0.1 > > Consumer settings: > max.partition.fetch.bytes=8048576 > fetch.max.bytes=128777216 > fetch.min.bytes=8048576 > receive.buffer.bytes=8048577 >
