I didn't test that. All other software is compatible with Kafka 0.11 and we can not change it easily.
On Mon, Jul 30, 2018 at 5:07 PM, M. Manna <manme...@gmail.com> wrote: > The release came out earlier today 2.0.0 - Does this not work for you if > you upgrade to that? > > On 30 July 2018 at 13:02, Soheil Pourbafrani <soheil.i...@gmail.com> > wrote: > > > In addition, I should say this behavior is for when topics are created > > automatically. > > > > On Mon, Jul 30, 2018 at 2:01 PM, Soheil Pourbafrani < > soheil.i...@gmail.com > > > > > wrote: > > > > > I have a 3 node Kafka cluster. Upgrading Kafka version from 0.11.0.2 to > > > 0.11.0.3, I notice that Kafka sometimes truncate new message's topic. > > Here > > > is the log: > > > > > > [2018-07-30 09:19:39,612] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in 7798-7. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,612] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-22. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,612] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-19. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,612] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in 7798-4. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,613] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in 7798-1. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,613] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-16. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,613] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-13. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,613] WARN [ReplicaFetcherThread-0-101]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-10. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,615] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-23. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,627] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in 7798-8. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,627] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in 7798-5. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,627] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-20. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,627] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-17. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,627] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in 7798-2. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,627] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-14. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,627] WARN [ReplicaFetcherThread-0-102]: Based on > > follower's leader epoch, leader replied with an unknown offset in > 7798-11. > > High watermark 0 will be used for truncation. (kafka.server. > > ReplicaFetcherThread) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-20 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-17 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-14 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-11 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-8 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-2 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-5 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,628] INFO Truncating 7798-23 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,613] INFO Truncating 7798-1 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,637] INFO Truncating 7798-22 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,637] INFO Truncating 7798-16 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,637] INFO Truncating 7798-19 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,637] INFO Truncating 7798-13 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,637] INFO Truncating 7798-10 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,637] INFO Truncating 7798-7 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,637] INFO Truncating 7798-4 to 0 has no effect as > > the largest offset in the log is -1. (kafka.log.Log) > > > [2018-07-30 09:19:39,708] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,8] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,708] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,2] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,708] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,23] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,708] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,17] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,708] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,5] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,708] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,14] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,708] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,11] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,709] ERROR [ReplicaFetcherThread-0-102]: Error for > > partition [7798,20] to broker 102:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,714] ERROR [ReplicaFetcherThread-0-101]: Error for > > partition [7798,7] to broker 101:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,714] ERROR [ReplicaFetcherThread-0-101]: Error for > > partition [7798,13] to broker 101:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,714] ERROR [ReplicaFetcherThread-0-101]: Error for > > partition [7798,22] to broker 101:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,715] ERROR [ReplicaFetcherThread-0-101]: Error for > > partition [7798,4] to broker 101:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,715] ERROR [ReplicaFetcherThread-0-101]: Error for > > partition [7798,10] to broker 101:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,715] ERROR [ReplicaFetcherThread-0-101]: Error for > > partition [7798,19] to broker 101:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > [2018-07-30 09:19:39,715] ERROR [ReplicaFetcherThread-0-101]: Error for > > partition [7798,1] to broker 101:org.apache.kafka.common.errors. > > UnknownTopicOrPartitionException: This server does not host this > > topic-partition. (kafka.server.ReplicaFetcherThread) > > > > > > t seems at the first Kafka save the message and after that, it started > to > > > truncate messages, right away! I guess it's because of resolving this > bug > > > <https://issues.apache.org/jira/browse/KAFKA-6857> in Kafka version > > > 11.0.3. This scenario happened even for __consumer_offset topic. Is > there > > > any way to solve the problem? > > > > > > > > > > > >
