[
https://issues.apache.org/jira/browse/KAFKA-8375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Markus Dybeck updated KAFKA-8375:
---------------------------------
Attachment: partition_lag_metrics.png
> Offset jumps back after commit
> ------------------------------
>
> Key: KAFKA-8375
> URL: https://issues.apache.org/jira/browse/KAFKA-8375
> Project: Kafka
> Issue Type: Bug
> Components: offset manager
> Affects Versions: 1.1.1
> Reporter: Markus Dybeck
> Priority: Major
> Attachments: partition_lag_metrics.png
>
>
> *Setup*
> Kafka: 1.1.1
> Kafka-client: 1.1.1
> Zookeeper: 3.4.11
> Akka streams: 0.20
> *Topic config*
> DELETE_RETENTION_MS_CONFIG: "5000"
> CLEANUP_POLICY_CONFIG: "compact,delete"
> RETENTION_BYTES_CONFIG: 200000000000L
> RETENTION_MS_CONFIG: 36000000
> *Behavior*
> We have 7 Consumers consuming from 7 partitions, and some of the consumers
> lag jumped back a bit randomly. No new messages were pushed to the topic
> during the time. We didn't see any strange logs during the time, and the
> brokers did not restart either.
> Either way, if there would be a restart or rebalance going on, we can not
> understand why the offset would jump back after it was committed?
> We did observe it both with logs and by watching metrics of the lag. Our logs
> pointed out that after we committed the offset, around 30-35 seconds later we
> consumed an earlier committed message and then the loop begun. The behavior
> was the same after a restart of all the consumers. The behavior then stopped
> after a while all by itself.
> We have no clue going forward, or if these might be an issue with akka. But
> is there any known issue that might cause this?
> Attaching a screendump with metrics that shows the lag for one partition.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
