[
https://issues.apache.org/jira/browse/KAFKA-9156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635367#comment-17635367
]
Bob Tiernay commented on KAFKA-9156:
------------------------------------
We too are hitting this on AWS MSK Kafka version 2.8.1:
{code:java}
ERROR Uncaught exception in scheduled task 'kafka-log-retention'
(kafka.utils.KafkaScheduler)
java.nio.BufferOverflowException
at java.base/java.nio.Buffer.nextPutIndex(Buffer.java:674)
at java.base/java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:882)
at kafka.log.TimeIndex.$anonfun$maybeAppend$1(TimeIndex.scala:134)
at kafka.log.TimeIndex.maybeAppend(TimeIndex.scala:114)
at kafka.log.LogSegment.onBecomeInactiveSegment(LogSegment.scala:506)
at kafka.log.Log.$anonfun$roll$8(Log.scala:2066)
at kafka.log.Log.$anonfun$roll$8$adapted(Log.scala:2066)
at scala.Option.foreach(Option.scala:437)
at kafka.log.Log.$anonfun$roll$2(Log.scala:2066)
at kafka.log.Log.roll(Log.scala:2482)
at kafka.log.Log.$anonfun$deleteSegments$2(Log.scala:1859)
at kafka.log.Log.deleteSegments(Log.scala:2482)
at kafka.log.Log.deleteRetentionMsBreachedSegments(Log.scala:1847)
at kafka.log.Log.deleteOldSegments(Log.scala:1916)
at kafka.log.LogManager.$anonfun$cleanupLogs$3(LogManager.scala:1092)
at
kafka.log.LogManager.$anonfun$cleanupLogs$3$adapted(LogManager.scala:1089)
at scala.collection.immutable.List.foreach(List.scala:333)
at kafka.log.LogManager.cleanupLogs(LogManager.scala:1089)
at
kafka.log.LogManager.$anonfun$startupWithConfigOverrides$2(LogManager.scala:429)
at kafka.utils.KafkaScheduler.$anonfun$schedule$2(KafkaScheduler.scala:114)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829) {code}
when this happens, disk starts to fill up and exhaust until a restart:
!image-2022-11-17-09-02-20-774.png!
> LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow in concurrent
> state
> -------------------------------------------------------------------------------
>
> Key: KAFKA-9156
> URL: https://issues.apache.org/jira/browse/KAFKA-9156
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.3.0, 2.3.1
> Reporter: shilin Lu
> Assignee: Alex Mironov
> Priority: Blocker
> Labels: regression
> Fix For: 2.4.0, 2.3.2
>
> Attachments: image-2019-11-07-17-42-13-852.png,
> image-2019-11-07-17-44-05-357.png, image-2019-11-07-17-46-53-650.png,
> image-2022-11-17-09-02-20-774.png
>
>
> !image-2019-11-07-17-42-13-852.png!
> this timeindex get function is not thread safe ,may cause create some
> timeindex.
> !image-2019-11-07-17-44-05-357.png!
> When create timeindex not exactly one ,may cause mappedbytebuffer position to
> end. Then write index entry to this mmap file will cause
> java.nio.BufferOverflowException.
>
> !image-2019-11-07-17-46-53-650.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
