[
https://issues.apache.org/jira/browse/KAFKA-15910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Jacot resolved KAFKA-15910.
---------------------------------
Fix Version/s: 3.7.0
Resolution: Fixed
> New group coordinator needs to generate snapshots while loading
> ---------------------------------------------------------------
>
> Key: KAFKA-15910
> URL: https://issues.apache.org/jira/browse/KAFKA-15910
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Jeff Kim
> Assignee: Jeff Kim
> Priority: Major
> Fix For: 3.7.0
>
>
> After the new coordinator loads a __consumer_offsets partition, it logs the
> following exception when making a read operation (fetch/list groups, etc):
>
> {{{}java.lang.RuntimeException: No in-memory snapshot for epoch 740745.
> Snapshot epochs are:{}}}{{{}at
> org.apache.kafka.timeline.SnapshotRegistry.getSnapshot(SnapshotRegistry.java:178){}}}{{{}at
>
> org.apache.kafka.timeline.SnapshottableHashTable.snapshottableIterator(SnapshottableHashTable.java:407){}}}{{{}at
>
> org.apache.kafka.timeline.TimelineHashMap$ValueIterator.<init>(TimelineHashMap.java:283){}}}{{{}at
>
> org.apache.kafka.timeline.TimelineHashMap$Values.iterator(TimelineHashMap.java:271){}}}
> {{...}}
>
> This happens because we don't have a snapshot at the last updated high
> watermark after loading. We cannot generate a snapshot at the high watermark
> after loading all batches because it may contain records that have not yet
> been committed. We also don't know where the high watermark will advance up
> to so we need to generate a snapshot for each offset the loader observes to
> be greater than the current high watermark. Then once we add the high
> watermark listener and update the high watermark we can delete all of the
> snapshots prior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
