James Hay created KAFKA-9995:
--------------------------------
Summary: IllegalStateException when offsets not found
Key: KAFKA-9995
URL: https://issues.apache.org/jira/browse/KAFKA-9995
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 2.4.1
Reporter: James Hay
I have a recently upgraded Kafka Streams 2.4.1 application and we have started
seeing the application periodically crash due to the following error:
{code:java}
2020-05-14T16:53:03.839Z DEBUG <>
[chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2]
o.a.k.c.c.i.ConsumerCoordinator - [Consumer
clientId=chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2-consumer,
groupId=chat] Fetching committed offsets for partitions:
[private.chat.endpoint-0, public.chat.message-0] 2020-05-14T16:53:03.841Z INFO
<> [chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2]
o.a.k.c.c.i.ConsumerCoordinator - [Consumer
clientId=chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2-consumer,
groupId=chat] Found no committed offset for partition private.chat.endpoint-0
2020-05-14T16:53:03.842Z ERROR <>
[chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2]
o.a.k.s.p.internals.StreamThread - stream-thread
[chat-c199e52b-9344-457c-a2ae-04a669a7e0fb-StreamThread-2] Encountered the
following error during processing: java.lang.IllegalStateException: Offset
limit should monotonically increase, but was reduced. New limit: 0. Previous
limit: 857859 at
org.apache.kafka.streams.processor.internals.StandbyTask.updateOffsetLimits(StandbyTask.java:215)
at
org.apache.kafka.streams.processor.internals.StandbyTask.update(StandbyTask.java:181)
at
org.apache.kafka.streams.processor.internals.StreamThread.maybeUpdateStandbyTasks(StreamThread.java:1048)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:825)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:698)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:671)
{code}
Is this a known issue? What can cause offsets to not be found?
Other things to note:
* We have TOPOLOGY_OPTIMIZATION set to OPTIMIZE
* We are only seeing this on two of our three environments (at the moment).
The two environments we are seeing a problem have little traffic and only seems
to impact the machines that are on the whole idle. Our prod environment which
consumes regular events is showing no signs of having the same problem.
* There is some evidence to suggest there is a pattern to the timing of this
error. Although not always the case, 24hrs between errors is common.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
