[
https://issues.apache.org/jira/browse/KAFKA-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606846#comment-16606846
]
Seweryn Habdank-Wojewodzki commented on KAFKA-7214:
---------------------------------------------------
Hi [~vvcephei],
This is what I had stayed in other problem reports.
Memory consumption or memory model described in Kafka documentation does not
fit to reality.
Now I am in the phase, when I am fully guessing, by obtaining mystic fatals
during data processing.
Is it expected, to increase memory when I get any kind of error in Kafka?
I would be really greatful when I can more less, even with 30% overhead, but
calculate, how much memory I need for my service to process X Msg/s with given
size and given retention and given whatever. But it is not the case.
And again. If error reporting would be -> Out Of Memory. I would also quickly
see that is is really memory issue and I would be able to calculate this
myself. But ending with:
{code}
2018-08-21 15:59:22 [] [ERROR] StreamTask:550 - task [0_0] Could not close
task due to the following error:
org.apache.kafka.streams.errors.TaskMigratedException: StreamsTask taskId: 0_0
{code}
Does not even point to the problem, and still I am not sure if it is really
memory problem, I see only, that when I give more memory, I do not see it that
often.
> Mystic FATAL error
> ------------------
>
> Key: KAFKA-7214
> URL: https://issues.apache.org/jira/browse/KAFKA-7214
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.11.0.3, 1.1.1
> Reporter: Seweryn Habdank-Wojewodzki
> Priority: Critical
>
> Dears,
> Very often at startup of the streaming application I got exception:
> {code}
> Exception caught in process. taskId=0_1, processor=KSTREAM-SOURCE-0000000000,
> topic=my_instance_medium_topic, partition=1, offset=198900203;
> [org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:212),
>
> org.apache.kafka.streams.processor.internals.AssignedTasks$2.apply(AssignedTasks.java:347),
>
> org.apache.kafka.streams.processor.internals.AssignedTasks.applyToRunningTasks(AssignedTasks.java:420),
>
> org.apache.kafka.streams.processor.internals.AssignedTasks.process(AssignedTasks.java:339),
>
> org.apache.kafka.streams.processor.internals.StreamThread.processAndPunctuate(StreamThread.java:648),
>
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:513),
>
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:482),
>
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:459)]
> in thread
> my_application-my_instance-my_instance_medium-72ee1819-edeb-4d85-9d65-f67f7c321618-StreamThread-62
> {code}
> and then (without shutdown request from my side):
> {code}
> 2018-07-30 07:45:02 [ar313] [INFO ] StreamThread:912 - stream-thread
> [my_application-my_instance-my_instance-72ee1819-edeb-4d85-9d65-f67f7c321618-StreamThread-62]
> State transition from PENDING_SHUTDOWN to DEAD.
> {code}
> What is this?
> How to correctly handle it?
> Thanks in advance for help.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
