I just added a comment to https://issues.apache.org/jira/browse/SAMZA-506
with details on our current approach to clean shutdown in Samza 0.8.0,
hopefully it's useful to others.
On Fri, Feb 6, 2015 at 2:39 PM, Chris Riccomini
wrote:
> Hey Jae,
>
> > If so, what's the best way to shutdown the co
Hey Jae,
> If so, what's the best way to shutdown the container without using command
topic?
YARN does send a SIGTERM before SIGKILL. The config in YARN to set the
latency is here:
yarn.nodemanager.sleep-delay-before-sigkill.ms
The default is 250ms. Samza does *not* currently handle the SIGTE
What I am doing is, consuming two topics, samza_input and samza_command.
samza_command will have some control command something like "shutdown,all"
because kill-yarn-job.sh does not gracefully shutdown SamzaContainer. Am I
correct? If so, what's the best way to shutdown the container without using
Hey Jae,
SamzaContainer polls for new messages by calling
consumerMultiplexer.choose. In a case where there are no messages
available, choose will return null. The next time choose is called, it will
be invoked with a timeout (the default is 10ms). This time, the poll call
will block until 1) the
Could you explain why consumerMultiplexer.choose returns null?
Can it happen when there's no message in the kafka topic?
If my theory is correct, its frequency is too high, in my testing
environment, it's more than 50 per second.
Thank you
Best, Jae
