Re: Shutting down a Streams job

2017-02-09 Thread Eno Thereska
Hi Dmitry, Elias, You raise a valid point, and thanks for opening https://issues.apache.org/jira/browse/KAFKA-4748 Elias. We'll hopefully have some ideas to share soon. Eno > On 9 Feb 2017, at 16:54, Dmitry Minkovsky wrote: > > That makes

Re: Shutting down a Streams job

2017-02-09 Thread Dmitry Minkovsky
That makes sense. That's what I was kind of worried about (launching soon). Hope someone else posts! ср, 8 февр. 2017 г. в 16:54, Elias Levy : > It is certainly possible, but when you got dozens of workers, that would > take a very long time, specially if you got a lot of state, as partitions > g

Re: Shutting down a Streams job

2017-02-08 Thread Elias Levy
It is certainly possible, but when you got dozens of workers, that would take a very long time, specially if you got a lot of state, as partitions get reassigned and state moved about. In fact, it is likely to fail at some point, as local state that can be stored in a multitude of nodes may not be

Re: Shutting down a Streams job

2017-02-08 Thread Dmitry Minkovsky
Can you take them down sequentially? Like, say, with a Kubernetes StatefulSet . On Wed, Feb 8, 2017 at 2:15 PM, Elias Levy wrote: > What are folks doing to cleanly shutdown a Streams job compr