1) you are right, a thread's restoration phase will not interfere will any
other threads' normal processing collocated within the same JVM / machine
etc at all. So you may have a Streams instance which contains some threads
already finished restoring and started processing tasks, while other
thread
Hi Guozhang -
Thanks for the replies, and directing me to the existing JIRAs. I think
that a two-phase rebalance will be quite useful.
1) For clarity's sake, I should have just asked: When a new thread / node
is created and tasks are rebalanced, are the state stores on the new
threads/nodes fully
Hello Adam,
Sorry for being late replying on this thread, I've put my comments inlined
below.
On Sun, Feb 3, 2019 at 7:34 AM Adam Bellemare
wrote:
> Hey Folks
>
> I have a few questions around the operations of stateful processing while
> scaling nodes up/down, and a possible KIP in question #4
Adam,
I don’t have an answer for you but I would also be interested in clarification
of this process if anyone can provide more details. If your reading is correct
I would welcome the KIP to reduce the scaling pauses.
Cheers,
Rhys McCaig
> On Feb 6, 2019, at 7:44 AM, Adam Bellemare wrote:
>
Bump - hoping someone has some insight. Alternately, redirection to a more
suitable forum.
Thanks
On Sun, Feb 3, 2019 at 10:25 AM Adam Bellemare
wrote:
> Hey Folks
>
> I have a few questions around the operations of stateful processing while
> scaling nodes up/down, and a possible KIP in questi
Hey Folks
I have a few questions around the operations of stateful processing while
scaling nodes up/down, and a possible KIP in question #4. Most of them have
to do with task processing during rebuilding of state stores after scaling
nodes up.
Scenario:
Single node/thread, processing 2 topics (1
