Hi Guozhang,
Thank for the feedback!
Please find my answers inline.
Best,
Bruno
On 14.12.20 23:33, Guozhang Wang wrote:
Hello Bruno,
Just a few more questions about the KIP:
1) If the internal topics exist but the calculated num.partitions do not
match the existing topics, what would Streams do;
Good point! I missed to explicitly consider misconfigurations in the KIP.
I propose to throw a fatal error in this case during manual and
automatic initialization. For the fatal error, we have two options:
a) introduce a second exception besides MissingInternalTopicException,
e.g. MisconfiguredInternalTopicException
b) rename MissingInternalTopicException to
MissingOrMisconfiguredInternalTopicException and throw that in both cases.
Since the process to react on such an exception user-side should be
similar, I am fine with option b). However, IMO option a) is a bit
cleaner. WDYT?
2) Since `init()` is a blocking call (we only return after all topics are
confirmed to be created), should we have a timeout for this call as well or
not;
I will add an overload with a timeout to the KIP.
3) If the configure is set to `MANUAL_SETUP`, then during rebalance do we
still check if number of partitions of the existing topic match or not; if
not, do we throw the newly added exception or throw a fatal
StreamsException? Today we would throw the StreamsException from assign()
which would be then thrown from consumer.poll() as a fatal error.
Yes, I think we should check if the number of partitions match. I
propose to throw the newly added exception in the same way as we throw
now the MissingSourceTopicException, i.e., throw it from
consumer.poll(). WDYT?
Guozhang
On Mon, Dec 14, 2020 at 12:47 PM John Roesler <vvcep...@apache.org> wrote:
Thanks, Bruno!
I'm +1 (binding)
-John
On Mon, 2020-12-14 at 09:57 -0600, Leah Thomas wrote:
Thanks for the KIP Bruno, LGTM. +1 (non-binding)
Cheers,
Leah
On Mon, Dec 14, 2020 at 4:29 AM Bruno Cadonna <br...@confluent.io>
wrote:
Hi,
I'd like to start the voting on KIP-698 that proposes an explicit user
initialization of broker-side state for Kafka Streams instead of
letting
Kafka Streams setting up the broker-side state automatically during
rebalance. Such an explicit initialization avoids possible data loss
issues due to automatic initialization.
https://cwiki.apache.org/confluence/x/7CnZCQ
Best,
Bruno
