Hey Walker, The proposal makes sense to me, but while reading up on those old tickets I started wondering if we should give users two options: one that would shut down the entire application, as described in the current KIP, and another that would only shut down an individual instance.
I think there are a number of reasons that shutting down only the erroring instance and not the entire application would be useful, for example if the local state is corrupted. Currently a user can achieve this, but the mechanism is pretty awkward and has caused issues in the past: calling close inside the uncaught exception handler. The uncaught exception handler is called from the stream thread, and close(timeout) is supposed to wait for all stream threads to shutdown. So you end up waiting for the full timeout to pass and even once it does, you know for a fact that not all threads have completed shutdown. Not to mention, if you happen to call the close() overload with no timeout parameter, your application will be stuck blocking forever. Giving users an exception to throw that means "shutdown the instance" would make this common practice significantly smoother. It shouldn't expand the scope of this KIP too much since we'll have a mechanism to shut down the instance already in place. WDYT? On that note, it might be a good idea to choose a more explicit name for the exception in the current proposal, eg `ShutdownApplicationException`, so it's obvious what exactly will shut down. Then we could have the second exception be named `ShutdownInstanceException` or so on Sophie On Fri, Sep 11, 2020 at 11:08 AM Matthias J. Sax <mj...@apache.org> wrote: > Thanks for the KIP. > > It seem that the new exception would need to be thrown by user code? > However, in the motivation you mention the scenario of a missing source > topic that a user cannot detect, but KafkaStreams runtime would be > responsible to handle. > > How do both things go together? > > > -Matthias > > On 9/11/20 10:31 AM, Walker Carlson wrote: > > Hello all, > > > > I have created KIP-671 to give the option to shutdown a streams > > application in response to an error. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-671%3A+Shutdown+Streams+Application+when+appropriate+exception+is+thrown > > > > This is because of the Jira ticket > > <https://issues.apache.org/jira/browse/KAFKA-9331> > > > > Please give it a look and let me know if you have any feedback. > > > > Thanks, > > Walker > > > >
