Kafka hangs on startup

2019-06-21 Thread Nik Hodgkinson
I'm experiencing an issue I'm not sure how to start to track down. My Kafka brokers often startup and then hang part way through startup. The log has nothing useful to point to a problem and I'm not sure where to go from here. I've pasted a startup log below; please advise how to proceed. Thanks,

Assunto: Kafka hangs on startup

2019-06-21 Thread Marcelo Barbosa
Hi Nik,  It seams that the problem is not on the startup because the logs shows that it took about 2 secs to start. How exactly did you detect that it stuck?  Cheers,  Barbosa Enviado do Yahoo Mail no Android Em sex, 21 21e jun 21e 2019 às 5:52, Nik Hodgkinson<11x...@gmail.com> escreveu:

Re: [VOTE] 2.3.0 RC3

2019-06-21 Thread Kamal Chandraprakash
+1 (non-binding) * Ran unit and integration test on 2.11 and 2.12 * Verified quick start * Ran internal apps on the 3 node cluster On Thu, Jun 20, 2019 at 3:33 AM Colin McCabe wrote: > Hi all, > > We discovered some problems with the second release candidate (RC2) of > 2.3.0. Specifically, KAF

Re: Kafka Streams - offset number affected by "exactly once processing guarantee"

2019-06-21 Thread Trinity Xia
Thank you so much for the detailed explanation, Matthias! It's really helpful! Hope you have a good weekend :) On Thu, Jun 20, 2019 at 5:30 PM Matthias J. Sax wrote: > The observed behavior is expected. > > > For example, if we send 2615 events to an empty topic, we expect the end > of > >> the

Re: Can kafka internal state be purged ?

2019-06-21 Thread John Roesler
Sure, the record cache attempts to save downstream operators from unnecessary updates by also buffering for a short amount of time before forwarding. It forwards results whenever the cache fills up or whenever there is a commit. If you're happy to wait at least "commit interval" amount of time for

Re: Can kafka internal state be purged ?

2019-06-21 Thread Parthasarathy, Mohan
That change "In the same partition" must explain what we are seeing. Unless you see one message per partition, all windows will not expire. That is an interesting twist. Thanks for the correction ( I will go back and confirm this. -mohan On 6/21/19, 12:40 PM, "John Roesler" wrote: Sure,

Re: Can kafka internal state be purged ?

2019-06-21 Thread John Roesler
No problem. It's definitely a subtlety. It occurs because each partition is processed completely independently of the others, so "stream time" is tracked per partition, and there's no way to look across at the other partitions to find out what stream time they have. In general, it's not a problem

Re: PR review

2019-06-21 Thread Matthias J. Sax
I would recommend to move this discussion to the dev list. -Matthias On 6/20/19 7:42 PM, M. Manna wrote: > It's done. Sorry for the confusion. > > The KIP table however, showed that it's been accepted. But yes it's better > to keep all places consistent. > > Thanks, > > On Fri, 21 Jun 2019 at