Sure, how to enable debug logs? Just adjust logback logger?

On Fri, Jun 30, 2017 at 11:24 Eno Thereska <eno.there...@gmail.com> wrote:

> It’s hard to tell, the logs do not contain much, I agree. It could be a
> number of things.
>
> If it’s happening as you say on restart as well (so it’s reproducible),
> any chance you could start streaming with DEBUG logs on and collect those
> logs? I’m hoping something shows up there.
>
> Thanks,
> Eno
>
>
> > On Jun 28, 2017, at 5:30 PM, Dmitriy Vsekhvalnov <dvsekhval...@gmail.com>
> wrote:
> >
> > Nothing for stat-change.log for giving time window. Last line logged 4
> > hours before app stopped.
> >
> > Any ideas so far? Personally i don't see anything relevant in logs.
> >
> > On Wed, Jun 28, 2017 at 6:33 PM, Bill Bejeck <b...@confluent.io> wrote:
> >
> >> Sure, couldn't hurt.
> >>
> >> Thanks,
> >> Bill
> >>
> >> On Wed, Jun 28, 2017 at 9:51 AM, Dmitriy Vsekhvalnov <
> >> dvsekhval...@gmail.com
> >>> wrote:
> >>
> >>> Here are logs:
> >>>
> >>> app:
> >>> https://gist.github.com/dvsekhvalnov/f98afc3463f0c63b1722417e3710a8
> >>> e7#file-kafka-streams-log
> >>> brokers:
> >>> https://gist.github.com/dvsekhvalnov/8e870f7347394e8d004c282880ef38
> >>> 5a#file-kafka-broker-1-2-3-log
> >>>
> >>> All broker logs are same, so single gist.
> >>>
> >>> There are also state-change.log files, do you want to take a look at
> >> those
> >>> as well?
> >>>
> >>> On Wed, Jun 28, 2017 at 4:31 PM, Bill Bejeck <b...@confluent.io>
> wrote:
> >>>
> >>>> Hi Dmitry,
> >>>>
> >>>> At the moment I don't have anything specific to look for, just trying
> >> to
> >>>> get more context around the issue.
> >>>>
> >>>> As for the logs maybe broker and streams logs for the last 30 minutes
> >> up
> >>> to
> >>>> the time the application stopped processing records.
> >>>>
> >>>> Thanks,
> >>>> Bill
> >>>>
> >>>> On Wed, Jun 28, 2017 at 9:04 AM, Dmitriy Vsekhvalnov <
> >>>> dvsekhval...@gmail.com
> >>>>> wrote:
> >>>>
> >>>>> Hi Bill,
> >>>>>
> >>>>> 1. sure, can extract some logs, what exactly to look for? There are
> >> 11
> >>>>> hours of logs and most of them looks like:
> >>>>>
> >>>>> [2017-06-27 03:30:50,553] [] [INFO ] [StreamThread-1]
> >>>>> [org.apache.kafka.streams.processor.internals.StreamThread]
> >>>> [stream-thread
> >>>>> [StreamThread-1] Committing all tasks because the commit interval
> >>> 5000ms
> >>>>> has elapsed]
> >>>>>
> >>>>> [2017-06-27 03:30:50,553] [] [INFO ] [StreamThread-1]
> >>>>> [org.apache.kafka.streams.processor.internals.StreamThread]
> >>>> [stream-thread
> >>>>> [StreamThread-1] Committing task StreamTask 0_0]
> >>>>>
> >>>>> [2017-06-27 03:30:50,554] [] [INFO ] [StreamThread-1]
> >>>>> [org.apache.kafka.streams.processor.internals.StreamThread]
> >>>> [stream-thread
> >>>>> [StreamThread-1] Committing task StreamTask 2_0]
> >>>>>
> >>>>> Something specific to search for?
> >>>>>
> >>>>> 2. Yes, there are more messages coming to topic.
> >>>>>
> >>>>> On Wed, Jun 28, 2017 at 3:43 PM, Bill Bejeck <b...@confluent.io>
> >>> wrote:
> >>>>>
> >>>>>> Hi Dimitry,
> >>>>>>
> >>>>>> I'm happy to help, but I could use more information.  Can you share
> >>> the
> >>>>>> streams logs and broker logs?
> >>>>>>
> >>>>>> Have you confirmed messages are still being delivered to topics
> >> (via
> >>>>>> console consumer)?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Bill
> >>>>>>
> >>>>>> On Wed, Jun 28, 2017 at 8:24 AM, Dmitriy Vsekhvalnov <
> >>>>>> dvsekhval...@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> looking for some assistance in debugging kafka-streams
> >> application.
> >>>>>>>
> >>>>>>> Kafka broker  0.10.2.1  - x3 Node cluster
> >>>>>>> kafka-streams 0.10.2.1 -  x2 application nodes x 1 stream thread
> >>>> each.
> >>>>>>>
> >>>>>>> In streams configuration only:
> >>>>>>> - SSL transport
> >>>>>>> - kafka.streams.commitIntervalMs set to 5000 (instead of default
> >>>> 30s).
> >>>>>>>
> >>>>>>> We running simple aggregation app with several grouping streams.
> >>>>> Running
> >>>>>> 2
> >>>>>>> instances of an app for redundancy. Both instances were working
> >>>> pretty
> >>>>>> fine
> >>>>>>> for 11 hours 15 minutes then stopped consuming new events from
> >>> topic.
> >>>>>>>
> >>>>>>> Hosting JVM processes were working fine, just streams stopped
> >>>> reacting
> >>>>> to
> >>>>>>> new data. No exceptions, errors, e.t.c. in logs.  After restart
> >>>> streams
> >>>>>>> still not consuming new messages.
> >>>>>>>
> >>>>>>> Below is 2 last entries from kafka-streams logs from both hosts:
> >>>>>>>
> >>>>>>> [2017-06-27 14:45:09,663] [] [INFO ] [StreamThread-1]
> >>>>>>> [org.apache.kafka.streams.processor.internals.StreamThread]
> >>>>>> [stream-thread
> >>>>>>> [StreamThread-1] Committing task StreamTask 4_2]
> >>>>>>>
> >>>>>>> [2017-06-27 14:45:09,723] [] [INFO ] [StreamThread-1]
> >>>>>>> [org.apache.kafka.streams.processor.internals.StreamThread]
> >>>>>> [stream-thread
> >>>>>>> [StreamThread-1] Committing task StreamTask 2_1]
> >>>>>>>
> >>>>>>> Pretty puzzling why they stopped exactly same moment (with
> >> respect
> >>> to
> >>>>>>> millis).
> >>>>>>>
> >>>>>>> Really appreciate any ideas where to dig to.
> >>>>>>>
> >>>>>>> Thank you.
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Reply via email to