Thanks Liam, I'm trying with Filebeat but i'm unsure what filter should i use for kafka logs. I need all the controller.log, server.log and state-change.log to be shipped. I tried using grok filter but that is throwing error. Do you have any info on the filter to be used for Kafka logs?
Regards On Tue, Sep 24, 2019 at 1:11 PM Liam Clarke <[email protected]> wrote: > Cc list > > ---------- Forwarded message --------- > From: Liam Clarke <[email protected]> > Date: Tue, 24 Sep. 2019, 7:40 pm > Subject: Re: Shipping Kafka logs > To: Eva Sheeva <[email protected]> > > > Ah, for that you need something like Filebeat - it's an agent that forwards > monitored log files to ELK. > > On Tue, 24 Sep. 2019, 7:17 pm Eva Sheeva, <[email protected]> wrote: > > > Thank you Jacob and Liam! > > > > I 'm looking for the steps on shipping Kafka application logs. I want the > > controller.log, state-change.log and server.log to be shipped to elk. Can > > you pls help! > > > > Regards! > > > > On Tue, Sep 24, 2019 at 8:08 AM Botuck, Jacob (STL) < > > [email protected]> wrote: > > > >> You can also try > >> > https://docs.confluent.io/current/connect/kafka-connect-elasticsearch/index.html > >> I think both links are about shipping kafka messages to elk, not actual > >> kafka application logs > >> > >> -----Original Message----- > >> From: Liam Clarke <[email protected]> > >> Sent: Monday, September 23, 2019 5:21 PM > >> To: [email protected] > >> Subject: [EXTERNAL] Re: Shipping Kafka logs > >> > >> Hi Eva, > >> > >> Hope this helps. > >> <company email system killed this link> > >> Kind regards, > >> > >> Liam Clarke > >> > >> On Tue, Sep 24, 2019 at 2:57 AM Eva Sheeva <[email protected]> > wrote: > >> > >> > Hello > >> > > >> > Can you please point me to the doc or guide me on shipping kafka logs > >> > to elk. I understand it is a little different than how we ship other > >> > logs to elk. Kindly help me with this > >> > > >> > Thanks > >> > > >> > > >
