Primarily we want to log below date(although this is not the exhaustive list):
+ any error/exception during kafka start/stop + any error/exception while broker is running + broker state changes like leader re-election, broker goes down, + Current live brokers + new topic creation + when messages are deleted by broker after specified limit + Broker health : memory usage Regards, Ravi On Tue, Jun 24, 2014 at 11:11 AM, Neha Narkhede <neha.narkh...@gmail.com> wrote: > What kind of broker metrics are you trying to push to this centralized > logging framework? > > Thanks, > Neha > On Jun 23, 2014 8:51 PM, "ravi singh" <rrs120...@gmail.com> wrote: > > > Thanks Guozhang/Neha for replies. > > Here's my use case: > > > > We use proprietary application logging in our apps. We are planning to > use > > Kafka brokers in production , but apart from the logs which are already > > logged using log4j in kafka we want to log the broker stats using our > > centralized application logging framework. > > > > Simply put I want to write an application which could start when the > kafka > > brokers starts, read the broker state and metrics and push it to the > > centralized logging servers. > > > > In ActiveMQ we have a plugin for our proprietary logging. We intercept > > broker operation and install the plugin into the interceptor chain of the > > broker. > > > > Regards, > > Ravi > > > > > > On Mon, Jun 23, 2014 at 9:29 PM, Neha Narkhede <neha.narkh...@gmail.com> > > wrote: > > > > > Ravi, > > > > > > Our goal is to provide the best implementation of a set of useful > > > abstractions and features in Kafka. The motivation behind this > philosophy > > > is performance and simplicity at the cost of flexibility. In most > cases, > > we > > > can argue that the loss in flexibility is minimal since you can always > > get > > > that functionality by modeling your application differently, especially > > if > > > the system supports high performance. ActiveMQ has to support the JMS > > > protocol and hence provide all sorts of hooks and plugins on the > brokers > > at > > > the cost of performance. > > > > > > Could you elaborate more on your use case? There is probably another > way > > to > > > model your application using Kafka. > > > > > > Thanks, > > > Neha > > > > > > > > > On Sat, Jun 21, 2014 at 9:24 AM, ravi singh <rrs120...@gmail.com> > wrote: > > > > > > > How do I intercept Kakfa broker operation so that features such as > > > > security,logging,etc can be implemented as a pluggable filter. For > > > example > > > > we have "BrokerFilter" class in ActiveMQ , Is there anything similar > in > > > > Kafka? > > > > > > > > -- > > > > *Regards,* > > > > *Ravi* > > > > > > > > > > > > > > > -- > > *Regards,* > > *Ravi* > > > -- *Regards,* *Ravi*
