Hello, Could you tell me if burrow or remora is compatible with ssl kafka clusters ?
Gabriel. 2017-08-16 15:39 GMT+02:00 Gabriel Machado <gmachado....@gmail.com>: > Hi Jens and Ian, > > Very usefuls projects :). > What's the difference between the 2 softwares ? > Do they support kafka ssl clusters ? > > Thanks, > Gabriel. > > 2017-08-13 3:29 GMT+02:00 Ian Duffy <i...@ianduffy.ie>: > >> Hi Jens, >> >> We did something similar to this at Zalando. >> >> https://github.com/zalando-incubator/remora >> >> It effectively supplies the kafka consumer group supply command as a http >> endpoint. >> >> On 12 August 2017 at 16:42, Subhash Sriram <subhash.sri...@gmail.com> >> wrote: >> >> > Hi Jens, >> > >> > Have you looked at Burrow? >> > >> > https://github.com/linkedin/Burrow/blob/master/README.md >> > >> > Thanks, >> > Subhash >> > >> > Sent from my iPhone >> > >> > > On Aug 12, 2017, at 8:55 AM, Jens Rantil <jens.ran...@gmail.com> >> wrote: >> > > >> > > Hi, >> > > >> > > I am one of the maintainers of prometheus-kafka-consumer- >> > group-exporter[1], >> > > which exports consumer group offsets and lag to Prometheus. The way we >> > > currently scrape this information is by periodically executing >> > > `kafka-consumer-groups.sh --describe` for each group and parse the >> > output. >> > > >> > > Recently the output from `kafka-consumer-groups.sh --describe` was >> > > changed[2]. While I am working on a patch[3] to accomodate for the new >> > > output format I was wondering if there is an easier, possibly more >> > > stable[4] and more future proof, way for our project to extract the >> > > information we are interested in. Does anyone know of a Go library >> that >> > > could extract the metrics we need? Or would it make sense to refactor >> > > `kafka-consumer-groups.sh` to support a more structured output? I'd >> love >> > to >> > > hear your input. >> > > >> > > Also, if Kafka exported the same metrics through JMX our project would >> > not >> > > exist, but maybe that's another story... >> > > >> > > Cheers, >> > > Jens >> > > >> > > [1] https://github.com/kawamuray/prometheus-kafka-consumer- >> > group-exporter >> > > [2] >> > > https://github.com/kawamuray/prometheus-kafka-consumer- >> > group-exporter/issues/24 >> > > [3] >> > > https://github.com/kawamuray/prometheus-kafka-consumer- >> > group-exporter/pull/29 >> > > [4] We've also encountered `kafka-consumer-groups.sh` hanging a few >> times >> > > in production. There's a race condition somewhere in the script, most >> > > likely when a topic is rebalancing. Currently we kill the process if >> it >> > > doesn't finish within a timeout. See >> > > https://github.com/kawamuray/prometheus-kafka-consumer- >> > group-exporter/blob/e4cdc3b1245f636d89d7e227066f02 >> > 578d732165/kafka/collector.go#L44 >> > > . >> > > >> > > -- >> > > Want to communicate with me securely? You can find my PGP key here >> > > <https://pgp.mit.edu/pks/lookup?op=get&search=0xBA632DB691DA03B8>. >> > >> > >
