Sure, the custom producer and consumer library I'm updating is Krackle: https://github.com/blackberry/Krackle
It's a very limited low overhead library that we're using to get syslog style messages from the Blackberry service provider infrastructure into Kafka. We have dozens of pools of servers containing dozens of hosts each which each host receiving and forwarding ~50K msg/sec into Kafka. We found that neither the Kafka proper producers nor Kafka could keep up with the stock packages when we initially looked (pre 0.8) ________________________________________ From: isma...@gmail.com [isma...@gmail.com] on behalf of Ismael Juma [ism...@juma.me.uk] Sent: Thursday, December 10, 2015 12:44 PM To: dev@kafka.apache.org Subject: Re: Interacting with a secured Kafka cluster via GSS-API On Thu, Dec 10, 2015 at 5:24 PM, Dave Ariens <dari...@blackberry.com> wrote: > Absolutely, currently I'm hoping to get authentication working and then > ultimately work towards encryption. We're also testing performance of > more out-of-the-box Kafka components but I fear our message volume will > require us to maintain our custom producers and consumers. > If it turns out that the out of the box components are not good enough, it would be interesting to know how your custom clients differ. Is that documented somewhere? Thanks, Ismael
