Thanks Radek, We also are thinking of Java / Scala for Consumers, for Producers whether franz-kafka is a good choice?
-- Thanks & Regards, Apoorva On Sat, Dec 22, 2012 at 9:38 PM, Radek Gruchalski < radek.gruchal...@portico.io> wrote: > We started using node-kafka before we learned franz-kafka was available. > In node, franz-kafka would be my preferred choice now. But tbh, our > consumers are all java. node-kafka does not support consumer settings like > autooffset.reset and so on (or it is not obvious how to use those). > > Afair franz-kafka offers those. Also, java zkconsumer gives you the jmx > monitoring tools, which may be helpful if you want to add some scaling > logic when consumer is lagging. > > Our first choice is node too but we're consuming exclusively with java. > > Hope this helps a little. > > On 22 Dec 2012, at 05:21, Apoorva Gaurav <apoorvagau...@gmail.com> wrote: > > > Which is the best ZK based implementation of kafka in node.js. Our use > case > > is that a pool of node js http servers will be listening to clients which > > will send json over http. Using node js we'll do minimal decoration and > > compression (preferably snappy) and write to brokers. We might also need > > json to avro conversion but thats not a deal breaker. Consumers will be > > writing these events to S3 (to begin with we don't plan to maintain HDFS > > cluster). To begin with we'll have to support a peak load of 50K events / > > second, average being much less, around 2K events / second. Suggestions > > please. Is any one using franz-kafka in production. I'm only two days > into > > kafka so don't know a lot, but franz-kafka looks clean and easy to work > > with. > > > > If none of the existing node.js implementation is capable of this then we > > are willing to move to Scala or Java but node.js is the first choice. > > > > Thanks & Regards, > > Apoorva > > > > On Sat, Dec 22, 2012 at 2:25 AM, Radek Gruchalski < > > radek.gruchal...@portico.io> wrote: > > > >> We are using https://github.com/radekg/node-kafka, occasionally pushing > >> about 2500 messages, 3.5K each / second. No issues so far. Different > story > >> with consumers. They are stable but under heavy load we experienced CPU > >> problems. I am the maintainer of that fork. The fork comes with ZK > >> integration. Another kafka module is this one: > >> https://github.com/dannycoates/franz-kafka. > >> > >> Kind regards, > >> Radek Gruchalski > >> radek.gruchal...@technicolor.com (mailto: > radek.gruchal...@technicolor.com) > >> | radek.gruchal...@portico.io (mailto:radek.gruchal...@portico.io) | > >> ra...@gruchalski.com (mailto:ra...@gruchalski.com) > >> 00447889948663 > >> > >> Confidentiality: > >> This communication is intended for the above-named person and may be > >> confidential and/or legally privileged. > >> If it has come to you in error you must take no action based on it, nor > >> must you copy or show it to anyone; please delete/destroy and inform the > >> sender immediately. > >> > >> > >> On Thursday, 20 December 2012 at 18:31, Jun Rao wrote: > >> > >>> Chris, > >>> > >>> Not sure how stable those node.js clients are. In 0.8, we plan to > >> provide a > >>> native C version of the producer. A thin node.js layer can potentially > be > >>> built on top of that. > >>> > >>> Thanks, > >>> > >>> Jun > >>> > >>> On Thu, Dec 20, 2012 at 8:46 AM, Christopher Alexander < > >>> calexan...@gravycard.com (mailto:calexan...@gravycard.com)> wrote: > >>> > >>>> During my due diligence to assess use of Kafka for both our activity > >> and > >>>> log message streams, I would like to ask the project committers and > >>>> community users about using Kafka with Node.js. Yes, I am aware that a > >>>> Kafka client exists for Node.js ( > >>>> https://github.com/marcuswestin/node-kafka), which has spurred > further > >>>> interest by our front-end team. Here are my questions, excuse me if > >> they > >>>> seem "noobish". > >>>> > >>>> 1. How reliable is the Node.js client ( > >>>> https://github.com/marcuswestin/node-kafka) in production > >> applications? > >>>> If there are issues, what are they (the GitHub repo currently lists > >> none)? > >>>> 2. To support real-time activity streams within Node.js, what is the > >>>> recommended consumer polling interval? > >>>> 3. General advise observations on integrating a front-end based > Node.js > >>>> application with Kafka mediated messaging. > >>>> > >>>> Thanks you! > >>>> > >>>> Chris > >> > >> >
