We use that fork of node-kafka without any issues. We have a 3 server cluster setup. Single topic, 3 partitions. No issues. Franz-kafka is on our "to check" list but no rush yet.
Kind regards, Radek Gruchalski On 22 Dec 2012, at 18:59, Apoorva Gaurav <apoorvagau...@gmail.com> wrote: > Thanks Radek, > We also are thinking of Java / Scala for Consumers, for Producers whether > franz-kafka is a good choice? > > -- > Thanks & Regards, > Apoorva > > On Sat, Dec 22, 2012 at 9:38 PM, Radek Gruchalski < > radek.gruchal...@portico.io> wrote: > >> We started using node-kafka before we learned franz-kafka was available. >> In node, franz-kafka would be my preferred choice now. But tbh, our >> consumers are all java. node-kafka does not support consumer settings like >> autooffset.reset and so on (or it is not obvious how to use those). >> >> Afair franz-kafka offers those. Also, java zkconsumer gives you the jmx >> monitoring tools, which may be helpful if you want to add some scaling >> logic when consumer is lagging. >> >> Our first choice is node too but we're consuming exclusively with java. >> >> Hope this helps a little. >> >> On 22 Dec 2012, at 05:21, Apoorva Gaurav <apoorvagau...@gmail.com> wrote: >> >>> Which is the best ZK based implementation of kafka in node.js. Our use >> case >>> is that a pool of node js http servers will be listening to clients which >>> will send json over http. Using node js we'll do minimal decoration and >>> compression (preferably snappy) and write to brokers. We might also need >>> json to avro conversion but thats not a deal breaker. Consumers will be >>> writing these events to S3 (to begin with we don't plan to maintain HDFS >>> cluster). To begin with we'll have to support a peak load of 50K events / >>> second, average being much less, around 2K events / second. Suggestions >>> please. Is any one using franz-kafka in production. I'm only two days >> into >>> kafka so don't know a lot, but franz-kafka looks clean and easy to work >>> with. >>> >>> If none of the existing node.js implementation is capable of this then we >>> are willing to move to Scala or Java but node.js is the first choice. >>> >>> Thanks & Regards, >>> Apoorva >>> >>> On Sat, Dec 22, 2012 at 2:25 AM, Radek Gruchalski < >>> radek.gruchal...@portico.io> wrote: >>> >>>> We are using https://github.com/radekg/node-kafka, occasionally pushing >>>> about 2500 messages, 3.5K each / second. No issues so far. Different >> story >>>> with consumers. They are stable but under heavy load we experienced CPU >>>> problems. I am the maintainer of that fork. The fork comes with ZK >>>> integration. Another kafka module is this one: >>>> https://github.com/dannycoates/franz-kafka. >>>> >>>> Kind regards, >>>> Radek Gruchalski >>>> radek.gruchal...@technicolor.com (mailto: >> radek.gruchal...@technicolor.com) >>>> | radek.gruchal...@portico.io (mailto:radek.gruchal...@portico.io) | >>>> ra...@gruchalski.com (mailto:ra...@gruchalski.com) >>>> 00447889948663 >>>> >>>> Confidentiality: >>>> This communication is intended for the above-named person and may be >>>> confidential and/or legally privileged. >>>> If it has come to you in error you must take no action based on it, nor >>>> must you copy or show it to anyone; please delete/destroy and inform the >>>> sender immediately. >>>> >>>> >>>> On Thursday, 20 December 2012 at 18:31, Jun Rao wrote: >>>> >>>>> Chris, >>>>> >>>>> Not sure how stable those node.js clients are. In 0.8, we plan to >>>> provide a >>>>> native C version of the producer. A thin node.js layer can potentially >> be >>>>> built on top of that. >>>>> >>>>> Thanks, >>>>> >>>>> Jun >>>>> >>>>> On Thu, Dec 20, 2012 at 8:46 AM, Christopher Alexander < >>>>> calexan...@gravycard.com (mailto:calexan...@gravycard.com)> wrote: >>>>> >>>>>> During my due diligence to assess use of Kafka for both our activity >>>> and >>>>>> log message streams, I would like to ask the project committers and >>>>>> community users about using Kafka with Node.js. Yes, I am aware that a >>>>>> Kafka client exists for Node.js ( >>>>>> https://github.com/marcuswestin/node-kafka), which has spurred >> further >>>>>> interest by our front-end team. Here are my questions, excuse me if >>>> they >>>>>> seem "noobish". >>>>>> >>>>>> 1. How reliable is the Node.js client ( >>>>>> https://github.com/marcuswestin/node-kafka) in production >>>> applications? >>>>>> If there are issues, what are they (the GitHub repo currently lists >>>> none)? >>>>>> 2. To support real-time activity streams within Node.js, what is the >>>>>> recommended consumer polling interval? >>>>>> 3. General advise observations on integrating a front-end based >> Node.js >>>>>> application with Kafka mediated messaging. >>>>>> >>>>>> Thanks you! >>>>>> >>>>>> Chris >>
