Thanks Radek, thanks David. On Sun, Dec 23, 2012 at 12:08 AM, Radek Gruchalski < radek.gruchal...@portico.io> wrote:
> That is exactly why we've decided to stick with java. Also support for all > consumer settings out of the box. > > Kind regards, > Radek Gruchalski > > On 22 Dec 2012, at 19:17, David Arthur <mum...@gmail.com> wrote: > > > FWIW, message production is quite simpler than consumption. It does > > not require the same complex coordination as the consumers. Producers > > only use ZooKeeper to locate available brokers > > > > Sent from my phone > > > > On Dec 22, 2012, at 1:00 PM, Apoorva Gaurav <apoorvagau...@gmail.com> > wrote: > > > >> Thanks Radek, > >> We also are thinking of Java / Scala for Consumers, for Producers > whether > >> franz-kafka is a good choice? > >> > >> -- > >> Thanks & Regards, > >> Apoorva > >> > >> On Sat, Dec 22, 2012 at 9:38 PM, Radek Gruchalski < > >> radek.gruchal...@portico.io> wrote: > >> > >>> We started using node-kafka before we learned franz-kafka was > available. > >>> In node, franz-kafka would be my preferred choice now. But tbh, our > >>> consumers are all java. node-kafka does not support consumer settings > like > >>> autooffset.reset and so on (or it is not obvious how to use those). > >>> > >>> Afair franz-kafka offers those. Also, java zkconsumer gives you the jmx > >>> monitoring tools, which may be helpful if you want to add some scaling > >>> logic when consumer is lagging. > >>> > >>> Our first choice is node too but we're consuming exclusively with java. > >>> > >>> Hope this helps a little. > >>> > >>> On 22 Dec 2012, at 05:21, Apoorva Gaurav <apoorvagau...@gmail.com> > wrote: > >>> > >>>> Which is the best ZK based implementation of kafka in node.js. Our use > >>> case > >>>> is that a pool of node js http servers will be listening to clients > which > >>>> will send json over http. Using node js we'll do minimal decoration > and > >>>> compression (preferably snappy) and write to brokers. We might also > need > >>>> json to avro conversion but thats not a deal breaker. Consumers will > be > >>>> writing these events to S3 (to begin with we don't plan to maintain > HDFS > >>>> cluster). To begin with we'll have to support a peak load of 50K > events / > >>>> second, average being much less, around 2K events / second. > Suggestions > >>>> please. Is any one using franz-kafka in production. I'm only two days > >>> into > >>>> kafka so don't know a lot, but franz-kafka looks clean and easy to > work > >>>> with. > >>>> > >>>> If none of the existing node.js implementation is capable of this > then we > >>>> are willing to move to Scala or Java but node.js is the first choice. > >>>> > >>>> Thanks & Regards, > >>>> Apoorva > >>>> > >>>> On Sat, Dec 22, 2012 at 2:25 AM, Radek Gruchalski < > >>>> radek.gruchal...@portico.io> wrote: > >>>> > >>>>> We are using https://github.com/radekg/node-kafka, occasionally > pushing > >>>>> about 2500 messages, 3.5K each / second. No issues so far. Different > >>> story > >>>>> with consumers. They are stable but under heavy load we experienced > CPU > >>>>> problems. I am the maintainer of that fork. The fork comes with ZK > >>>>> integration. Another kafka module is this one: > >>>>> https://github.com/dannycoates/franz-kafka. > >>>>> > >>>>> Kind regards, > >>>>> Radek Gruchalski > >>>>> radek.gruchal...@technicolor.com (mailto: > >>> radek.gruchal...@technicolor.com) > >>>>> | radek.gruchal...@portico.io (mailto:radek.gruchal...@portico.io) | > >>>>> ra...@gruchalski.com (mailto:ra...@gruchalski.com) > >>>>> 00447889948663 > >>>>> > >>>>> Confidentiality: > >>>>> This communication is intended for the above-named person and may be > >>>>> confidential and/or legally privileged. > >>>>> If it has come to you in error you must take no action based on it, > nor > >>>>> must you copy or show it to anyone; please delete/destroy and inform > the > >>>>> sender immediately. > >>>>> > >>>>> > >>>>> On Thursday, 20 December 2012 at 18:31, Jun Rao wrote: > >>>>> > >>>>>> Chris, > >>>>>> > >>>>>> Not sure how stable those node.js clients are. In 0.8, we plan to > >>>>> provide a > >>>>>> native C version of the producer. A thin node.js layer can > potentially > >>> be > >>>>>> built on top of that. > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Jun > >>>>>> > >>>>>> On Thu, Dec 20, 2012 at 8:46 AM, Christopher Alexander < > >>>>>> calexan...@gravycard.com (mailto:calexan...@gravycard.com)> wrote: > >>>>>> > >>>>>>> During my due diligence to assess use of Kafka for both our > activity > >>>>> and > >>>>>>> log message streams, I would like to ask the project committers and > >>>>>>> community users about using Kafka with Node.js. Yes, I am aware > that a > >>>>>>> Kafka client exists for Node.js ( > >>>>>>> https://github.com/marcuswestin/node-kafka), which has spurred > >>> further > >>>>>>> interest by our front-end team. Here are my questions, excuse me if > >>>>> they > >>>>>>> seem "noobish". > >>>>>>> > >>>>>>> 1. How reliable is the Node.js client ( > >>>>>>> https://github.com/marcuswestin/node-kafka) in production > >>>>> applications? > >>>>>>> If there are issues, what are they (the GitHub repo currently lists > >>>>> none)? > >>>>>>> 2. To support real-time activity streams within Node.js, what is > the > >>>>>>> recommended consumer polling interval? > >>>>>>> 3. General advise observations on integrating a front-end based > >>> Node.js > >>>>>>> application with Kafka mediated messaging. > >>>>>>> > >>>>>>> Thanks you! > >>>>>>> > >>>>>>> Chris > >>> > -- Thanks & Regards, Apoorva
