Thunder, Have you had a chance to go to production with the C# producer and any updates on documentation?
I will be checking out the C# library sometime next week. Thanks -Matti -----Original Message----- From: Thunder Stumpges [mailto:tstump...@ntent.com] Sent: Monday, December 22, 2014 1:10 PM To: users@kafka.apache.org Subject: RE: Kafka in C# We definitely will. Wanted to wait until we got more confident the API was solid, ran in production for a little, and added a bit more documentation. Cheers, Thunder -----Original Message----- From: Joe Stein [joe.st...@stealth.ly] Received: Monday, 22 Dec 2014, 10:05AM To: users@kafka.apache.org [users@kafka.apache.org] Subject: Re: Kafka in C# Thunder, can you add that to https://cwiki.apache.org/confluence/display/KAFKA/Clients#Clients-.net didn't know it existed but cool that it uses Rx On Mon, Dec 22, 2014 at 12:52 PM, Thunder Stumpges <tstump...@ntent.com> wrote: > Hi there, > > We looked at both of these a while back and ended up writing our own ( > https://github.com/ntent-ad/kafka4net). > > The first one in the list was completely synchronous, and had no concept > of batching. We initially attempted to use the second one (kafka-net) but > had some issues with detecting leader changes, temporary errors, and > general topic metadata changes. Also had some problems with the use of > async and the tracking of correlation IDs on tcp messages. > > I know our client does not have a lot of documentation (yet) and says > "Work in progress, not ready yet!" but we have been testing this a lot, and > have been running in staging under load with good results. We will be > entering production in the next few weeks (waiting until after the > holidays). I wouldn't be comfortable with it going straight to production > without some testing in your environment, but especially the Producer is > very robust, and is resilient to all sorts of changes in the environment > (the integration tests use Vagrant and a set of VMs and test Producing to > non-existent partition and waiting for auto-creation of topic, partition > rebalancing, broker down or other partition re-assignment, etc.) The client > is fully Async and leverages Rx (https://rx.codeplex.com/) and an > event-loop-scheduler to do all processing. Changes in partition state are > broadcast Rx style to listening components (Producer, Consumer, > PartitionRecoveryMonitor). > > One of the reasons we have not finalized the documentation and notice on > the github page is we weren't sure if the API might change based on usage. > To this end, we'd like to know what you think and if you have any use-cases > not handled by the API. > > I understand if you're not comfortable with the beta state of the client, > but we'd love to have you check it out. We are actively developing on this > and can help with any issues. > > Thanks, > Thunder > > > -----Original Message----- > From: Matti Waarna [mailto:mwaa...@sapient.com] > Sent: Monday, December 22, 2014 7:55 AM > To: users@kafka.apache.org > Subject: Kafka in C# > > We are using kafka version 0.8.1.1 and trying to produce from a C# app. > > I realize that there is no official C# library release and want to get > your experience with the existing solutions that are currently available. > > I am looking for a solution that is a) stable enough for production > environment and b) performs well. > > 1) A couple of active github projects are available along with a few forks > each. > Has Anybody worked on either of the following two options to contribute > their findings? > https://github.com/Jroland/kafka-net > > https://github.com/miknil/Kafka4n > > 2) Also there is the option of IKVM to import kafka JARS into a .net DLL. > > Maybe even another solution? > > Thanks > > MATTI > >
