Hi Ewen, Thanks for your response. I'll experiment and benchmark it with the normal proxy and NGinx as well and update the results.
Regards, prabcs On Mon, Jul 27, 2015 at 11:10 PM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > Hi Prabhjot, > > Confluent has a REST proxy with docs that may give some guidance: > http://docs.confluent.io/1.0/kafka-rest/docs/intro.html The new producer > that it uses is very efficient, so you should be able to get pretty good > throughput. You take a bit of a hit due to the overhead of sending data > through a proxy, but with appropriate batching you can get about 2/3 the > performance as you would get using the Java producer directly. > > There are also a few other proxies you can find here: > https://cwiki.apache.org/confluence/display/KAFKA/Clients#Clients-HTTPREST > > You can also put nginx (or HAProxy, or a variety of other solutions) in > front of REST proxies for load balancing, HA, SSL termination, etc. This is > yet another hop, so it might affect throughput and latency. > > -Ewen > > On Mon, Jul 27, 2015 at 6:55 AM, Prabhjot Bharaj <prabhbha...@gmail.com> > wrote: > > > Hi Folks, > > > > I would like to understand the best practices when using kafka as the > > source-of-truth, given the fact that I want to pump in data to Kafka > using > > http methods. > > > > What are the current production configurations for such a use case:- > > > > 1. Kafka-http-client - is it scalable the way Nginx is ?? > > 2. Using Kafka and Nginx together - If anybody has used this, please > > explain > > 3. Any other scalable method ? > > > > Regards, > > prabcs > > > > > > -- > Thanks, > Ewen > -- --------------------------------------------------------- "There are only 10 types of people in the world: Those who understand binary, and those who don't"
