Re: Kafka behind AWS ELB

2015-05-04 Thread Jiangjie Qin
Hi Dillian, Yeah, ELB + ASG will is pretty popular however might need further tricks to use them for Kafka brokers. As I replied to Chandrashekhar in another email. You can use ELB as bootstrap.servers/metadata.broker.list to serve as client bootstrap use case. But all the producing/consuming tr

Re: Kafka behind AWS ELB

2015-05-04 Thread Christian Csar
Dillian, On Mon, May 4, 2015 at 1:52 PM, Dillian Murphey wrote: > > I'm interested in this topic as well. If you put kafka brokers inside an > autoscaling group, then AWS will automatically add brokers if demand > increases, and the ELB will automatically round-robin across all of your > kafka i

Re: Kafka behind AWS ELB

2015-05-04 Thread Dillian Murphey
I'm interested in this topic as well. If you put kafka brokers inside an autoscaling group, then AWS will automatically add brokers if demand increases, and the ELB will automatically round-robin across all of your kafka instances. So in your config files and code, you only need to provide a sing

Re: Kafka behind AWS ELB

2015-05-04 Thread Mayuresh Gharat
Ok. You can deploy kafka in AWS. You can have brokers on AWS servers. Kafka is not a push system. So you will need someone writing to kafka and consuming from kafka. It will work. My suggestion will be to try it out on a smaller instance in AWS and see the effects. As I do not know the actual use

Re: Kafka behind AWS ELB

2015-05-04 Thread Mayuresh Gharat
Hi Chandrashekar, Can you please elaborate the use case for Kafka here, like how you are planning to use it. Thanks, Mayuresh On Sat, May 2, 2015 at 9:08 PM, Chandrashekhar Kotekar < shekhar.kote...@gmail.com> wrote: > Hi, > > I am new to Apache Kafka. I have played with it on my laptop. > >

Re: Kafka behind AWS ELB

2015-05-04 Thread Chandrashekhar Kotekar
I am sorry but I cannot reveal those details due to confidentiality issues. I hope you understand. Regards, Chandrash3khar Kotekar Mobile - +91 8600011455 On Mon, May 4, 2015 at 9:18 PM, Mayuresh Gharat wrote: > Hi Chandrashekar, > > Can you please elaborate the use case for Kafka here, like h

Kafka behind AWS ELB

2015-05-02 Thread Chandrashekhar Kotekar
Hi, I am new to Apache Kafka. I have played with it on my laptop. I want to use Kafka in AWS. Currently we have tomcat web servers based REST API. We want to replace REST API with Apache Kafka, web servers are behind ELB. I would like to know if we can keep Kafka brokers behind ELB? Will it work