Thank you Guozhang for an updated link. I read the answer for "Why can't my consumers/producers connect to the brokers?". I am confused on a couple things. I can ping the private IP of A (Cluster Launcher) from B (EC2 instance). I can't ping the public ip of A (or google.com) from B which makes sense as B is in a private subnet. But when I try to connect to the zookeeper at private.ip.A:2181 I get a No Route to Host error. Is there a reason why I can ping private.ip.A but not connect to private.ip.A:2181?
I have not changed any of my server or producer properties. I have tried to change advertised host name, but no luck. Thanks for the help! On Thu, Jan 29, 2015 at 8:12 AM, Guozhang Wang <wangg...@gmail.com> wrote: > Sorry my previous link was not complete: > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whycan%27tmyconsumers/producersconnecttothebrokers > ? > > > > On Wed, Jan 28, 2015 at 10:56 PM, Su She <suhsheka...@gmail.com> wrote: > > > Thank you Dillian and Guozhang for the responses. > > > > Yes, Dillian you are understanding my issue correctly. I am not sure what > > the best approach to this is...I'm not sure if there's a way to whitelist > > certain IPs, create a VPC, use the cluster launcher as the kafka > > zookeeper/broker. I guess this is more of an AWS question, but I thought > > this is a problem some Kafka users must have solved already. > > > > Edit: I just tried using the cluster launcher as an intermediate. I > started > > Zookeeper/Kafka Server on my Cluster launcher and then created a > > topic/produced messages. I set up a kafka consumer on one of my private > EC2 > > instances, but I got a No Route to host error. I pinged the cluster > > launcher <-> private instance and it works fine. I was hoping I could use > > this is as a temporary solution...any suggestions on this issue would > also > > be greatly appreciated. Thanks! > > > > Best, > > > > Su > > > > > > On Wed, Jan 28, 2015 at 9:11 PM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > Su, > > > > > > Does this help for your case? > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ > > > > > > Guozhang > > > > > > On Wed, Jan 28, 2015 at 3:36 PM, Dillian Murphey < > > crackshotm...@gmail.com> > > > wrote: > > > > > > > Am I understanding your question correctly... You're asking how do > you > > > > establish connectivity to an instance in a private subnet from the > > > outside > > > > world? Are you thinking in terms of zookeeper or just general aws > > > network > > > > connectivity? > > > > > > > > On Wed, Jan 28, 2015 at 11:03 AM, Su She <suhsheka...@gmail.com> > > wrote: > > > > > > > > > Hello All, > > > > > > > > > > I have set up a cluster of EC2 instances using this method: > > > > > > > > > > > > > > > > > > > > > > > > > http://blogs.aws.amazon.com/bigdata/post/Tx2D0J7QOVRJBRX/Deploying-Cloudera-s-Enterprise-Data-Hub-on-AWS > > > > > > > > > > As you can see the instances are w/in a private subnet. I was > > wondering > > > > if > > > > > anyone has any advice on how I can set up a Kafka zookeeper/server > on > > > an > > > > > instance that receives messages from a Kafka Producer outside of > the > > > > > private subnet. I have tried using the cluster launcher, but I feel > > > like > > > > it > > > > > is not a best practice and only a temporary situation. > > > > > > > > > > Thank you for the help! > > > > > > > > > > Best, > > > > > > > > > > Su > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > -- > -- Guozhang >
