I tried a new approach and have started ec2 instances in a public subnet
rather than a private subnet. However, now when I try to start zookeeper I
get this error. How can I go about solving this issue? Thank you.
ERROR Unexpected exception, exiting abnormally
(org.apache.zookeeper.server.ZooKeeperServerMain)
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:95)
at
org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:111)
at
org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)
at
org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)
at
org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
On Thu, Jan 29, 2015 at 9:44 AM, Su She <suhsheka...@gmail.com> wrote:
> Thank you Guozhang for an updated link.
>
> I read the answer for "Why can't my consumers/producers connect to the
> brokers?". I am confused on a couple things. I can ping the private IP of
> A (Cluster Launcher) from B (EC2 instance). I can't ping the public ip of
> A (or google.com) from B which makes sense as B is in a private subnet.
> But when I try to connect to the zookeeper at private.ip.A:2181 I get a No
> Route to Host error. Is there a reason why I can ping private.ip.A but not
> connect to private.ip.A:2181?
>
> I have not changed any of my server or producer properties. I have tried
> to change advertised host name, but no luck.
>
> Thanks for the help!
>
>
>
> On Thu, Jan 29, 2015 at 8:12 AM, Guozhang Wang <wangg...@gmail.com> wrote:
>
>> Sorry my previous link was not complete:
>>
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whycan%27tmyconsumers/producersconnecttothebrokers
>> ?
>>
>>
>>
>> On Wed, Jan 28, 2015 at 10:56 PM, Su She <suhsheka...@gmail.com> wrote:
>>
>> > Thank you Dillian and Guozhang for the responses.
>> >
>> > Yes, Dillian you are understanding my issue correctly. I am not sure
>> what
>> > the best approach to this is...I'm not sure if there's a way to
>> whitelist
>> > certain IPs, create a VPC, use the cluster launcher as the kafka
>> > zookeeper/broker. I guess this is more of an AWS question, but I thought
>> > this is a problem some Kafka users must have solved already.
>> >
>> > Edit: I just tried using the cluster launcher as an intermediate. I
>> started
>> > Zookeeper/Kafka Server on my Cluster launcher and then created a
>> > topic/produced messages. I set up a kafka consumer on one of my private
>> EC2
>> > instances, but I got a No Route to host error. I pinged the cluster
>> > launcher <-> private instance and it works fine. I was hoping I could
>> use
>> > this is as a temporary solution...any suggestions on this issue would
>> also
>> > be greatly appreciated. Thanks!
>> >
>> > Best,
>> >
>> > Su
>> >
>> >
>> > On Wed, Jan 28, 2015 at 9:11 PM, Guozhang Wang <wangg...@gmail.com>
>> wrote:
>> >
>> > > Su,
>> > >
>> > > Does this help for your case?
>> > >
>> > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ
>> > >
>> > > Guozhang
>> > >
>> > > On Wed, Jan 28, 2015 at 3:36 PM, Dillian Murphey <
>> > crackshotm...@gmail.com>
>> > > wrote:
>> > >
>> > > > Am I understanding your question correctly... You're asking how do
>> you
>> > > > establish connectivity to an instance in a private subnet from the
>> > > outside
>> > > > world? Are you thinking in terms of zookeeper or just general aws
>> > > network
>> > > > connectivity?
>> > > >
>> > > > On Wed, Jan 28, 2015 at 11:03 AM, Su She <suhsheka...@gmail.com>
>> > wrote:
>> > > >
>> > > > > Hello All,
>> > > > >
>> > > > > I have set up a cluster of EC2 instances using this method:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://blogs.aws.amazon.com/bigdata/post/Tx2D0J7QOVRJBRX/Deploying-Cloudera-s-Enterprise-Data-Hub-on-AWS
>> > > > >
>> > > > > As you can see the instances are w/in a private subnet. I was
>> > wondering
>> > > > if
>> > > > > anyone has any advice on how I can set up a Kafka
>> zookeeper/server on
>> > > an
>> > > > > instance that receives messages from a Kafka Producer outside of
>> the
>> > > > > private subnet. I have tried using the cluster launcher, but I
>> feel
>> > > like
>> > > > it
>> > > > > is not a best practice and only a temporary situation.
>> > > > >
>> > > > > Thank you for the help!
>> > > > >
>> > > > > Best,
>> > > > >
>> > > > > Su
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > -- Guozhang
>> > >
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>
>
>
