Some additional info from the client side
[zk: rhes75:4300(CONNECTED) 3] *ls /brokers/topics * #Gives the list of
topics
[test, md]
[zk: rhes75:4300(CONNECTED) 5] *ls /brokers/ids*
[1]
[zk: rhes75:4300(CONNECTED) 4] *get /brokers/ids/1 * #detailed info about
broker ID '1'
{"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://localhost:12092"],"jmx_port":9999,"host":"localhost","timestamp":"1531512017557","port":12092,"version":4}
cZxid = 0x16f
ctime = Fri Jul 13 21:00:17 BST 2018
mZxid = 0x16f
mtime = Fri Jul 13 21:00:17 BST 2018
pZxid = 0x16f
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x164953a10100000
dataLength = 192
numChildren = 0
[zk: rhes75:4300(CONNECTED) 6] *get /controller*
{"version":1,"brokerid":1,"timestamp":"1531512017622"}
cZxid = 0x170
ctime = Fri Jul 13 21:00:17 BST 2018
mZxid = 0x170
mtime = Fri Jul 13 21:00:17 BST 2018
pZxid = 0x170
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x164953a10100000
dataLength = 54
numChildren = 0
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Sat, 14 Jul 2018 at 21:20, Mich Talebzadeh <mich.talebza...@gmail.com>
wrote:
> Apologies correction. Got the port number incorrect. It should be 4300
>
> on the docker itself confirming Zookeeper port
>
> *netstat -plten|grep 4300*
> (Not all processes could be identified, non-owned process info
> will not be shown, you would have to be root to see it all.)
> tcp 0 0 0.0.0.0:4300 0.0.0.0:*
> LISTEN 1000 4638251 7026/java
>
> *jps|grep 7026*7026 QuorumPeerMain
>
> On the remote host
>
> hduser@rhes564: /home/hduser/zookeeper-3.4.6/bin> ./zkCli.sh -server
> rhes75:4300
> *Connecting to rhes75:4300*
> 2018-07-14 21:27:35,560 [myid:] - INFO [main:Environment@100] - Client
> environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
> 2018-07-14 21:27:35,563 [myid:] - INFO [main:Environment@100] - Client
> environment:host.name=rhes564
> 2018-07-14 21:27:35,563 [myid:] - INFO [main:Environment@100] - Client
> environment:java.version=1.8.0_77
> 2018-07-14 21:27:35,565 [myid:] - INFO [main:Environment@100] - Client
> environment:java.vendor=Oracle Corporation
> 2018-07-14 21:27:35,565 [myid:] - INFO [main:Environment@100] - Client
> environment:java.home=/usr/java/jdk1.8.0_77/jre
> 2018-07-14 21:27:35,565 [myid:] - INFO [main:Environment@100] - Client
> environment:java.class.path=/home/hduser/zookeeper-3.4.6/bin/../build/classes:/home/hduser/zookeeper-3.4.6/bin/../build/lib/*.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/home/hduser/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/home/hduser/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/home/hduser/zookeeper-3.4.6/bin/../conf:
> 2018-07-14 21:27:35,565 [myid:] - INFO [main:Environment@100] - Client
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:java.io.tmpdir=/tmp
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:java.compiler=<NA>
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:os.name=Linux
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:os.arch=amd64
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:os.version=2.6.18-92.el5
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:user.name=hduser
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:user.home=/home/hduser
> 2018-07-14 21:27:35,566 [myid:] - INFO [main:Environment@100] - Client
> environment:user.dir=/home/hduser/zookeeper-3.4.6/bin
> 2018-07-14 21:27:35,567 [myid:] - INFO [main:ZooKeeper@438] - Initiating
> client connection, connectString=rhes75:4300 sessionTimeout=30000
> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@506c589e
> Welcome to ZooKeeper!
> 2018-07-14 21:27:35,588 [myid:] - INFO
> [main-SendThread(rhes75:4300):ClientCnxn$SendThread@975] - Opening socket
> connection to server rhes75/50.140.197.220:4300. Will not attempt to
> authenticate using SASL (unknown error)
> JLine support is enabled
> 2018-07-14 21:27:35,644 [myid:] - INFO
> [main-SendThread(rhes75:4300):ClientCnxn$SendThread@852] - Socket
> connection established to rhes75/50.140.197.220:4300, initiating session
> [zk: rhes75:4300(CONNECTING) 0] 2018-07-14 21:27:35,666 [myid:] - INFO
> [main-SendThread(rhes75:4300):ClientCnxn$SendThread@1235] - Session
> establishment complete on server rhes75/50.140.197.220:4300, sessionid =
> 0x164953a1010000a, negotiated timeout = 30000
> WATCHER::
> WatchedEvent state:SyncConnected type:None path:null
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Sat, 14 Jul 2018 at 21:01, Mich Talebzadeh <mich.talebza...@gmail.com>
> wrote:
>
>> Hi
>>
>> This is from remote host
>>
>> ./zkCli.sh -server rhes75:3400
>> *Connecting to rhes75:3400*
>> 2018-07-14 21:08:41,104 [myid:] - INFO [main:Environment@100] - Client
>> environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
>> 2018-07-14 21:08:41,107 [myid:] - INFO [main:Environment@100] - Client
>> environment:host.name=rhes564
>> 2018-07-14 21:08:41,107 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.version=1.8.0_77
>> 2018-07-14 21:08:41,109 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.vendor=Oracle Corporation
>> 2018-07-14 21:08:41,109 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.home=/usr/java/jdk1.8.0_77/jre
>> 2018-07-14 21:08:41,109 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.class.path=/home/hduser/zookeeper-3.4.6/bin/../build/classes:/home/hduser/zookeeper-3.4.6/bin/../build/lib/*.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/home/hduser/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/home/hduser/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/home/hduser/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/home/hduser/zookeeper-3.4.6/bin/../conf:
>> 2018-07-14 21:08:41,109 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> 2018-07-14 21:08:41,109 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.io.tmpdir=/tmp
>> 2018-07-14 21:08:41,109 [myid:] - INFO [main:Environment@100] - Client
>> environment:java.compiler=<NA>
>> 2018-07-14 21:08:41,110 [myid:] - INFO [main:Environment@100] - Client
>> environment:os.name=Linux
>> 2018-07-14 21:08:41,110 [myid:] - INFO [main:Environment@100] - Client
>> environment:os.arch=amd64
>> 2018-07-14 21:08:41,110 [myid:] - INFO [main:Environment@100] - Client
>> environment:os.version=2.6.18-92.el5
>> 2018-07-14 21:08:41,110 [myid:] - INFO [main:Environment@100] - Client
>> environment:user.name=hduser
>> 2018-07-14 21:08:41,110 [myid:] - INFO [main:Environment@100] - Client
>> environment:user.home=/home/hduser
>> 2018-07-14 21:08:41,110 [myid:] - INFO [main:Environment@100] - Client
>> environment:user.dir=/home/hduser/zookeeper-3.4.6/bin
>> 2018-07-14 21:08:41,111 [myid:] - INFO [main:ZooKeeper@438] -
>> Initiating client connection, connectString=rhes75:3400
>> sessionTimeout=30000
>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@506c589e
>> Welcome to ZooKeeper!
>> 2018-07-14 21:08:41,131 [myid:] - INFO
>> [main-SendThread(rhes75:3400):ClientCnxn$SendThread@975] - Opening
>> socket connection to server rhes75/50.140.197.220:3400. Will not attempt
>> to authenticate using SASL (unknown error)
>> JLine support is enabled
>> 2018-07-14 21:08:41,191 [myid:] - WARN
>> [main-SendThread(rhes75:3400):ClientCnxn$SendThread@1102] - Session 0x0
>> for server null, unexpected error, closing socket connection and attempting
>> reconnect
>> java.net.ConnectException: Connection refused
>> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
>> at
>> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
>> [zk: rhes75:3400(CONNECTING) 0] 2018-07-14 21:08:42,299 [myid:] - INFO
>> [main-SendThread(rhes75:3400):ClientCnxn$SendThread@975] - Opening
>> socket connection to server rhes75/50.140.197.220:3400. Will not attempt
>> to authenticate using SASL (unknown error)
>> 2018-07-14 21:08:42,300 [myid:] - WARN
>> [main-SendThread(rhes75:3400):ClientCnxn$SendThread@1102] - Session 0x0
>> for server null, unexpected error, closing socket connection and attempting
>> reconnect
>>
>> Thanks
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn *
>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>>
>> On Sat, 14 Jul 2018 at 20:34, M. Manna <manme...@gmail.com> wrote:
>>
>>> Can you print out zookeeper admin:
>>>
>>> ./zookeeper-shell.sh rhes75:3400
>>>
>>> And show us here?
>>>
>>> On Sat, 14 Jul 2018, 19:43 Mich Talebzadeh, <mich.talebza...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > I have created an Ubuntu image docker on a physical host running
>>> RHES75.
>>> >
>>> > I installed both Zookeeper and Kafka binaries in this container and
>>> they
>>> > both started OK, I ran my Zookeeper on port 3400 and Kafka running on
>>> port
>>> > 12092
>>> >
>>> > hduser@bbe7e47d5609: /home/hduser> jps
>>> > 7026 QuorumPeerMain
>>> > 7079 Kafka
>>> >
>>> > I can connect from contain to any host as long /etc/hosts has the
>>> relevant
>>> > host information.
>>> >
>>> > I mapped port 3400 and 12092 on the physical host running the
>>> container as
>>> > port forwarding.
>>> >
>>> > [root@rhes75 ~]# docker ps -a
>>> > CONTAINER ID IMAGE COMMAND
>>> > CREATED STATUS PORTS NAMES
>>> > bbe7e47d5609 ubuntu "bash" 28 hours
>>> > ago Up 25 hours testdocker
>>> >
>>> > I did the mapping of ports as follows:
>>> >
>>> > iptables -t nat -A DOCKER -p tcp --dport 4300 -j DNAT --to-destination
>>> > 172.17.0.2:4300
>>> > iptables -t nat -A DOCKER -p tcp --dport 12092 -j DNAT --to-destination
>>> > 172.17.0.2:12092
>>> >
>>> > So if I go to another host and try to telnet to port 4300 on the
>>> docker it
>>> > works
>>> >
>>> >
>>> > hduser@rhes564: /home/hduser/dba/log> telnet rhes75 4300
>>> > Trying 50.140.197.220...
>>> > Connected to rhes75 (50.140.197.220).
>>> > Escape character is '^]'.
>>> >
>>> >
>>> > hduser@rhes564: /home/hduser/dba/log> telnet rhes75 12092
>>> > Trying 50.140.197.220...
>>> > Connected to rhes75 (50.140.197.220).
>>> > Escape character is '^]'.
>>> >
>>> >
>>> > If I stop Zookeeper (running on 4300), the connection will be refused
>>> >
>>> > I created a topic called "test" on a remote host as follows
>>> >
>>> > ${KAFKA_HOME}/bin/kafka-topics.sh --create --zookeeper rhes75:4300
>>> > --replication-factor 1 --partitions 1 --topic test
>>> >
>>> > And if I go into docker I can see that topic on localhost:4300
>>> >
>>> > ${KAFKA_HOME}/bin/kafka-topics.sh --describe --zookeeper
>>> *localhost:4300*
>>> > --topic test
>>> > Topic:test PartitionCount:1 ReplicationFactor:1
>>> Configs:
>>> > Topic: test Partition: 0 Leader: 1 Replicas: 1
>>> > Isr: 1
>>> >
>>> > The problem I see is that I cannot publish messages to the broker on
>>> this
>>> > container
>>> >
>>> > From remote host I start publishing some test prices
>>> > cat ${IN_FILE} | ${KAFKA_HOME}/bin/kafka-console-producer.sh
>>> --broker-list
>>> > rhes75:12092 --topic test --property "parse.key=true" --property
>>> > "key.separator=:"
>>> >
>>> > I get this error that indicates there is an issue talking to broker on
>>> the
>>> > container!
>>> >
>>> > [2018-07-14 19:49:22,838] WARN [Producer clientId=console-producer]
>>> > Connection to node 1 could not be established. Broker may not be
>>> available.
>>> > (org.apache.kafka.clients.NetworkClient)
>>> > [2018-07-14 19:49:22,925] WARN [Producer clientId=console-producer]
>>> > Connection to node 1 could not be established. Broker may not be
>>> available.
>>> > (org.apache.kafka.clients.NetworkClient)
>>> > [2018-07-14 19:49:23,088] WARN [Producer clientId=console-producer]
>>> > Connection to node 1 could not be established. Broker may not be
>>> available.
>>> > (org.apache.kafka.clients.NetworkClient)
>>> > [2018-07-14 19:49:23,491] WARN [Producer clientId=console-producer]
>>> > Connection to node 1 could not be established. Broker may not be
>>> available.
>>> > (org.apache.kafka.clients.NetworkClient)
>>> > [2018-07-14 19:49:24,188] WARN [Producer clientId=console-producer]
>>> > Connection to node 1 could not be established. Broker may not be
>>> available.
>>> > (org.apache.kafka.clients.NetworkClient)
>>> > [2018-07-14 19:49:25,124] WARN [Producer clientId=console-producer]
>>> > Connection to node 1 could not be established. Broker may not be
>>> available.
>>> > (org.apache.kafka.clients.NetworkClient)
>>> > [2018-07-14 19:49:25,326] ERROR Error when sending message to topic
>>> test
>>> > with key: 54 bytes, value: 12 bytes with error:
>>> > (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
>>> > org.apache.kafka.common.errors.TimeoutException: Expiring 10 record(s)
>>> for
>>> > test-0: 1542 ms has passed since batch creation plus linger time
>>> > [2018-07-14 19:49:25,329] ERROR Error when sending message to topic
>>> test
>>> > with key: 54 bytes, value: 12 bytes with error:
>>> > (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
>>> > org.apache.kafka.common.errors.TimeoutException: Expiring 10 record(s)
>>> for
>>> > test-0: 1542 ms has passed since batch creation plus linger time
>>> > [2018-07-14 19:49:25,329] ERROR Error when sending message to topic
>>> test
>>> > with key: 55 bytes, value: 11 bytes with error:
>>> > (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
>>> > org.apache.kafka.common.errors.TimeoutException: Expiring 10 record(s)
>>> for
>>> > test-0: 1542 ms has passed since batch creation plus linger time
>>> >
>>> > I have been working on this for a day or two and have not found any
>>> > resolution yet!
>>> >
>>> > Thanks
>>> >
>>> > Dr Mich Talebzadeh
>>> >
>>> >
>>> >
>>> > LinkedIn *
>>> >
>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> > <
>>> >
>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> > >*
>>> >
>>> >
>>> >
>>> > http://talebzadehmich.wordpress.com
>>> >
>>> >
>>> > *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any
>>> > loss, damage or destruction of data or any other property which may
>>> arise
>>> > from relying on this email's technical content is explicitly
>>> disclaimed.
>>> > The author will in no case be liable for any monetary damages arising
>>> from
>>> > such loss, damage or destruction.
>>> >
>>>
>>
