Hi,
Al right, I am running kafka from an Debian Linux operating system – no docker
images are involved. I just can exclude any problems with my producer
application – the following command fails too:
/home/kafka/kafka/bin/kafka-topics.sh --list --bootstrap-server 127.0.0.1:9092
Error while executing topic command :
org.apache.kafka.common.errors.TimeoutException: Call(callName=listTopics,
deadlineMs=1594206614222) timed out at 1594206614223 after 1 attempt(s)
[2020-07-08 13:10:14,227] ERROR java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException: Call(callName=listTopics,
deadlineMs=1594206614222) timed out at 1594206614223 after 1 attempt(s)
at
org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at
org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at
org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at
org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at
kafka.admin.TopicCommand$AdminClientTopicService.getTopics(TopicCommand.scala:333)
at
kafka.admin.TopicCommand$AdminClientTopicService.listTopics(TopicCommand.scala:252)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:66)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException:
Call(callName=listTopics, deadlineMs=1594206614222) timed out at 1594206614223
after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting
for a node assignment.
(kafka.admin.TopicCommand$)
This means, that there is a connection problem with the client, supporting what
you had said. I just deactivated the firewall (ufw in this case), but it does
not look better.
Could you please suggest any options for further investigations?
Is there any logging that could help?
In the kafka log I found the following exception:
2020-07-08 13:09:33,066] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 0ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:09:39,072] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:09:45,075] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:09:51,079] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 0ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:09:57,085] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:03,088] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:09,092] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:14,239] DEBUG [SocketServer brokerId=1] Connection with
/127.0.0.1 disconnected (org.apache.kafka.common.network.Selector)
java.io.EOFException
at
org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:97)
at
org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:448)
at
org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:398)
at
org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580)
at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
at kafka.network.Processor.poll(SocketServer.scala:861)
at kafka.network.Processor.run(SocketServer.scala:760)
at java.lang.Thread.run(Thread.java:748)
[2020-07-08 13:10:15,096] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:21,099] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:27,104] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:33,108] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:39,112] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:45,115] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:51,120] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:10:57,124] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:03,131] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:09,136] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:15,140] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:21,144] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:27,148] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:33,152] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:39,155] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:45,162] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:51,168] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:11:57,172] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:03,175] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:09,181] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:15,187] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:21,193] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:27,199] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 0ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:33,204] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:39,207] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:45,212] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:51,216] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:12:57,220] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:13:03,226] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:13:09,231] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:13:15,236] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:13:21,240] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:13:27,246] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
[2020-07-08 13:13:33,251] DEBUG Got ping response for sessionid:
0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn)
Kind regards,
Sebastian
Von: Ricardo Ferreira <[email protected]>
Gesendet: Dienstag, 7. Juli 2020 17:58
An: [email protected]; [email protected]
Betreff: Re: Problem with replication?!
Given the stack trace you've shared below I can tell that this is not a
replication issue but rather -- your producer is not being able to write
records into the partitions because the brokers that host them are unavailable.
Now, I know that they are indeed running so "unavailable" here means that from
the network perspective your client app is not being able to establish a TCP
connection with them.
Firewall is something you should look into, as well as SELinux, and Docker
networking (if this is running on Docker).
Thanks,
-- Ricardo
On 7/6/20 10:37 AM, [email protected]
<mailto:[email protected]> wrote:
Hi there,
I just have a problem with my kafka brokers, maybe a firewall issue, but I
don’t know. I have got 3 Brokers at three different Servers (each with another
IP) and on the first server running zookeeper:
Server1:9092 (zookeeper:2182)
Server2:9092
Server3:9092
And I have got a topic with a replication factor of three. If I try to publish
new messages to that topic, I got the following error:
Confluent.Kafka.ProduceException`2[System.Byte[],System.String]: Local: Message
timed out
at Confluent.Kafka.Producer`2.ProduceAsync(TopicPartition topicPartition,
Message`2 message, CancellationToken cancellationToken)
at
FluentSoftware.EventSourcing.KafkaProducerEventPublisher.PublishEvents[TEvent](String
topic, IEnumerable`1 eventsToApply)
at FluentSoftware.EventSourcing.CommandBus.SendAsync[TCommand](TCommand
command)
at
Heliprinter.Mediator.Communication.OrderService.Controllers.OrderController.Post(PostOrderViewModel
newOrder) in
C:\agent\_work\15\s\Source\Services\Heliprinter.Mediator.Communication.OrderService\Controllers\OrderController.cs:line
55
and the REST API responses with an Gateway timeout. Could anyone tell me
* If this error could be an replication issue?
* How I can debug these issue?
* And where I can find replication details in the logs?
Kind regards,
Sebastian.
