[
https://issues.apache.org/jira/browse/KAFKA-14494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17648241#comment-17648241
]
Kirk True commented on KAFKA-14494:
-----------------------------------
[~taiga-elephant] - thanks for finding and reporting this! I appreciate all the
context and background in the Jira :)
Were you planning to submit a patch to fix?
Thanks!
> Kafka Java client can't send data when behind SOCKS proxy - while native
> client can
> -----------------------------------------------------------------------------------
>
> Key: KAFKA-14494
> URL: https://issues.apache.org/jira/browse/KAFKA-14494
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 3.3.1
> Reporter: Oleg Zhovtanyuk
> Priority: Major
> Attachments: client-java.tar.gz, client-nodejs.tar.gz,
> kafka-client-java-direct.log.gz, kafka-client-java-socks.log.gz,
> kafka-client-node-socks.log.gz
>
>
> When Kafka Java client sits behind the SOCK5 proxy, it connects to the
> cluster, gets the list of brokers, but enters the infinite loop trying to
> detect the least loaded broker.
> To the contrary, NodeJS client (a wrapper for librdkafka) with exactly the
> same setup, proceeds further to the binary data exchange.
>
> The TRACE logs for Java client (with and without proxy) and for the NodeJS
> client (with proxy) are attached.
> Diff'ing the logs highlights the issue.
>
> {code:java}
> grep ' Found least loaded connecting node' kafka-client-java-direct.log | wc
> -l
> 28
> grep ' Found least loaded connecting node' kafka-client-java-socks.log | wc
> -l
> 434
> {code}
>
>
> Some debugging pointed to the place where it stucks -
> [https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/NetworkClient.java#L712]
> Call stack:
>
> {code:java}
> leastLoadedNode:672, NetworkClient (org.apache.kafka.clients)
> leastLoadedNode:141, ConsumerNetworkClient
> (org.apache.kafka.clients.consumer.internals)
> lookupCoordinator:300, AbstractCoordinator
> (org.apache.kafka.clients.consumer.internals)
> ensureCoordinatorReady:264, AbstractCoordinator
> (org.apache.kafka.clients.consumer.internals)
> ensureCoordinatorReady:240, AbstractCoordinator
> (org.apache.kafka.clients.consumer.internals)
> coordinatorUnknownAndUnreadySync:492, ConsumerCoordinator
> (org.apache.kafka.clients.consumer.internals)
> poll:524, ConsumerCoordinator (org.apache.kafka.clients.consumer.internals)
> updateAssignmentMetadataIfNeeded:1276, KafkaConsumer
> (org.apache.kafka.clients.consumer)
> poll:1240, KafkaConsumer (org.apache.kafka.clients.consumer)
> poll:1220, KafkaConsumer (org.apache.kafka.clients.consumer)
> main:32, AppConsumer (com.example.test.kafka)
> {code}
>
> I'm ready to fix the issue, just need some initial guidance...
>
>
> Both test clients are attached too.
> As Java client ignores JVM proxy options (it uses custom socket factory),
> I've employed
> [socksify|https://manpages.debian.org/testing/dante-client/socksify.1.en.html]
> native library, that forcefully forwards all socket calls to the proxy
> server.
> It can be installed with the package manager on any UNIX, e.g. for Ubuntu
> {code:java}
> sudo apt-get install dante-client{code}
>
> Java client can be run as
> * Kafka cluster URI + credentials -> 'client.properties',
> *
> {code:java}
> mvn clean package{code}
> *
> {code:java}
> ./run-with-socksify.sh{code}
> NodeJS client can be run as
> * Kafka cluster URI + credentials -> 'consumer.js',
> *
> {code:java}
> npm install .{code}
> *
> {code:java}
> ./run-with-socksify.sh {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
