Hi, all. I am trying to produce on my local dev box to an ec2 broker via
an ec2 jump server. I have tried with the following topology below using
port forwarding w/no success:
*Setup*
local workstation: local
EC2 Jump Server: server1
EC2 Broker : server2
If I ssh from local to server2 I do so like this:
ssh -o StrictHostKeyChecking=no -o ProxyCommand="ssh -W %h:%p ubunt@server1"
> ubuntu@server2
What I would like to do is instead of simply ssh, I would like to produce
from local to server2.
I have tried this via port forwarding:
ssh -v -Nntf -o ProxyCommand="ssh -W %h:%p ubuntu@server1" -L
> 9092:localhost:9092 ubuntu@server2
> Server2 / broker / server.properties file has:
> listeners=PLAINTEXT://0.0.0.0:9092
> advertised.listeners=PLAINTEXT://server2:9092
My producer on local is writing to localhost:9092
When I try to write,
val message = new ProducerRecord[String, String]("topic", *key*, json)
> val sent = producer.send(message)
> val metadata = sent.get()
>
It fails on the sent.get() line with this erro:
Exception in thread "main" java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
> topic-0: 30036 ms has passed since batch creation plus linger time
> at
> org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:70)
There are no errors in the broker server.log.
The ssh verbose output looks like this when starting:
debug1: Authentication succeeded (publickey).
> Authenticated to server2 (via proxy).
> debug1: Local connections to LOCALHOST:9092 forwarded to remote address
> localhost:9092
> debug1: Local forwarding listening on ::1 port 9092.
> debug1: channel 0: new [port listener]
> debug1: Local forwarding listening on 127.0.0.1 port 9092.
> debug1: channel 1: new [port listener]
> debug1: Requesting no-more-sessi...@openssh.com
> debug1: forking to background
> debug1: Entering interactive session.
> debug1: pledge: proc
and when in code the producer sends the message I see:
> debug1: Connection to port 9092 forwarding to localhost port 9092
> requested.
> debug1: channel 2: new [direct-tcpip]
> debug1: channel 2: free: direct-tcpip: listening port 9092 for localhost
> port 9092, connect from 127.0.0.1 port 64791 to 127.0.0.1 port 9092,
> nchannels 3
Has anyone had success trying this scenario before? Any suggestions on
where else to look where the error is happening?
Thank you very much!
Robert
