Thanks Per for the feedback. My comments below.
1) There is a load balancer in front of the cluster.
2) Kafka does run in a cluster mode. I was referring to the producer that
doesn't. Furthermore, the data sources are not heterogeneous and we cannot
install anything on them. All we can do have th
Hi Clay,
I agree with Gwen in thinking that you might want to take a second look at
streaming protobuf data to Kafka and then having connectors read that from
Kafka. To address the issues in order:
1. You say you have hundreds of machines sending data, but if you run a
connector that is not tied
Well I guess there is one: https://github.com/dhanuka84/kafka-connect-tcp
Maybe you can use or build on top of that.
On 05/07/17 05:45, Gwen Shapira wrote:
I don't remember seeing one. There is no reason not to write one (let us
know if you do, so we can put it on the connector hub!).
Few thing
Hello Gwen,
Thanks for the reply. My comments/answers inline.
1. Connectors that listen on sockets typically run in stand-alone mode, so
they can tied to a specific machine (in distributed mode, connectors can
move around).
[Clay:] Even if the connectors move around, they can still listen to a
sp
I don't remember seeing one. There is no reason not to write one (let us
know if you do, so we can put it on the connector hub!).
Few things:
1. Connectors that listen on sockets typically run in stand-alone mode, so
they can tied to a specific machine (in distributed mode, connectors can
move aro
Hello All,
I'd appreciate your help with the following questions.
1) Is there kafka connect for listening to tcp sockets?
2) If, can the messages be in protobuf, with each messaged prefixed with
the length of the message?
thanks
Clay
