Hello James, I'm not aware of anything like that for Kafka, but you can use MirrorMaker for network segmentation. With this approach you have one Kafka cluster in each segment and a MM cluster in the more privileged segment. You don't need to expose the privileged segment at all -- you just need to let MM reach the external segment(s).
Ryanne On Tue, Mar 19, 2019, 10:20 AM James Grant <ja...@queeg.org> wrote: > Hello, > > We would like to expose a Kafka cluster running on one network to clients > that are running on other networks without having to have full routing > between the two networks. In this case these networks are in different AWS > accounts but the concept applies more widely. We would like to access Kafka > over a single (or very few) host names. > > In addition we would like to filter incoming messages to enforce some level > of data quality and also impose some access control. > > A solution we are looking into is to provide a Kafka protocol level proxy > that presents to clients as a single node Kafka cluster holding all the > topics and partitions of the cluster behind it. This proxy would be able to > operate in a load balanced cluster behind a single DNS entry and would also > be able to intercept and filter/alter messages as they passed through. > > The advantages we see in this approach over the HTTP proxy is that it > presents the Kafka protocol whilst also meaning that we can use a typical > TCP level load balancer that it is easy to route connections to. This means > that we continue to use native Kafka clients. > > Does anything like this already exist? Does anybody think it would useful? > Does anybody know of any reason it would be impossible (or a bad idea) to > do? > > James Grant > > Developer - Expedia Group >
