I would have a couple of ideas about Kafka and would love to get your input. If I am totally off, if it is possible today without me noticing or if you can embrace/do disregard the idea - appreciate all your feedback.
Statement 1: A typical scenario for Kafka can be coupling on-premise system with a Kafka based cloud application. Today Kafka requires a direct connection but the on-prem side does allow http only. Kafka Rest is obviously one solution, but it is slow and has overhead. Suggestion: In Kafka Connect add the ability to run connectors on-prem. Each opens one long running http connection with the Kafka server streaming the data. Or maybe a WebSocket as a bridge. Statement 2: For above use cases you need to support security and multi-tenant. Suggestion: Above http connection should validate that a connector has the proper certificate and is entitled to write into a Kafka topic. Statement 3: Connect needs an UI for being configured by a non-Kafka user. Suggestion: The connectors are hosted by a Tomcat which also acts as the configuration, administration and monitoring UI for the on-prem side. I have a few more thoughts about metadata (schema registry), a mapping (Kafka Connect Transform) UI,... but above would be the most pressing points for me. Thanks in advance
