Hello everybody out there using Apache Kafka, I have been designing a (free and open) way to connect event streams of different event brokers, including of course Apache Kafka. The motivation is to facilitate cross-organizational event flow connections over the Internet.
One can use MirrorMaker 2 when needing to connect topics across Apache Kafka instances. However, this may not be the best approach when the instances are owned by different organizations. On the other hand, if we need to connect events from an Apache Kafka instance to a different technology (i.e. Apache Pulsar, RabbitMQ) the only alternative is to develop your own stream application. The idea of an Event Streaming Open Network is to have an open framework for the discovery, name resolution and overall communication of event streams. The participants of this network could use whatever event broker implementation they want. They would be able to consume event flows independently of the event broker used by the peer, and vice versa. Imagine you can share an event flow just by sending your peer an URI like flow://temperature.office.mycompany.com and have the network discover the connection details. You would avoid hardcoded details like bootstrap_servers, topic name, tls, etc. If in the future you need to move an event flow to a different instance/technology, the consumers and producers would not need any refactoring (as long as they resolve the event flow using its URI). Here is a diagram of the interaction between two different network participants: [image: image.png] You can take a look at the full specification in the following IETF Internet Draft: https://github.com/syndeno/draft-spinella-event-streaming-open-network Finally, I have built an initial implementation of the network components. I will be posting it to this mailing list in the following weeks in case anyone is interested in seeing how it works. Any comment or question is extremely welcome! Thanks, Emiliano -- _LEGAL NOTICE: The content of this email message, including the attached files, is confidential and is protected by article 18.3 of the Spanish Constitution, which guarantees the secrecy of communications. If you receive this message in error, please contact the sender to inform them of this fact, and do not broadcast its content or make copies. _ _*** This message has been verified with removal tools for viruses and malicious content *** _ _This legal notice has been automatically incorporated into the message. _ *---------------------------------------------* *AVISO LEGAL: El contenido de este mensaje de correo electrónico, incluidos los ficheros adjuntos, es confidencial y está protegido por el artículo 18.3 de la Constitución Española, que garantiza el secreto de las comunicaciones. Si usted recibe este mensaje por error, por favor póngase en contacto con el remitente para informarle de este hecho, y no difunda su contenido ni haga copias. * _*** Este mensaje ha sido verificado con herramientas de eliminación de virus y contenido malicioso *** _ _Este aviso legal ha sido incorporado automáticamente al mensaje._
