Hi, I have a scenario in which a streaming pipeline should read update messages from MySQL binlog (through Debezium). To implement this pipeline using Beam, I understand there is a KafkaIO which I can use. But I also want to support a local mode in which there is no Kafka and the messages are directly consumed using embedded Debezium because this is a much simpler architecture (no Kafka, ZooKeeper, and Kafka Connect).
I did a little bit of search and it seems there is no IO connector for Debezim, hence I have to implement one following this guide <https://beam.apache.org/documentation/io/developing-io-java/>. I wonder: 1) Does this approach make sense or is it better to rely on Kafka even for the local single machine use case? 2) Beside the above guide, is there any simple example IO that I can follow to implement the UnboundedSource/Reader? I have looked at some examples here <https://github.com/apache/beam/tree/master/sdks/java/io> but was wondering if there is a recommended/simple one as a tutorial. Thanks -B P.S. If this is better suited for dev@, please feel free to move it to that list.
