Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

2023-01-05 Thread Leonard Xu
Hi, Adam & Márton Thanks for bringing the discussion here. The Flink CDC project provides the Oracle CDC Connector, which can be used to capture historical and transaction log data from the Oracle database and ingest it into Flink. In the latest version 2.3, the Oracle CDC Connector already

Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

2023-01-04 Thread Adam Leszczyński
H Márton, Thank you very much for your answer. The point with Kafka makes sense. It offers huge bag of potential connectors that could be used. But … not everybody wants or needs Kafka. This brings additional architectural complication and delays, which might not be acceptable by everybody. Tha

Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

2023-01-04 Thread Márton Balassi
(cc Leonard) Hi Adam, >From an architectural perspective if you land the records to Kafka or other message broker Flink will be able to process them, at this point I do not see much merit trying to circumvent this step. There is a related project in the Flink space called CDC connectors [1], I hi

CDC from Oracle database reading directly logs - integration with OpenLogReplicator

2023-01-03 Thread Adam Leszczyński
Hi Flink Team, I’m the author of OpenLogReplictor - open source parser of Oracle redo logs which allows to send transactions to some message bus. Currently the sink that is implemented is just text file or Kafka topic. Also transactions can be sent using plain tcp connection or some message qu