> On Fri, 15 Feb. 2019, 7:03 am John Ziebro <jzie...@mbsicorp.com wrote: > > > We are planning on implementing Kafka to stream changes from a MariaDB as > > well as a MS SQL Server database into an S3 bucket and [...] > > What are your suggestions for handling this scenario? > > > > Thank you, > > John Z
On 2019-02-15 07:26:30, Liam Clarke spake thus: > Kafka Connect JDBC source can pick up changes based on primary keys and mod > time columns. Debezium consumes the DBs change log directly. [...] > Cheers, > > Liam Clarke One slight clarification for the Debezium connectors: the connector for MariaDB[0] reads the DB's change log directly, but the connector for MS SQL Server[1] does not have the log directly available to it; it instead polls the DB's CDC tables repeatedly. Either way, to Liam's point, debezium would be a good choice for the source data side in your scenario. Take care, -Al [0] https://debezium.io/docs/connectors/mysql/ [1] https://debezium.io/docs/connectors/sqlserver/ Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
