Re: Intial Data Load

2020-05-31 Thread Liam Clarke-Hutchinson
Hi Suresh, So to achieve what you're after, you can use Kafka Connect's JDBC source with a log compacted topic. It can pick up on new entities (a new primary id) or updated entities (a mod_time column). If you want just the deltas, you want to look into Debezium. Kind regards, Liam Clarke-Hutch

Re: Intial Data Load

2020-05-31 Thread Suresh Chidambaram
Hi Liam, Thank you for the reply. One Time Load from database to Kafka Topic. Kindly let me know if more input required. Thanks C Suresh On Sunday, May 31, 2020, Liam Clarke-Hutchinson wrote: > Hi Suresh, > > I'm afraid you're not giving us much to work with. Initial/delta load from > what? >

Re: Intial Data Load

2020-05-31 Thread Liam Clarke-Hutchinson
Hi Suresh, I'm afraid you're not giving us much to work with. Initial/delta load from what? Thanks, Liam Clarke-Hutchinson On Sun, 31 May 2020, 4:50 pm Suresh Chidambaram, wrote: > Hi Team, > > Could someone help me with my request below? > > Thanks > C Suresh > > On Saturday, May 30, 2020, S

Re: Intial Data Load

2020-05-30 Thread Suresh Chidambaram
Hi Team, Could someone help me with my request below? Thanks C Suresh On Saturday, May 30, 2020, Suresh Chidambaram wrote: > Hi Team, > > My requirement is that I have to perform an One Time Data Load(intial > load) to the topic, later I have to perform the delta load to the topic. So > could

Intial Data Load

2020-05-29 Thread Suresh Chidambaram
Hi Team, My requirement is that I have to perform an One Time Data Load(intial load) to the topic, later I have to perform the delta load to the topic. So could someone guide me how to achieve this requirement in Confluent Kafka/Apache Kafka? Thanks C Suresh