RE: Live data streaming from Oracle to oracle using Kafka

2019-06-11 Thread Kailash Kota
Hi Joe, I understand Oracle Golden Gate is a data replication tool which uses log-based technology to stream all changes to a database from source, to target. Can you please help me in understand what is the role of Kafka after the data is provided to it by OGG ? Also if we just need to da

Re: Live data streaming from Oracle to oracle using Kafka

2019-06-11 Thread Joe Ammann
Hi Kailash On 6/11/19 9:24 AM, Kailash Kota wrote: > I understand Oracle Golden Gate is a data replication tool which uses > log-based technology to stream all changes to a database from source, to > target. Can you please help me in understand what is the role of Kafka after > the data is prov

RE: Repeating UNKNOWN_PRODUCER_ID errors for Kafka streams applications

2019-06-11 Thread Pieter Hameete
Hi Guozhang, Thanks that clarifies a lot. The segment size or segment ms is not the issue here then, changing these properties does not make the errors go away either 😊 Regarding the proposed solutions: 1) We use EOS, so the commit interval is defaulted to 100 ms in our case. We don't want to

Re: Repeating UNKNOWN_PRODUCER_ID errors for Kafka streams applications

2019-06-11 Thread Jonathan Santilli
Hello, we currently see the same WARN logs when the App is processing low rate of records, as soon the app starts processing a high volume of records, those WARN stop showing in the logs. According to other email threads, this should be fixed with https://issues.apache.org/jira/browse/KAFKA-7190.