Re: Source Kafka and Sink Hive managed tables via Flink Job

2019-07-03 Thread Bowen Li
BTW, I'm adding user@ mailing list since this is a user question and should be asked there. dev@ mailing list is only for discussions of Flink development. Please see https://flink.apache.org/community.html#mailing-lists On Wed, Jul 3, 2019 at 12:34 PM Bowen Li wrote: > Hi Youssef, > > You nee

Re: Source Kafka and Sink Hive managed tables via Flink Job

2019-07-03 Thread Bowen Li
Hi Youssef, You need to provide more background context: - Which Hive sink are you using? We are working on the official Hive sink for community and will be released in 1.9. So did you develop yours in house? - What do you mean by 1st, 2nd, 3rd window? You mean the parallel instances of the same

Source Kafka and Sink Hive managed tables via Flink Job

2019-07-03 Thread Youssef Achbany
Dear all, I'm working for a big project and one of the challenge is to read Kafka topics and copy them via Hive command into Hive managed tables in order to enable ACID HIVE properties. I try it but I have a issue with back pressure: - The first window read 20.000 events and wrote them in Hive ta