Dear all, I'm working for a big project and one of the challenge is to read Kafka topics and copy them via Hive command into Hive managed tables in order to enable ACID HIVE properties.
I try it but I have a issue with back pressure: - The first window read 20.000 events and wrote them in Hive tables - The second, third, ... send only 100 events because the write in Hive take more time than the read of a Kafka topic. But writing 100 events or 50.000 events takes +/- the same time for Hive. Someone have already do this source and sink? Could you help on this? Or have you some tips? It seems that defining a size window on number of event instead time is not possible. Is it true? Thank you for your help Youssef -- ♻ Be green, keep it on the screen
