We have many app logs on our app server  and want to parse the logs to structed 
table format and then sink to hive.
Seems it is good to use batch mode. The app log is hourly compressed and it is 
convenience to do partitioning.

We want to use streaming mode. Tail the app logs to Kafka,  then use flink to 
read kafka topic  and then sink to Hive.
I have several questions.

1  Is there any flink-hive-connector that i can use to write to hive 
streamingly?
2  Since HDFS is not friendly to frequently append and hive's data is stored to 
hdfs,  is it  OK if the throughput is high? 

Thanks,
Lei



wangl...@geekplus.com.cn 

Reply via email to