Re: Use Flink for OLAP

2021-11-04 Thread Ww J
Thank you for the information! This is a very interesting topic. In last year’s Flink Forward conference, there was an interesting talk about Hologres. https://www.flink-forward.org/sf-2020/conference-program#data-warehouse--data-lakes--what-s-next-

Re: Use Flink for OLAP

2021-11-04 Thread Caizhi Weng
Hi! Flink is a distributed, stateful streaming data-flow engine (with optimizations for batch or olap jobs too) and it currently is not shipped with any storage system. It needs to be used along with external storage / computation system like hdfs, hive, kafka, iceberg, etc. to build a data wareho

Re: Use Flink for OLAP

2021-11-04 Thread Ww J
Thanks. Can Flink replace the popular OLAP databases, for example, AWS redshift? It seems to me that generally Flink is used as ETL for OLAP. > On Nov 4, 2021, at 9:33 PM, Caizhi Weng wrote: > > Hi! > > Yes you can. Note that it is recommended to run Flink in session cluster mode > (instead o

Re: Use Flink for OLAP

2021-11-04 Thread Caizhi Weng
Hi! Yes you can. Note that it is recommended to run Flink in session cluster mode (instead of per job mode) to minimize distribution and scheduling time for each OLAP query. Ww J 于2021年11月5日周五 下午12:30写道: > Hi, > > Can Flink be used for OLAP queries? > > Thanks, > > Jack >