Thank you for the information! This is a very interesting topic. In last year’s Flink Forward conference, there was an interesting talk about Hologres. https://www.flink-forward.org/sf-2020/conference-program#data-warehouse--data-lakes--what-s-next- <https://www.flink-forward.org/sf-2020/conference-program#data-warehouse--data-lakes--what-s-next->
What do you think? > On Nov 4, 2021, at 10:16 PM, Caizhi Weng <tsreape...@gmail.com> wrote: > > Hi! > > Flink is a distributed, stateful streaming data-flow engine (with > optimizations for batch or olap jobs too) and it currently is not shipped > with any storage system. It needs to be used along with external storage / > computation system like hdfs, hive, kafka, iceberg, etc. to build a data > warehouse [1] or data lake [2]. > > Most use cases [3] of Flink includes continuous streaming or long-running > batch analytical jobs (which are all levels in a data warehouse, including > etl jobs) so I can't say Flink is specialized to etl or olap. But as for olap > there are a few companies currently using Flink as their olap execution > engine. If you're interested, you can keep an eye on Flink Forward Asia this > year, in which two talks are about using Flink as a olap execution engine in > production (search olap in [4] for more detail). > > [1] > https://www.alibabacloud.com/blog/flink-is-attempting-to-build-a-data-warehouse-simply-by-using-a-set-of-sql-statements_596346 > > <https://www.alibabacloud.com/blog/flink-is-attempting-to-build-a-data-warehouse-simply-by-using-a-set-of-sql-statements_596346> > [2] > https://www.alibabacloud.com/blog/building-an-enterprise-level-real-time-data-lake-based-on-flink-and-iceberg_597755 > > <https://www.alibabacloud.com/blog/building-an-enterprise-level-real-time-data-lake-based-on-flink-and-iceberg_597755> > [3] https://flink.apache.org/usecases.html > <https://flink.apache.org/usecases.html> > [4] https://flink-forward.org.cn/#agenda > <https://flink-forward.org.cn/#agenda> > Ww J <junww2...@gmail.com <mailto:junww2...@gmail.com>> 于2021年11月5日周五 > 下午12:49写道： > Thanks. Can Flink replace the popular OLAP databases, for example, AWS > redshift? > It seems to me that generally Flink is used as ETL for OLAP. > >> On Nov 4, 2021, at 9:33 PM, Caizhi Weng <tsreape...@gmail.com >> <mailto:tsreape...@gmail.com>> wrote: >> >> Hi! >> >> Yes you can. Note that it is recommended to run Flink in session cluster >> mode (instead of per job mode) to minimize distribution and scheduling time >> for each OLAP query. >> >> Ww J <junww2...@gmail.com <mailto:junww2...@gmail.com>> 于2021年11月5日周五 >> 下午12:30写道： >> Hi, >> >> Can Flink be used for OLAP queries? >> >> Thanks, >> >> Jack >
