Hi, Nirmal Flink SQL is standard ANSI SQL and extends upon it. Flink SQL provides rich Join and Aggregate syntax including Regular Streaming Join, Interval Join, Temporal Join, Lookup Join[2], Window Join[3], unbounded group aggregate[4] and window aggregate[5], and so on. Theoretically, it can support your complex query needs. You need to spend some time learning these syntaxes so that you can see which ones meet your needs from a business point of view!
[1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/tableapi/ [2] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/ [3] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/window-join/ [4] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/group-agg/ [5] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/window-agg/ Best, Ron Nirmal Chhatrala <nirmal.chhatr...@gmail.com> 于2023年8月28日周一 04:36写道： > Hello! > > We have a use case requirement to implement complex joins and aggregation > on multiple sql tables. Today, it is happening at SQLServer level which is > degrading the performance of SQLServer Database. > > Is it a good idea to implement it through Apache Flink Table API for > real-time data joins? > > ie: there are 10 different sql tables with complex join queries with 10 > different conditions. (Approx. - The accumulative 10 tables size is ~100GB > and could grow in future). We don't have experience with Flink Table API > and never measured the performance on such a huge dataset. > > Looking for your advice and best alternative if it is not suitable with > Apache Flink Table API. > > Thanks! >
