Re: improve the performance of flink sql job which lookup 40+ table

2020-07-27 Thread Jark Wu
Hi, Yes, currently, multiple lookup join is not parallel and execute one by one. Async lookup + cache is the suggested way to improve performance. If the lookup tables are not large, you can also implement a ALL cache for the LookupTableSource to cache all the data in the database, and reload peri

improve the performance of flink sql job which lookup 40+ table

2020-07-27 Thread snack white
HI: My flink version is 1.10 use per-job mode , my sql like ``` select column1, t2.xx2, t3.xx3,t4.xx4 … t40.xx40 from main_table left join lookup_1 FOR SYSTEM_TIME AS OF t1.proc_time AS t2 on t1.xx= t2.xx left join lookup_2 FOR SYSTEM_TIME AS OF t1.proc_time AS t3 on t1.xx=