Re:Re: Flink caching mechanism

2024-01-11 Thread Xuyang
Hi, Вова. Junrui is right. As far as I know, every time a SQL is re-executed, Flink will regenerate the plan, generate jobgraph, and execute the job again. There is no cache to speed up this process. State beckend is used when your job is stopped and you want to continue running from the state b

Re: Flink caching mechanism

2024-01-11 Thread Junrui Lee
Hi Вова In Flink, there is no built-in mechanism for caching SQL query results; every query execution is independent, and results are not stored for future queries. The StateBackend's role is to maintain operational states within jobs, such as aggregations or windowing, which is critical for ensur

Flink caching mechanism

2024-01-11 Thread Вова Фролов
Hi Everyone, I'm currently looking to understand the caching mechanism in Apache Flink in general. As part of this exploration, I have a few questions related to how Flink handles data caching, both in the context of SQL queries and more broadly. When I send a SQL query for example to PostgreS