As my company is also a heavy user of Flink SQL, the state migration story
is very important to us.
I as well believe that adding new fields should start to accumulate state
from the point in time of the change forward.
Is anyone actively working on this? Is there anyway to get involved?
On Tue,
Thanks for JING & Kurt's reply. I think we prefer to choose the option (a)
that will not take the history data into account.
IMO, if we want to process all the historical data, we have to store the
original data, which may be a big overhead to backend. But if we just
aggregate after the new adde
What kind of expectation do you have after you add the "max(a)" aggregation:
a. Keep summing a and start to calculate max(a) after you added. In other
words, max(a) won't take the history data into account.
b. First process all the historical data to get a result of max(a), and
then start to compu
Hi aitozi,
This is a popular demand that many users mentioned, which appears in user
mail list for several times.
Unfortunately, it is not supported by Flink SQL yet, maybe would be solved
in the future. BTW, a few company try to solve the problem in some
specified user cases on their internal Flin
