Hi Ralph, can you explain a bit more? When you say "barriers" you should be referring to the checkpoints, but from your description seems more like watermarks. What functionality is supported in Flink and not Flink SQL? In terms of watermarks, there were a few shortcomings between the two APIs which are addressed in the upcoming 1.18 release https://cwiki.apache.org/confluence/display/FLINK/FLIP-296%3A+Extend+watermark-related+features+for+SQL What operations are you running and you see incorrect results?
Best On Wed, Oct 25, 2023 at 9:51 PM Ralph Matthias Debusmann < matthias.debusm...@gmail.com> wrote: > Hi, > > one question - it seems that "barriers" are perfectly supported by Flink, > but not yet supported in Flink SQL. > > When I e.g. do a UNION of two views derived from one source table fed by > Kafka, I get thousands of intermediate results which are incorrect (the > example I am using is this one: > https://www.scattered-thoughts.net/writing/internal-consistency-in-streaming-systems/), > and I can only be sure to get the correct result if I stop producing new > messages into the source table. > > 1) Is my understanding correct that barriers are not implemented for Flink > SQL? > 2) Why is it not implemented/is this on the roadmap? > > Best, Ralph > >
