As an aside, I conceptually view temporal table joins to be semantically equivalent to look up table joins. They are just two different ways of consuming the same data.
Seth On Mon, Jul 6, 2020 at 8:56 AM Seth Wiesman <sjwies...@gmail.com> wrote: > Hi Leonard, > > Regarding DELETE operations I tend to have the opposite reaction. I spend > a lot of time working with production Flink users across a large number of > organizations and to say we don't support temporal tables that include > DELETEs will be a blocker for adoption. Even organizations that claim to > never delete rows still occasionally due so per GDPR requests or other > regulations. > > I actually do think users will understand the limitations. Flink today has > a very clear value proposition around correctness, your results are as > correct as the input data provided. This does not change under support for > DELETE records. Flink is providing the most correct results possible based > on the resolution of the fields as generated by 3rd party systems. As > Debezium and other CDC libraries become more accurate, so will Flink. > > Seth > > On Fri, Jul 3, 2020 at 11:00 PM Leonard Xu <xbjt...@gmail.com> wrote: > >> Hi, Konstantin >> >> . Would we support a temporal join with a changelog stream with >> event time semantics by ignoring DELETE messages or would it be completed >> unsupported. >> >> >> I don’t know the percentage of this feature in temporal scenarios. >> >> Comparing to support the approximate event time join by ignoring DELETE >> message or by extracting an approximate event time for DELET message, I’m >> not sure is this acceptable for user even if we have explained the >> limitation of approximate event time join, I tend to do not support this >> feature, because we can not ensure the semantic of event time and it may >> lead an incorrect result for user in some scenarios. >> >> If the percentage is highly enough and most user cases can accept the >> approximate event time, I'm ok to support it for usability although it >> doesn’t implements the event time semantic strictly. >> >> Cheers, >> Leonard Xu >> >> >>
