Hey Timo, Thanks a lot for this answer! I was mostly using the DataStream API, so that's good to know the difference. I have followup questions then, I will be glad for clarification:
1) So, for the SQL Join operator, is the *partition *the parallel instance of operator or is it the table partitioning as defined by *partitionBy ??* 2) Assuming that this is instance of parallel operator, does this mean that we need output from ALL operators so that the watermark progresses and the output is generated? Best Regards, Dom. wt., 24 mar 2020 o 10:01 Timo Walther <twal...@apache.org> napisał(a): > Hi Dominik, > > the big conceptual difference between DataStream and Table API is that > record timestamps are part of the schema in Table API whereas they are > attached internally to each record in DataStream API. When you call > `y.rowtime` during a stream to table conversion, the runtime will > extract the internal timestamp and will copy it into the field `y`. > > Even if the timestamp is not internally anymore, Flink makes sure that > the watermarking (which still happens internally) remains valid. > However, this means that timestamps and watermarks must already be > correct when entering the Table API. If they were not correct before, > they will also not trigger time-based operations correctly. > > If there is no output for a parallelism > 1, usually this means that one > source parition has not emitted a watermark to have progress globally > for the job: > > watermark of operator = min(previous operator partition 1, previous > operator partition 2, ...) > > I hope this helps. > > Regards, > Timo > > > On 19.03.20 16:38, Dominik Wosiński wrote: > > I have created a simple minimal reproducible example that shows what I > > am talking about: > > https://github.com/DomWos/FlinkTTF/tree/sql-ttf > > > > It contains a test that shows that even if the output is in order which > > is enforced by multiple sleeps, then for parallelism > 1 there is no > > output and for parallelism == 1, the output is produced normally. > > > > Best Regards, > > Dom. > >
