Hi, A WindowAll is executed in a single task. If you sort the data before the window, the sorting must also happen in a single task, i.e., with parallelism 1. The reasons is that an operator somewhat randomly merges multiple input partitions. So even if each input partition is sorted, the merging will result in out-of-order data.
Best, Fabian Am Sa., 2. Feb. 2019 um 17:11 Uhr schrieb morin.david....@gmail.com < morin.david....@gmail.com>: > Hello, > > I use Watermarks and a function to sort events at the end of my pipeline. > I've used this tutorial to sort my data: > https://training.da-platform.com/exercises/carSort.html > SingleOutputStreamOperator<XXXX> sortStream = streamKeyed.process(new > SortEventFunction()).. > > Then I want to apply a Window and use AggregateFunction to obtain a group > of data. Thus when a trigger is launched, I can push all these data to my > backend at the same time (with puts method for Hbase for example) > But the order here must be guaranteed. > Can I use a windowAll on that stream ? > sortStream.windowAll(... > > Thanks in advance > David > > >
