Re: Using CustomPartitionerWrapper with KeyedStream

2016-08-12 Thread Philippe Caparroy
Hi Max, Thanks for the answer. I needed to ensure that in a parallel window operation (which relies on a KeyedStream) each partition contains a single key, in the output stream of the window. I can obtain this using a customPartitioner just after the window, but relying on the partitioner of th

Re: Using CustomPartitionerWrapper with KeyedStream

2016-08-12 Thread Maximilian Michels
Hi Philippe, There is no particular reason other than hash partitioning is a sensible default for most users. It seems like this is rarely an issue. When the number of keys is close to the parallelism, having idle partitions is usually not a problem due to low data volume. I see that it could be a

Using CustomPartitionerWrapper with KeyedStream

2016-08-12 Thread Philippe Caparroy
Hi there, It seems not possible to use some custom partitioner in the context of the KeyedStream, without modifying the KeyedStream. protected DataStream setConnectionType(StreamPartitioner partitioner) { throw new UnsupportedOperationException("Cannot override partitioning for