Re: Source reinterpretAsKeyedStream

2019-04-05 Thread Fabian Hueske
Hi, Konstantin is right. reinterpreteAsKeyedStream only works if you call it on a DataStream that was keyBy'ed before (with the same parallelism). Flink cannot reuse the partioning of another system like Kafka. Best, Fabian Adrienne Kole schrieb am Do., 4. Apr. 2019, 14:33: > Thanks a lot for

Re: Source reinterpretAsKeyedStream

2019-04-04 Thread Adrienne Kole
Thanks a lot for the replies. Below I paste my code: DataStreamSource source = env.addSource(new MySource()); KeyedStream keyedStream = DataStreamUtils.reinterpretAsKeyedStream(source, new DummyKeySelector(), TypeInformation.of(Integer.class) ); keyedStream.timeWindow(Tim

Re: Source reinterpretAsKeyedStream

2019-04-03 Thread Konstantin Knauf
Hi Adrienne, you can only use DataStream#reinterpretAsKeyedStream on a stream, which has previously been keyed/partitioned by Flink with exactly the same KeySelector as given to reinterpretAsKeyedStream. It does not work with a key-partitioned stream, which has been partitioned by any other proces

Re: Source reinterpretAsKeyedStream

2019-03-29 Thread Rong Rong
Hi Adrienne, I think you should be able to reinterpretAsKeyedStream by passing in a DataStreamSource based on the ITCase example [1]. Can you share the full code/error logs or the IAE? -- Rong [1] https://github.com/apache/flink/blob/release-1.7.2/flink-streaming-java/src/test/java/org/apache/fl

Source reinterpretAsKeyedStream

2019-03-29 Thread Adrienne Kole
Dear community, I have a use-case where sources are keyed. For example, there is a source function with parallelism 10, and each instance has its own key. I used reinterpretAsKeyedStream to convert source DataStream to KeyedStream, however, I get an IllegalArgument exception. Is reinterpretAsKeyed