Re: Testing DataStreams

2019-10-07 Thread Chesnay Schepler
Are you asking how the elements from the source are distributed across the subtasks of the next operation? That differs a bit across operations; in this case (a map/sink after a source) AFAIK they are distributed in a round-robin manner. On 07/10/2019 11:26, Dominik Wosiński wrote: Thanks a

Re: Testing DataStreams

2019-10-07 Thread Dominik Wosiński
Thanks a lot for Your answer here Chesnay. I have one more question, since the `fromElements` is creating the stream with parallelism 1, and I can see that the env created for my local machine has a default parallelism == 12. So I assume that the entries from the stream are propagated for the first

Re: Testing DataStreams

2019-10-07 Thread Chesnay Schepler
I've filed FLINK-14335 for fixing the java example. On 07/10/2019 11:08, Chesnay Schepler wrote: Are you referring to "ExampleIntegrationTest"? If so, then I'm afraid this test is slightly misleading since the order isn't guaranteed in this case. 1) As long as the parallelism of the sink is 1

Re: Testing DataStreams

2019-10-07 Thread Chesnay Schepler
Are you referring to "ExampleIntegrationTest"? If so, then I'm afraid this test is slightly misleading since the order isn't guaranteed in this case. 1) As long as the parallelism of the sink is 1 the elements should arrive in order. 2) The order is maintained if parallelism=1 since elements

Testing DataStreams

2019-10-02 Thread Dominik Wosiński
Hello, I have a question, since I am observing quite weird behavior. In the documentation[1] the example of FlinkMiniCluster usage, shows that we can expect the results to appear in the same order as they were injected to the stream by use of *fromElements(). *I mean that Java version of the code i