Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-15 Thread Sathi Chowdhury
I have redone the pipeline with flink-1.3-SNAPSHOT and running on EMR 5.4, the aws-sdk-java latest libraries directly in flink lib dir. I have come to the point where I get the java.net.SocketException: Broken pipe (Write failed). Eager to get a reply and a clue on this! java.io.IOException: Err

Re: Maintaining Stream Partitioning after Mapping?

2017-04-15 Thread Chesnay Schepler
Hello, I think if you have multiple keyBy() transformations with identical parallelism the partitioning should be "preserved". The second keyBy() will still go through the partitioning process, but since both the key and parallelism are identical the resulting partition should be identical as