[GitHub] flink pull request #5710: [FLINK-8948][runtime] Fix IllegalStateException wh...

2018-03-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5710#discussion_r175402536 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferBuilderAndConsumerTest.java --- @@ -81,6 +81,24 @@ public void

[GitHub] flink pull request #5710: [FLINK-8948][runtime] Fix IllegalStateException wh...

2018-03-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5710#discussion_r175404351 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferBuilder.java --- @@ -99,10 +99,11 @@ public void commit

[GitHub] flink pull request #5710: [FLINK-8948][runtime] Fix IllegalStateException wh...

2018-03-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5710#discussion_r175414806 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferBuilder.java --- @@ -99,10 +99,11 @@ public void commit

[GitHub] flink pull request #5710: [FLINK-8948][runtime] Fix IllegalStateException wh...

2018-03-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5710#discussion_r175425893 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferBuilderAndConsumerTest.java --- @@ -202,22 +202,30 @@ private static

[GitHub] flink pull request #5747: [FLINK-9057][network] fix an NPE when cleaning up ...

2018-03-22 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5747 [FLINK-9057][network] fix an NPE when cleaning up before requesting a subpartition view ## What is the purpose of the change In `PartitionRequestServerHandler`, the view reader is created

[GitHub] flink pull request #5748: [FLINK-9053][runtime] only release outputs under t...

2018-03-22 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5748 [FLINK-9053][runtime] only release outputs under the checkpoint lock ## What is the purpose of the change Releasing an operator chain's outputs will call `RecordWriter#clearBuffers()

[GitHub] flink pull request #5624: [FLINK-8402][s3][tests] fix hadoop/presto S3 IT ca...

2018-03-22 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5624#discussion_r176492623 --- Diff: flink-filesystems/flink-s3-fs-presto/src/test/java/org/apache/flink/fs/s3presto/PrestoS3FileSystemITCase.java --- @@ -143,7 +157,20 @@ public void

[GitHub] flink issue #5624: [FLINK-8402][s3][tests] fix hadoop/presto S3 IT cases for...

2018-03-22 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5624 @StephanEwen unfortunately not, for example: `org.apache.flink.runtime.fs.hdfs.HadoopFileSystem#create()` -> `org.apache.hadoop.fs.FileSystem#create()` -> `org.apache.hadoop.fs.s3a.S3AFile

[GitHub] flink issue #5709: [FLINK-8941][network][serializer] improve SpanningRecordS...

2018-03-26 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5709 I'd actually like to keep this collision detection since it does not hurt although it may not fix the problem. Both commits are actually somewhat shots in the dark trying to tackle the ori

[GitHub] flink issue #5624: [FLINK-8402][s3][tests] fix hadoop/presto S3 IT cases for...

2018-03-26 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5624 That's actually the check for whether or not to overwrite the file - let me drop the whole code of this example to give some more context: ``` public FSDataOutputStream create(P

[GitHub] flink issue #5747: [FLINK-9057][network] fix an NPE when cleaning up before ...

2018-03-26 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5747 ok, done ---

[GitHub] flink issue #5624: [FLINK-8402][s3][tests] fix hadoop/presto S3 IT cases for...

2018-03-29 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5624 Indeed, Presto-S3 does better in `com.facebook.presto.hive.PrestoS3FileSystem#create()`: ``` if ((!overwrite) && exists(path)) { throw new IOException("File already ex

[GitHub] flink pull request #5788: [FLINK-9110][docs] fix local bundler installation

2018-03-29 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5788 [FLINK-9110][docs] fix local bundler installation ## What is the purpose of the change The fallback to installing `bundler` locally within `docs/build_docs.sh` did not work for several

[GitHub] flink pull request #5695: [FLINK-8704][tests] Port PartialConsumerPipelinedR...

2018-03-29 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5695#discussion_r178069787 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/PartialConsumePipelinedResultTest.java --- @@ -32,41 +31,51

[GitHub] flink pull request #5790: [FLINK-9107][docs] document timer coalescing for P...

2018-03-29 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5790 [FLINK-9107][docs] document timer coalescing for ProcessFunction ## What is the purpose of the change In a ProcessFunction, registering timers for each event via `ctx.timerService

[GitHub] flink issue #5790: [FLINK-9107][docs] document timer coalescing for ProcessF...

2018-03-29 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5790 Actually, more advanced schemes using `current watermark + 1` (which fires with the next watermark) for the event time timer should also go into the documentation. I'll extend the PR ... ---

[GitHub] flink issue #5790: [FLINK-9107][docs] document timer coalescing for ProcessF...

2018-04-03 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5790 thanks - fixed and extended with an example for "schedule with the next watermark" for event-time timers ---

[GitHub] flink pull request #5672: [FLINK-8872][flip6] fix yarn detached mode command...

2018-04-03 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5672#discussion_r178804566 --- Diff: flink-yarn/pom.xml --- @@ -326,6 +334,62 @@ under the License

[GitHub] flink issue #5672: [FLINK-8872][flip6] fix yarn detached mode command parsin...

2018-04-03 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5672 I reworked the code as desired, i.e. adding `-yd` and `--yarndetached` as deprecated parameters to `CliFrontendParser` and adapting `ProgramOptions` and `FlinkYarnSessionCli` accordingly. ---

[GitHub] flink pull request #5802: [FLINK-9087] [runtime] close the BufferConsumer in...

2018-04-05 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5802#discussion_r179453256 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordWriterTest.java --- @@ -309,8 +309,6 @@ public void

[GitHub] flink pull request #5802: [FLINK-9087] [runtime] close the BufferConsumer in...

2018-04-05 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5802#discussion_r179451610 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java --- @@ -164,7 +164,7 @@ public BufferConsumer

[GitHub] flink issue #5672: [FLINK-8872][flip6] fix yarn detached mode command parsin...

2018-04-05 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5672 ok, after fixing one typo, this should pass Travis now ---

[GitHub] flink pull request #5841: [FLINK-9163[e2e-tests] harden signal traps and con...

2018-04-12 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5841 [FLINK-9163[e2e-tests] harden signal traps and config restoration ## What is the purpose of the change Signal traps on certain systems, e.g. Linux, may be called concurrently when the trap

[GitHub] flink pull request #5842: [FLINK-9144][network] fix SpillableSubpartition ca...

2018-04-12 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5842 [FLINK-9144][network] fix SpillableSubpartition causing jobs to hang when spilling ## What is the purpose of the change This should fix various scenarios where the backlog accounting in the

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-12-04 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r154673167 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClient.java --- @@ -167,6 +167,13 @@ public void

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-12-04 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r154674880 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -108,17 +117,75 @@ public void

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-12-04 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r154675125 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -280,94 +280,120 @@ public String

[GitHub] flink issue #4504: [FLINK-7395] [metrics] Count bytesIn/Out without synchron...

2017-12-04 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4504 or, we may add these fields as a metric, actually - but let's discuss this elsewhere ---

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-12-04 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r154685383 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -244,54 +237,89 @@ public void

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-12-04 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r154697358 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -378,6 +381,11 @@ public void

[GitHub] flink issue #4533: [FLINK-7416][network] Implement Netty receiver outgoing p...

2017-12-05 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4533 ok, I fiddled with my remarks a bit and you are right for the problems of reducing the visibility, which is partly caused by the `CreditBasedClientHandler` not replacing `PartitionRequestClientHandler

[GitHub] flink issue #4533: [FLINK-7416][network] Implement Netty receiver outgoing p...

2017-12-05 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4533 oh, you were faster... ---

[GitHub] flink pull request #5127: [FLINK-7749][network] refactor the ResultPartition...

2017-12-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5127 [FLINK-7749][network] refactor the ResultPartitionWriter into an interface ## What is the purpose of the change After moving task event notification into the `TaskEventDispatcher` (#4761

[GitHub] flink issue #4762: [FLINK-7749][network] remove the ResultPartitionWriter wr...

2017-12-06 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4762 closed in favour of #5127 ---

[GitHub] flink pull request #4762: [FLINK-7749][network] remove the ResultPartitionWr...

2017-12-06 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/4762 ---

[GitHub] flink issue #4759: [FLINK-7746][network] move ResultPartitionWriter#writeBuf...

2017-12-06 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4759 closed in favour of a different approach in #5127 ---

[GitHub] flink pull request #4759: [FLINK-7746][network] move ResultPartitionWriter#w...

2017-12-06 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/4759 ---

[GitHub] flink issue #4559: [FLINK-7468][network] Implement sender backlog logic for ...

2017-12-12 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4559 I'd opt for decreasing under the lock as well, or (even better?) the following alternative: We remove the `buffersInBacklog` member and return the size of the current backlog alon

[GitHub] flink issue #5147: [FLINK-8241][tests] Remove ResultPartitionWriter-related ...

2017-12-12 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5147 FYI: I actually also included these changes into #5127 but wasn't aware that this is unrelated to the change in that PR ---

[GitHub] flink issue #5064: [FLINK-8145][tests] fix various IOManagerAsync instances ...

2017-12-12 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5064 true - here you go ---

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156389253 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordWriterTest.java --- @@ -558,4 +565,10 @@ public void read

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156418759 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferBuilder.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to the

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156426879 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordWriterTest.java --- @@ -155,13 +159,13 @@ public Void call

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156404244 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordOrEventCollectingResultPartitionWriter.java --- @@ -0,0 +1,104

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156407089 --- Diff: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/tasks/StreamMockEnvironment.java --- @@ -144,51 +136,12 @@ public void

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156397312 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPoolDestroyTest.java --- @@ -104,11 +104,10 @@ public void

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156422394 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferBuilderTestUtils.java --- @@ -0,0 +1,36 @@ +/* + * Licensed to

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156392377 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordWriterTest.java --- @@ -360,7 +367,7 @@ public void

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156425243 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordWriterTest.java --- @@ -118,7 +122,7 @@ public Buffer answer

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156420605 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java --- @@ -188,10 +189,16 @@ public Buffer requestBuffer

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156411078 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/SpanningRecordSerializer.java --- @@ -50,24 +51,15

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156386302 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SpilledSubpartitionViewTest.java --- @@ -48,8 +47,7

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156426121 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/util/TestPooledBufferProvider.java --- @@ -28,66 +29,62 @@ import

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156387848 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/util/TestBufferFactory.java --- @@ -40,37 +39,31 @@ private final

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156403467 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordOrEventCollectingResultPartitionWriter.java --- @@ -0,0 +1,104

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156414085 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java --- @@ -129,8 +130,8 @@ private void sendToTarget(T

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156403311 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordOrEventCollectingResultPartitionWriter.java --- @@ -0,0 +1,104

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156421194 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java --- @@ -179,7 +179,8 @@ public void setBufferPoolOwner

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156399551 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordCollectingResultPartitionWriter.java --- @@ -0,0 +1,88

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156420568 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java --- @@ -179,7 +179,8 @@ public void setBufferPoolOwner

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156427022 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferBuilderTestUtils.java --- @@ -0,0 +1,36 @@ +/* + * Licensed to

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156418933 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferBuilder.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to the

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156412102 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/SpanningRecordSerializer.java --- @@ -169,33 +151,29 @@ public

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156416892 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferBuilder.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to the

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156402616 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/operators/testutils/MockEnvironment.java --- @@ -101,7 +90,8 @@ private final

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156418279 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferBuilder.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to the

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156404597 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/writer/RecordOrEventCollectingResultPartitionWriter.java --- @@ -0,0 +1,104

[GitHub] flink pull request #5105: [FLINK-8178][network] Introduce not threadsafe wri...

2017-12-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5105#discussion_r156413073 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/SpanningRecordSerializer.java --- @@ -169,33 +151,29 @@ public

[GitHub] flink issue #5105: [FLINK-8178][network] Introduce not threadsafe write only...

2017-12-12 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5105 oh, and FYI: since nothing (except the hotfix) is actually left of FLINK-8172 after this PR, you should squash it away and close #5104 as well ---

[GitHub] flink issue #4559: [FLINK-7468][network] Implement sender backlog logic for ...

2017-12-13 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4559 good catch, I completely missed the code path in `SpillableSubpartition` - I agree with putting it under `synchronized(buffers)` ---

[GitHub] flink issue #4581: [FLINK-7499][io] fix double buffer release in SpillableSu...

2017-12-13 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4581 ...rebased onto latest master ---

[GitHub] flink pull request #5175: [FLINK-8280][checkstyle] fix checkstyle in BlobSer...

2017-12-18 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5175 [FLINK-8280][checkstyle] fix checkstyle in BlobServer and BlobUtils ## What is the purpose of the change This commits enables checkstyle in the `org.apache.flink.runtime.blob` package and

[GitHub] flink pull request #5176: [FLINK-8279][blob] fall back to TaskManager temp d...

2017-12-18 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5176 [FLINK-8279][blob] fall back to TaskManager temp directories first ## What is the purpose of the change Instead of falling back to `java.io.tmpdir` directly, the BLOB server and cache

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157538818 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionTest.java --- @@ -181,10 +182,27 @@ public void

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157544794 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultSubpartition.java --- @@ -99,6 +82,23 @@ protected Throwable

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157548895 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartition.java --- @@ -77,6 +78,10 @@ /** Flag

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157541024 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SubpartitionTestBase.java --- @@ -62,7 +70,14 @@ public void

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157545208 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartition.java --- @@ -237,6 +243,29 @@ public boolean

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157539147 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionTest.java --- @@ -239,6 +261,10 @@ public void

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157538061 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartitionTest.java --- @@ -103,16 +104,35 @@ public void

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157544965 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java --- @@ -161,6 +172,29 @@ public boolean

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157548033 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java --- @@ -52,6 +54,10 @@ /** Flag

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157540910 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SubpartitionTestBase.java --- @@ -47,7 +48,14 @@ public void

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157703075 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultSubpartition.java --- @@ -99,6 +82,23 @@ protected Throwable

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157706995 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/PipelinedSubpartition.java --- @@ -52,6 +54,10 @@ /** Flag

[GitHub] flink pull request #4559: [FLINK-7468][network] Implement sender backlog log...

2017-12-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4559#discussion_r157707628 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultSubpartition.java --- @@ -99,6 +82,23 @@ protected Throwable

[GitHub] flink pull request #5183: [FLINK-8295][cassandra][build] properly shade nett...

2017-12-19 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5183 [FLINK-8295][cassandra][build] properly shade netty for the datastax driver ## What is the purpose of the change `com.datastax.driver.core.NettyUtil` expects netty to be present either at

[GitHub] flink issue #5183: [FLINK-8295][cassandra][build] properly shade netty for t...

2017-12-20 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5183 true - I adapted the comment accordingly ---

[GitHub] flink pull request #5175: [FLINK-8280][checkstyle] fix checkstyle in BlobSer...

2018-01-02 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5175#discussion_r159209751 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobUtils.java --- @@ -150,8 +150,7 @@ static File initLocalStorageDirectory(String

[GitHub] flink pull request #5231: [FLINK-8346][docs] add v4 signature workaround for...

2018-01-02 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5231 [FLINK-8346][docs] add v4 signature workaround for manual S3 setups ## What is the purpose of the change As per https://lists.apache.org/thread.html

[GitHub] flink issue #5176: [FLINK-8279][blob] fall back to TaskManager temp director...

2018-01-02 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5176 @EronWright this PR originated from a user problem at https://lists.apache.org/thread.html/177644ad38af34fb7ae7fb12a3bfcef3a70cd70dfccb6f7008a24eb6@%3Cuser.flink.apache.org%3E And yes, this

[GitHub] flink issue #5176: [FLINK-8279][blob] fall back to TaskManager temp director...

2018-01-02 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5176 For the change regarding the JobManager use of a TaskManager configuration parameter, I created a separate issue and included the appropriate commits here (FLINK-8250) as this commit probably does not

[GitHub] flink pull request #5243: [FLINK-8362][elasticsearch] shade all dependencies

2018-01-05 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5243 [FLINK-8362][elasticsearch] shade all dependencies ## What is the purpose of the change The Elasticsearch connectors have some dependencies that need to be available and should not conflict

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r157760366 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandler.java --- @@ -41,30 +42,33 @@ import

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r157760482 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandler.java --- @@ -94,6 +98,15 @@ void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r157760652 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandler.java --- @@ -318,192 +307,56 @@ else if

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r157760560 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandler.java --- @@ -158,32 +164,44 @@ public void

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r159401061 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -88,6 +93,35 @@ public void run

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r159400956 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -88,6 +93,35 @@ public void run

[GitHub] flink pull request #4552: [FLINK-7456][network] Implement Netty sender incom...

2018-01-08 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4552#discussion_r159399376 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -88,6 +93,35 @@ public void run

<    4   5   6   7   8   9   10   11   >