[GitHub] flink pull request #4419: [hotfix][tests] fix IncrementalRocksDbBackendEvent...

2017-07-28 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/4419 [hotfix][tests] fix IncrementalRocksDbBackendEventTimeWindowCheckpointingITCase taking too long for the TravisCI output check ## What is the purpose of the change Fix test failures due to

[GitHub] flink issue #4419: [hotfix][tests] fix IncrementalRocksDbBackendEventTimeWin...

2017-07-28 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4419 sure, it does not hurt at least, although probably at the moment, only the two RocksDB tests may be affected (also the non-incremental one that I missed) --- If your project is set up for it, you can

[GitHub] flink issue #4416: [FLINK-7279][minicluster] fix a deadlock between TM and c...

2017-07-28 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4416 A simple asynchronous call as in `TaskExecutor#onFatalErrorAsync()` is not enough though because this is what is actually already done and led to me finding this error in the first place. Please see

[GitHub] flink pull request #6115: [FLINK-9435][java] Remove per-key selection Tuple ...

2018-06-04 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6115 [FLINK-9435][java] Remove per-key selection Tuple instantiation via reflection in ComparableKeySelector and ArrayKeySelector ## What is the purpose of the change Inside `KeySelectorUtil

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-06-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r194900211 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-06-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r194900846 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-06-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r194901455 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-06-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r194901682 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-06-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r194902217 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-06-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r194902416 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink issue #5900: [FLINK-9222][docs] add documentation for setting up Gradl...

2018-06-12 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5900 Thanks for the review, Timo. Regarding the copy of the gradle build file: since we did not really have a pristine version (only one that is generated by the shell script) I wanted to put one

[GitHub] flink pull request #6238: [FLINK-9636][network] fix inconsistency with faile...

2018-07-02 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6238 [FLINK-9636][network] fix inconsistency with failed buffer redistribution ## What is the purpose of the change If an exception is thrown in `NetworkBufferPool#requestMemorySegments()`&#

[GitHub] flink pull request #6238: [FLINK-9636][network] fix inconsistency with faile...

2018-07-03 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/6238#discussion_r199702444 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPool.java --- @@ -147,7 +151,12 @@ public void recycle

[GitHub] flink issue #6238: [FLINK-9636][network] fix inconsistency with failed buffe...

2018-07-03 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/6238 actually, it was quite easy to reproduce and the fix was also just as you proposed - please see the new commits (the old one was only renamed since I created a separate issue for that now) ---

[GitHub] flink pull request #6254: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-04 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6254 [FLINK-9676][network] clarify contracts of BufferListener#notifyBufferAvailable() and fix a deadlock ## What is the purpose of the change When recycling exclusive buffers of a

[GitHub] flink pull request #6257: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-04 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6257 [FLINK-9676][network] clarify contracts of BufferListener#notifyBufferAvailable() and fix a deadlock ## What is the purpose of the change When recycling exclusive buffers of a

[GitHub] flink pull request #6254: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-04 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/6254 ---

[GitHub] flink issue #6254: [FLINK-9676][network] clarify contracts of BufferListener...

2018-07-04 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/6254 I updated the commit with a fixed unit test and added a big TODO on what else would need to be adapted in order to go with this solution (no updated commit message yet). Since this is even

[GitHub] flink pull request #6257: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-05 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/6257#discussion_r200274038 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java --- @@ -251,27 +257,56 @@ private MemorySegment

[GitHub] flink pull request #6257: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-05 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/6257#discussion_r200274433 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java --- @@ -251,27 +257,56 @@ private MemorySegment

[GitHub] flink pull request #6257: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-05 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/6257#discussion_r200283214 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java --- @@ -251,27 +257,56 @@ private MemorySegment

[GitHub] flink pull request #6271: [FLINK-9766][network][tests] fix cleanup in Remote...

2018-07-05 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6271 [FLINK-9766][network][tests] fix cleanup in RemoteInputChannelTest ## What is the purpose of the change If an assertion in the tests of `RemoteInputChannelTest` fails and as a result the

[GitHub] flink pull request #6272: [FLINK-9755][network] forward exceptions in Remote...

2018-07-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6272 [FLINK-9755][network] forward exceptions in RemoteInputChannel#notifyBufferAvailable() to the responsible thread ## What is the purpose of the change Exceptions in `RemoteInputChannel

[GitHub] flink pull request #6291: [FLINK-9785][network] add remote address informati...

2018-07-10 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6291 [FLINK-9785][network] add remote address information to LocalTransportException instances ## What is the purpose of the change In contrast to the messages inside `RemoteTransportException`s

[GitHub] flink pull request #6272: [FLINK-9755][network] forward exceptions in Remote...

2018-07-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/6272#discussion_r201260221 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -360,32 +360,45 @@ public boolean

[GitHub] flink issue #6272: [FLINK-9755][network] forward exceptions in RemoteInputCh...

2018-07-10 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/6272 thanks for the review - I changed the code as requested ---

[GitHub] flink pull request #6254: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/6254#discussion_r201272052 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -594,22 +626,22 @@ public String

[GitHub] flink pull request #6254: [FLINK-9676][network] clarify contracts of BufferL...

2018-07-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/6254#discussion_r201273133 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -479,6 +508,9 @@ void

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-07-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r201286036 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-07-10 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5900#discussion_r201310579 --- Diff: docs/quickstart/java_api_quickstart.md --- @@ -108,16 +118,201 @@ In Eclipse, choose box: `-Xmx800m`. In IntelliJ IDEA recommended way to

[GitHub] flink issue #5900: [FLINK-9222][docs] add documentation for setting up Gradl...

2018-07-10 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5900 Hi @twalthr, I just created a fixup commit for this PR and the one for `flink-web`. Can you have a second look? ---

[GitHub] flink issue #6272: [FLINK-9755][network] forward exceptions in RemoteInputCh...

2018-07-10 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/6272 actually, I forgot to adapt the unit test which relied on the previous order of the calls and assumed that the buffer got recycled. With the changes from the fixup, the buffer is not recycled by the

[GitHub] flink pull request #5872: [FLINK-9206][checkpoints] add job IDs to Checkpoin...

2018-04-18 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5872 [FLINK-9206][checkpoints] add job IDs to CheckpointCoordinator log messages ## What is the purpose of the change The `CheckpointCoordinator` exists per job but its log messages (all bundled

[GitHub] flink pull request #5872: [FLINK-9206][checkpoints] add job IDs to Checkpoin...

2018-04-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5872#discussion_r182455202 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java --- @@ -1214,7 +1216,7 @@ public void run

[GitHub] flink pull request #5872: [FLINK-9206][checkpoints] add job IDs to Checkpoin...

2018-04-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5872#discussion_r182455513 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java --- @@ -857,7 +859,7 @@ public void run

[GitHub] flink pull request #5873: [FLINK-9208][tests] fix naming of StreamNetworkThr...

2018-04-18 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5873 [FLINK-9208][tests] fix naming of StreamNetworkThroughputBenchmarkTest This PR fixes the naming of `StreamNetworkThroughputBenchmarkTest` so that it gets included in the maven tests. You can merge

[GitHub] flink issue #5872: [FLINK-9206][checkpoints] add job IDs to CheckpointCoordi...

2018-04-18 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5872 alright...there you go ---

[GitHub] flink pull request #5872: [FLINK-9206][checkpoints] add job IDs to Checkpoin...

2018-04-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5872#discussion_r182478661 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java --- @@ -857,7 +859,7 @@ public void run

[GitHub] flink pull request #5874: [FLINK-9076][network] reduce minimum number of flo...

2018-04-18 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5874 [FLINK-9076][network] reduce minimum number of floating buffers to 0 ## What is the purpose of the change Currently, floating buffers (per gate) are always required in case credit-based flow

[GitHub] flink pull request #5870: [FLINK-8967][tests] Port NetworkStackThroughputITC...

2018-04-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5870#discussion_r182668925 --- Diff: flink-tests/src/test/java/org/apache/flink/test/runtime/NetworkStackThroughputITCase.java --- @@ -234,43 +232,51 @@ public void testThroughput

[GitHub] flink pull request #5870: [FLINK-8967][tests] Port NetworkStackThroughputITC...

2018-04-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5870#discussion_r182699788 --- Diff: flink-tests/src/test/java/org/apache/flink/test/runtime/NetworkStackThroughputITCase.java --- @@ -327,7 +335,8 @@ private JobGraph createJobGraph

[GitHub] flink pull request #5900: [FLINK-9222][docs] add documentation for setting u...

2018-04-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5900 [FLINK-9222][docs] add documentation for setting up Gradle projects ## What is the purpose of the change Together with https://github.com/apache/flink-web/pull/105, this will add a project

[GitHub] flink pull request #5914: [FLINK-9256][network] fix NPE in SingleInputGate#u...

2018-04-25 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5914 [FLINK-9256][network] fix NPE in SingleInputGate#updateInputChannel() for non-credit based flow control ## What is the purpose of the change `SingleInputGate#updateInputChannel()` fails to

[GitHub] flink pull request #5915: [FLINK-9243][tests] fix flaky SuccessAfterNetworkB...

2018-04-25 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5915 [FLINK-9243][tests] fix flaky SuccessAfterNetworkBuffersFailureITCase this adds some `rebalance()` operations into `SuccessAfterNetworkBuffersFailureITCase#runKMeans` which should be set up to fail

[GitHub] flink pull request #5916: [hotfix][tests] remove redundant rebalance in Succ...

2018-04-26 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5916 [hotfix][tests] remove redundant rebalance in SuccessAfterNetworkBuffersFailureITCase This removes a redundant `rebalance()` operation introduced by #5915. You can merge this pull request into a Git

[GitHub] flink issue #5915: [FLINK-9243][tests] fix flaky SuccessAfterNetworkBuffersF...

2018-04-26 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5915 true, `points.rebalance()` is redundant, thanks for having a look, I created a follow-up fix in #5916 ---

[GitHub] flink pull request #5923: [FLINK-9253][network] make the maximum floating bu...

2018-04-26 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5923 [FLINK-9253][network] make the maximum floating buffers count channel-type independent ## What is the purpose of the change The credit-based flow control path assigns exclusive buffers only

[GitHub] flink pull request #5942: [FLINK-9274][kafka] add thread name for partition ...

2018-04-30 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5942 [FLINK-9274][kafka] add thread name for partition discovery ## What is the purpose of the change For debugging, threads should have names to filter on and get a quick overview. The Kafka

[GitHub] flink pull request #5943: [FLINK-9275][streaming] add taskName to the output...

2018-04-30 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5943 [FLINK-9275][streaming] add taskName to the output flusher thread's name ## What is the purpose of the change All output flusher threads are named "OutputFlusher" while at the

[GitHub] flink issue #5900: [FLINK-9222][docs] add documentation for setting up Gradl...

2018-04-30 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5900 I'm pretty sure I got all this covered by using a separate "configuration": `myShadowJar` - not the standard dependency configuration*, but does the trick: Everything added to `mySh

[GitHub] flink pull request #5963: [FLINK-9305][s3] also register flink-s3-fs-hadoop'...

2018-05-07 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5963 [FLINK-9305][s3] also register flink-s3-fs-hadoop's factory for the s3a:// scheme ## What is the purpose of the change For enhanced user experience, we should also register our Hadoo

[GitHub] flink issue #5923: [FLINK-9253][network] make the maximum floating buffers c...

2018-05-07 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5923 Honestly, I did not find the proposed change too convincing either due to the additional branch into bounded vs. unbounded partitions which caused the variable names to be wrong in the unbounded case

[GitHub] flink issue #5923: [FLINK-9253][network] make the maximum floating buffers c...

2018-05-07 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5923 Apparently, there's a problem with unknown input channels that surfaced with the newly-merged extra tests: they don't get exclusive buffers (naturally) but the floating buffers are calc

[GitHub] flink pull request #6064: [FLINK-9409][mvn] add missing assembly dependencie...

2018-05-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/6064 [FLINK-9409][mvn] add missing assembly dependencies to flink-dist/pom… ## What is the purpose of the change Building Flink via `mvn clean install -pl flink-dist -am` currently fails

[GitHub] flink issue #6064: [FLINK-9409][mvn] add missing assembly dependencies to fl...

2018-05-23 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/6064 apparently, `flink-json` and `flink-avro` should not even be in `opt/` - closing this PR ---

[GitHub] flink pull request #6064: [FLINK-9409][mvn] add missing assembly dependencie...

2018-05-23 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/6064 ---

[GitHub] flink pull request #5583: [FLINK-8737][network] disallow creating a union of...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5583#discussion_r170987211 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/UnionInputGate.java --- @@ -189,11 +189,11 @@ public void

[GitHub] flink pull request #5583: [FLINK-8737][network] disallow creating a union of...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5583#discussion_r170988543 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/UnionInputGate.java --- @@ -61,7 +61,7

[GitHub] flink pull request #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in ...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5581#discussion_r170990471 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SubpartitionTestBase.java --- @@ -138,11 +145,68 @@ static void

[GitHub] flink pull request #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in ...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5581#discussion_r170993496 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionTest.java --- @@ -222,59 +219,24 @@ public void

[GitHub] flink issue #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in spilled...

2018-02-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5581 rebased (not including newest FLINK-8694 changes!) and addressed the comments ---

[GitHub] flink pull request #5588: [FLINK-8750][runtime] Improve detection of no rema...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5588#discussion_r171001301 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/EventSerializer.java --- @@ -122,38 +121,16 @@ private static

[GitHub] flink pull request #5588: [FLINK-8750][runtime] Improve detection of no rema...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5588#discussion_r171000868 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/EventSerializer.java --- @@ -107,10 +107,9 @@ else if (eventClass

[GitHub] flink pull request #5588: [FLINK-8750][runtime] Improve detection of no rema...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5588#discussion_r171001862 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/EventSerializer.java --- @@ -318,13 +295,9 @@ public static

[GitHub] flink pull request #5588: [FLINK-8750][runtime] Improve detection of no rema...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5588#discussion_r171002283 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/serialization/EventSerializerTest.java --- @@ -117,49 +116,55 @@ public void

[GitHub] flink pull request #5588: [FLINK-8750][runtime] Improve detection of no rema...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5588#discussion_r171001535 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/serialization/EventSerializer.java --- @@ -107,10 +107,9 @@ else if (eventClass

[GitHub] flink pull request #5588: [FLINK-8750][runtime] Improve detection of no rema...

2018-02-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5588#discussion_r171004092 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/api/serialization/EventSerializerTest.java --- @@ -117,49 +116,55 @@ public void

[GitHub] flink issue #5570: [FLINK-8768][network] Let NettyMessageDecoder inherit fro...

2018-02-28 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5570 ok, done ---

[GitHub] flink pull request #4529: [FLINK-7428][network] avoid buffer copies when rec...

2018-02-28 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/4529 ---

[GitHub] flink issue #4529: [FLINK-7428][network] avoid buffer copies when receiving ...

2018-02-28 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4529 let me close this PR since this will be done by upgrading netty instead ---

[GitHub] flink pull request #5588: [FLINK-8750][runtime] Improve detection of no rema...

2018-02-28 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5588#discussion_r171183475 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGate.java --- @@ -553,6 +553,12 @@ public void

[GitHub] flink pull request #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in ...

2018-02-28 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5581#discussion_r171210203 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SubpartitionTestBase.java --- @@ -182,7 +182,7 @@ private static void

[GitHub] flink issue #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in spilled...

2018-02-28 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5581 ok, fixed the tests again which broke with 11981d9173ccef2236fce6b3f30d1ef467a91d6f ---

[GitHub] flink issue #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in spilled...

2018-02-28 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5581 one more rebase onto latest master to fix merge conflicts - good to go after travis is green ---

[GitHub] flink pull request #5601: [FLINK-8336][yarn/s3][tests] harden YarnFileStageT...

2018-02-28 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5601 [FLINK-8336][yarn/s3][tests] harden YarnFileStageTest upload test for eventual consistent read-after-write ## What is the purpose of the change In case the newly written object cannot be

[GitHub] flink pull request #5602: [FLINK-8801][yarn/s3] fix Utils#setupLocalResource...

2018-02-28 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5602 [FLINK-8801][yarn/s3] fix Utils#setupLocalResource() relying on consistent read-after-write ## What is the purpose of the change > Amazon S3 provides read-after-write consistency for PUTS

[GitHub] flink issue #5581: [FLINK-8755][FLINK-8786][network] fix two bugs in spilled...

2018-03-01 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5581 alright - test failure is unrelated and appeared separately before (see [FLINK-4387](https://issues.apache.org/jira/browse/FLINK-4387)) ---

[GitHub] flink pull request #5606: [FLINK-4387][QS] don't wait and process requests a...

2018-03-01 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5606 [FLINK-4387][QS] don't wait and process requests at Netty servers after shutdown request ## What is the purpose of the change There is a race condition on an assertion in Netty's

[GitHub] flink pull request #5611: [FLINK-8769][flip6] do not print error causing exc...

2018-03-01 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5611 [FLINK-8769][flip6] do not print error causing exceptions without debugging ## What is the purpose of the change Quickstart job execution in IDE logs these two exceptions which may mislead

[GitHub] flink pull request #5621: [FLINK-8517] fix missing synchronization in TaskEv...

2018-03-02 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5621 [FLINK-8517] fix missing synchronization in TaskEventDispatcher ## What is the purpose of the change The `TaskEventDispatcher` was missing synchronization accessing the `registeredHandlers

[GitHub] flink pull request #5449: [FLINK-8623] ConnectionUtilsTest.testReturnLocalHo...

2018-03-02 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5449#discussion_r171858504 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/net/ConnectionUtilsTest.java --- @@ -61,7 +61,7 @@ public void

[GitHub] flink pull request #5624: [FLINK-8402][s3][tests] fix hadoop/presto S3 IT ca...

2018-03-02 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5624 [FLINK-8402][s3][tests] fix hadoop/presto S3 IT cases for eventually-consistent operations ## What is the purpose of the change `HadoopS3FileSystemITCase` and `PrestoS3FileSystemITCase` rely

[GitHub] flink pull request #5637: [FLINK-8860][flip6] stop SlotManager spamming logs...

2018-03-05 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5637 [FLINK-8860][flip6] stop SlotManager spamming logs for every TM heartbeat at log level 'info' ## What is the purpose of the change For every `TaskManager` heartbeat message, `S

[GitHub] flink issue #5621: [FLINK-8517] fix missing synchronization in TaskEventDisp...

2018-03-05 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5621 FYI: test failure is unrelated and relates to #5606 which apparently does not (completely) fix the issue ---

[GitHub] flink issue #5601: [FLINK-8818][yarn/s3][tests] harden YarnFileStageTest upl...

2018-03-05 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5601 The main issue was actually described in FLINK-8801: According to https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel: > Amazon S3 provides read-af

[GitHub] flink issue #5601: [FLINK-8818][yarn/s3][tests] harden YarnFileStageTest upl...

2018-03-06 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5601 Different FS implementations try to circumvent the problem (or not), e.g. Presto's implementation implements the retries itself whenever it cannot find an object, while Hadoop's implementati

[GitHub] flink issue #5601: [FLINK-8818][yarn/s3][tests] harden YarnFileStageTest upl...

2018-03-06 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5601 also see further test changes in this regard in #5624 ---

[GitHub] flink issue #5637: [FLINK-8860][flip6] stop SlotManager spamming logs for ev...

2018-03-08 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5637 makes sense, since this is not indicating an error after all ---

[GitHub] flink pull request #5661: [FLINK-8896][kafka08] fix Kafka08Fetcher trying to...

2018-03-08 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5661 [FLINK-8896][kafka08] fix Kafka08Fetcher trying to look up topic "n/a" on partiton "-1" ## What is the purpose of the change `Kafka08Fetcher` uses a `MARKER` to make

[GitHub] flink pull request #5670: [FLINK-8904][cli][tests] always restore the previo...

2018-03-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5670 [FLINK-8904][cli][tests] always restore the previous sysout when changing it in the test ## What is the purpose of the change Several tests inside the `org.apache.flink.client.cli` package

[GitHub] flink pull request #5671: [FLINK-8906][flip6][tests] also test Flip6DefaultC...

2018-03-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5671 [FLINK-8906][flip6][tests] also test Flip6DefaultCLI in org.apache.flink.client.cli tests ## What is the purpose of the change Various tests in `org.apache.flink.client.cli` only test with

[GitHub] flink pull request #5672: [FLINK-8872][flip6] fix yarn detached mode command...

2018-03-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5672 [FLINK-8872][flip6] fix yarn detached mode command parsing ## What is the purpose of the change Running yarn per-job cluster in detached mode via `-yd` currently does not detach and instead

[GitHub] flink pull request #5670: [FLINK-8904][cli][tests] always restore the previo...

2018-03-12 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5670#discussion_r173746711 --- Diff: flink-clients/src/test/java/org/apache/flink/client/cli/CliFrontendTestUtils.java --- @@ -65,9 +67,14 @@ public static String getInvalidConfigDir

[GitHub] flink issue #5449: [FLINK-8623] ConnectionUtilsTest.testReturnLocalHostAddre...

2018-03-12 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5449 Thanks @StephanEwen indeed - I also did not grasp the full intend of the change. Looking at the code again, basically `ConnectionUtils#findConnectingAddress()` tries to connect with a set of

[GitHub] flink issue #5671: [FLINK-8906][flip6][tests] also test Flip6DefaultCLI in o...

2018-03-15 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5671 Thanks for the review - indeed I copied that part too easily. Fixed now - let's wait for travis to approve the changes though. ---

[GitHub] flink pull request #5709: [FLINK-8941][network][serializer] improve Spanning...

2018-03-16 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5709 [FLINK-8941][network][serializer] improve SpanningRecordSerializationTest and ensure unique spilling files ## What is the purpose of the change This PR contains two commits trying to tackle

[GitHub] flink issue #5672: [FLINK-8872][flip6] fix yarn detached mode command parsin...

2018-03-16 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5672 thanks for the review, I also did not like the side-effect approach and after thinking a bit about your first message, I independently came up with the same thing as you proposed in the second one :p

[GitHub] flink issue #5710: [FLINK-8948][runtime] Fix IllegalStateException when clos...

2018-03-16 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5710 it may be better to make `BufferBuilder#finish()` idempotent ---

[GitHub] flink issue #5624: [FLINK-8402][s3][tests] fix hadoop/presto S3 IT cases for...

2018-03-16 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/5624 Unfortunately, this is not entirely under our control, since we rely on the underlying `FileSystem` implementation. I can probably reduce some parts of the tests that may lead to eventual consistent

[GitHub] flink pull request #5710: [FLINK-8948][runtime] Fix IllegalStateException wh...

2018-03-19 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/5710#discussion_r175363048 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/BufferBuilder.java --- @@ -99,10 +99,11 @@ public void commit

<    3   4   5   6   7   8   9   10   11   >