[GitHub] flink issue #3076: [FLINK-5129] make the BlobServer use a distributed file s...

2017-01-09 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3076 fixed a typo in the unit test that lead to the tests passing although there was still something wrong which is now fixed as well --- If your project is set up for it, you can reply to this email and

[GitHub] flink pull request #3084: [FLINK-5129] make the BlobServer use a distributed...

2017-01-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3084 [FLINK-5129] make the BlobServer use a distributed file system Make the BlobCache use the BlobServer's distributed file system in HA mode: previously even in HA mode and if the cache has acce

[GitHub] flink pull request #3085: [FLINK-5178] allow BlobCache to use a distributed ...

2017-01-09 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3085 [FLINK-5178] allow BlobCache to use a distributed file system irrespective of the HA mode Allow the BlobServer and BlobCache to use a distributed file system for distributing BLOBs even if not in HA

[GitHub] flink pull request #3120: [FLINK-5482] QueryableStateClient does not recover...

2017-01-13 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3120 [FLINK-5482] QueryableStateClient does not recover from a failed lookup due to a non-running job This PR checks each cached lookup query whether it is complete and removes any failed lookup from the

[GitHub] flink pull request #3129: [FLINK-5507] remove KeyedStream#asQueryableState(n...

2017-01-16 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3129 [FLINK-5507] remove KeyedStream#asQueryableState(name, ListStateDescriptor) The queryable state "sink" using ListState stores all incoming data forever and is never cleaned. Eventually, it

[GitHub] flink pull request #3131: [FLINK-5515] remove unused kvState.getSerializedVa...

2017-01-16 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3131 [FLINK-5515] remove unused kvState.getSerializedValue call in KvStateServerHandler this seems like a simple left-over from a merge that is doing unnecessary extra work You can merge this pull

[GitHub] flink pull request #3135: [FLINK-5521] remove unused KvStateRequestSerialize...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3135 [FLINK-5521] remove unused KvStateRequestSerializer#serializeList Also make sure that the serialization via the state backends' list states matches the deserialization o

[GitHub] flink pull request #3139: [FLINK-5528][query][tests] reduce the retry delay ...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3139 [FLINK-5528][query][tests] reduce the retry delay in QueryableStateITCase Using 100ms instead of the 1s previously used does not impose too much additional query load and reduces the test suite&#

[GitHub] flink pull request #3142: [FLINK-5527][query] querying a non-existing key do...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3142 [FLINK-5527][query] querying a non-existing key does not return the default value Querying for a non-existing key for a state that has a default value set currently results in an

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-17 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3143 [FLINK-5530] fix race condition in AbstractRocksDBState#getSerializedValue `AbstractRocksDBState#getSerializedValue()` uses the same key serialisation stream as the ordinary state access methods

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638451 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/state/StateBackendTestBase.java --- @@ -242,6 +245,132 @@ public void testValueState() throws

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638478 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,95

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638710 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,95

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-18 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r96638594 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,95

[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...

2017-01-18 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3142 I saw that deprecation but nonetheless the default value is exposed which is why a consistent behaviour is needed. Since the state descriptor says "that is the value if nothing is se

[GitHub] flink pull request #3171: [FLINK-5561] fix DataInputDeserializer#available()...

2017-01-19 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3171 [FLINK-5561] fix DataInputDeserializer#available() 1 smaller than correct This also adds a unit test for `DataInputDeserializer#available()` - the first one for `DataInputDeserializer` unfortunately

[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...

2017-01-19 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3172 [FLINK-5559] let KvStateRequestSerializer#deserializeKeyAndNamespace() throw a proper IOException This adds the hint that a deserialisation failure probably results from a `"mismatch in th

[GitHub] flink pull request #3174: [FLINK-5576] extend deserialization functions of K...

2017-01-19 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3174 [FLINK-5576] extend deserialization functions of KvStateRequestSerializer to detect unconsumed bytes `KvStateRequestSerializer#deserializeValue()` deserializes a given byte array. This is used by

[GitHub] flink pull request #3171: [FLINK-5561] fix DataInputDeserializer#available()...

2017-01-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3171#discussion_r97063511 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/util/DataInputDeserializerTest.java --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #3143: [FLINK-5530] fix race condition in AbstractRocksDB...

2017-01-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3143#discussion_r97065423 --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/AbstractRocksDBState.java --- @@ -132,55 +132,91

[GitHub] flink pull request #3172: [FLINK-5559] let KvStateRequestSerializer#deserial...

2017-01-23 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3172#discussion_r97279222 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/query/netty/message/KvStateRequestSerializer.java --- @@ -377,22 +376,24 @@ public static

[GitHub] flink pull request #3142: [FLINK-5527][query] querying a non-existing key do...

2017-01-23 Thread NicoK
Github user NicoK closed the pull request at: https://github.com/apache/flink/pull/3142 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...

2017-01-23 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3142 Ok, let's not introduce the (now deprecated) default values in the queryable state API. I'll create a new Jira and PR for removing that part from the RocksDB back-end and consistently re

[GitHub] flink pull request #3193: [FLINK-5527][query] querying a non-existing key is...

2017-01-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3193 [FLINK-5527][query] querying a non-existing key is inconsistent among state backends Querying for a non-existing key for a state that has a default value set currently results in an

[GitHub] flink pull request #3194: [FLINK-5615][query] execute the QueryableStateITCa...

2017-01-23 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3194 [FLINK-5615][query] execute the QueryableStateITCase for all three state back-ends This extends the `QueryableStateITCase` so that it is able to run with any selected state backend. Some

[GitHub] flink pull request #3218: [FLINK-5642][query] fix a race condition with Head...

2017-01-26 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3218 [FLINK-5642][query] fix a race condition with HeadListState The idiom behind `AppendingState#get()` is to return a copy of the value behind or at least not to allow changes to the underlying state

[GitHub] flink pull request #3222: [FLINK-5666] add unit tests verifying that BlobSer...

2017-01-26 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3222 [FLINK-5666] add unit tests verifying that BlobServer#delete() deletes from HDFS this does not fix FLINK-5666 but adds some more unit tests verifying intended behaviour You can merge this pull

[GitHub] flink issue #3218: [FLINK-5642][query] fix a race condition with HeadListSta...

2017-02-03 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/3218 1. Actually, RocksDB state's get() method has the idiom of returning a (deserialized) **copy** with which the user can do whatever he likes to, knowing that changes are not reflected in the state

[GitHub] flink pull request #3275: [FLINK-5618][docs] add queryable state (user) docu...

2017-02-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3275 [FLINK-5618][docs] add queryable state (user) documentation This adds initial documentation of the queryable state from a user's perspective. You can merge this pull request into a Git reposito

[GitHub] flink pull request #3272: [FLINK-5721] Add FoldingState to State Documentati...

2017-02-06 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3272#discussion_r99614763 --- Diff: docs/dev/stream/state.md --- @@ -113,9 +113,19 @@ be retrieved using `Iterable get()`. added to the state. The interface is the same as for

[GitHub] flink pull request #3272: [FLINK-5721] Add FoldingState to State Documentati...

2017-02-06 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/3272#discussion_r99615295 --- Diff: docs/dev/stream/state.md --- @@ -126,8 +136,8 @@ To get a state handle, you have to create a `StateDescriptor`. This holds the na (as we will

[GitHub] flink pull request #3279: [FLINK-5618][docs] createSerializer must actually ...

2017-02-06 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/3279 [FLINK-5618][docs] createSerializer must actually get a non-null ExecutionConfig providing `null` fails with a NPE You can merge this pull request into a Git repository by running: $ git pull

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r13440 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r138853431 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -155,6 +156,57 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r138897087 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGate.java --- @@ -333,7 +333,7 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r138889183 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r138852758 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -155,6 +156,57 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r138879343 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPool.java --- @@ -159,23 +159,20 @@ public void recycle

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r13514 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r138891582 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r138889161 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r13882 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r136027041 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java --- @@ -281,6 +284,67 @@ public void

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138916859 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -72,6 +79,21

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138909210 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java --- @@ -64,7 +63,7 @@ * Buffer availability

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138923001 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -183,18 +214,40 @@ void

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r136056467 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -72,6 +79,21

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r136056661 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -72,6 +79,21

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138915141 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -72,6 +79,21

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138915958 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -72,6 +79,21

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138915417 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -72,6 +79,21

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138922984 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -183,18 +214,40 @@ void

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-15 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r138918917 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -100,7 +122,16 @@ public

[GitHub] flink issue #4591: [FLINK-7514][tests] fix BackPressureStatsTrackerITCase re...

2017-09-19 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4591 @zentol unfortunately, when changing to using Netty's buffer counting, it will check any form of illegal reference count usages, and a double-free is one of them. Even without Netty, this pa

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139926257 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/HDFSTest.java --- @@ -257,7 +265,88 @@ public void testBlobServerRecovery() throws Exception

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139955371 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobClient.java --- @@ -415,13 +393,17 @@ private BlobKey putBuffer(@Nullable JobID jobId

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139956243 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobClient.java --- @@ -601,7 +564,39 @@ public void deleteInternal(@Nullable JobID jobId

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139957010 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobServer.java --- @@ -174,7 +176,6 @@ public ServerSocket createSocket(int port) throws

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139962735 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobServer.java --- @@ -389,95 +413,332 @@ public File getFile(JobID jobId, BlobKey key

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139966738 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/PermanentBlobCache.java --- @@ -0,0 +1,429 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139968617 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/PermanentBlobCache.java --- @@ -0,0 +1,429 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139969076 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/TransientBlobCache.java --- @@ -0,0 +1,322 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #4358: [FLINK-7068][blob] change BlobService sub-classes ...

2017-09-20 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4358#discussion_r139969639 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/blob/TransientBlobCache.java --- @@ -0,0 +1,322 @@ +/* + * Licensed to the Apache

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r140184571 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -186,7 +186,7 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r140189504 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -188,14 +195,14 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r140187056 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -155,6 +156,57 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r140190096 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -264,11 +271,13 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r140184098 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -150,7 +150,7 @@ public void

[GitHub] flink issue #4590: [FLINK-7513][tests] remove TestBufferFactory#MOCK_BUFFER

2017-09-21 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4590 alright - I integrated the `buffer.retain()` calls ---

[GitHub] flink pull request #4571: [FLINK-7487][tests] fix ClassLoaderITCase#testDisp...

2017-09-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4571#discussion_r140235301 --- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java --- @@ -391,5 +391,8 @@ public void run

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-21 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r140271306 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -186,7 +186,7 @@ public void

[GitHub] flink issue #4358: [FLINK-7068][blob] change BlobService sub-classes for per...

2017-09-25 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4358 After an offline discussion with @tillrohrmann, we agreed to have more of the transient vs. permanent BLOB handling inside the `BlobServer` by including the type into the `BlobKey`. The caches

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140809272 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -70,6 +78,15

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140810853 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -99,8 +114,24 @@ public

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140821090 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -183,18 +214,40 @@ void

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140811033 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -100,7 +122,16 @@ public

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140822508 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -183,18 +214,40 @@ void

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140829015 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -215,6 +269,48 @@ public String

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140831302 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -183,18 +214,40 @@ void

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140833417 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java --- @@ -281,6 +286,73 @@ public void

[GitHub] flink pull request #4499: [FLINK-7394][core] Manage exclusive buffers in Rem...

2017-09-25 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r140831859 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java --- @@ -281,6 +286,73 @@ public void

[GitHub] flink issue #4698: [FLINK-7661][network] Add credit field in PartitionReques...

2017-09-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4698 Actually, what worries me a bit are the test failures of the 3rd and 4th test profile. I think, they're unrelated (especially the Kafka ones which are kind of instable at the moment in general)

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141293477 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -169,95 +182,19 @@ public void

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141281604 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283 @@ +/* + * Licensed

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141288898 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283 @@ +/* + * Licensed

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141291264 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -122,6 +113,28 @@ public void

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141292253 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283 @@ +/* + * Licensed

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141288399 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283 @@ +/* + * Licensed

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141290289 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -193,7 +193,8 @@ private void

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141283335 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283 @@ +/* + * Licensed

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141285605 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283 @@ +/* + * Licensed

[GitHub] flink pull request #4733: [FLINK-7701][network] really fix watermark configu...

2017-09-27 Thread NicoK
GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/4733 [FLINK-7701][network] really fix watermark configuration order this time ## What is the purpose of the change Netty (in the version we use) only allows setting high and low watermarks after

[GitHub] flink issue #4447: [FLINK-7312][checkstyle] activate checkstyle for flink/co...

2017-09-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4447 Just curious whether we can merge this now - it has been laying around for too long. ---

[GitHub] flink issue #4506: [FLINK-7400][cluster] fix off-heap limits set to conserva...

2017-09-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4506 @tillrohrmann can you have a look at this? ---

[GitHub] flink pull request #4517: [FLINK-7411][network] minor performance improvemen...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4517#discussion_r141341300 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/NettyMessage.java --- @@ -130,34 +128,31 @@ protected void decode

[GitHub] flink pull request #4517: [FLINK-7411][network] minor performance improvemen...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4517#discussion_r141345178 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/NettyMessage.java --- @@ -242,21 +239,22 @@ ByteBuf write(ByteBufAllocator

[GitHub] flink issue #4517: [FLINK-7411][network] minor performance improvements in N...

2017-09-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4517 how about now? ---

[GitHub] flink pull request #4518: [FLINK-7412][network] optimise NettyMessage.TaskEv...

2017-09-27 Thread NicoK
Github user NicoK commented on a diff in the pull request: https://github.com/apache/flink/pull/4518#discussion_r141373324 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/NettyMessage.java --- @@ -440,12 +440,11 @@ ByteBuf write(ByteBufAllocator

[GitHub] flink issue #4529: [FLINK-7428][network] avoid buffer copies when receiving ...

2017-09-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4529 I haven't had time to test this yet - I doubt that this is simply solved by using our memory segments but I need to understand the problem better. Maybe this also goes away with [flow co

[GitHub] flink issue #4591: [FLINK-7514][tests] fix BackPressureStatsTrackerITCase re...

2017-09-27 Thread NicoK
Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4591 rebased to make it mergable again ---

<    1   2   3   4   5   6   7   8   9   10   >