[GitHub] flink issue #3455: [FLINK-5134] [runtime] [FLIP-6] Aggregate ResourceSpec fo...

2017-03-13 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3455 @StephanEwen , I have submitted the modifications as we confirmed before. It needs to change some previous codes and excuse my late update because of a little busy last week. BTW, I found

[GitHub] flink issue #3467: [FLINK-4545] preparations for removing the network buffer...

2017-03-13 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3467 @NicoK ,thank you for explanation, and I already trace the code in your local branch. Wish your further change commit in global pool. @StephanEwen , thanks for further elaboration. From

[GitHub] flink issue #3455: [FLINK-5134] [runtime] [FLIP-6] Aggregate ResourceSpec fo...

2017-03-14 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3455 @StephanEwen , yes, I already noticed that today and this test can pass in my local machine. And I executed it for many loops and found that it may fail sometimes because of the sort sequence of

[GitHub] flink issue #3455: [FLINK-5134] [runtime] [FLIP-6] Aggregate ResourceSpec fo...

2017-03-15 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3455 @StephanEwen , I fixed the iteration tests to make it stable yesterday, and I think it may be related with the static incremental id in `StreamTransformation`, especially for multi tests to be

[GitHub] flink issue #3151: [FLINK-4364] [runtime] [FLIP-6] Implement TaskManager sid...

2017-03-15 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3151 @tillrohrmann , just remind to review my modifications for your free time, because this would block my next pull request of heartbeat between `TaskManager` and `ResourceManager`. Thank you

[GitHub] flink issue #3151: [FLINK-4364] [runtime] [FLIP-6] Implement TaskManager sid...

2017-03-16 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3151 @tillrohrmann , yeah, I will submit the following PRs soon after this one is merged into master, because there are some common codes base on this PR. Thank you for merging! --- If your project

[GitHub] flink issue #3555: [FLINK-6076] Refactor HeartbeatManager to extend Heartbea...

2017-03-16 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3555 @tillrohrmann , it is really good to pass the `HeartbeatListener` directly when construct the `HeartbeatManagerImpl`. When I realized the heartbeat logic between different components like TM and

[GitHub] flink pull request #3645: [FLINK-6120][Distributed Coordinator]Implement hea...

2017-03-29 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/3645 [FLINK-6120][Distributed Coordinator]Implement heartbeat logic between JobManager and ResourceManager It is part of work for Flip-6. The `HeartbeatManager` is mainly used for monitoring

[GitHub] flink issue #3645: [FLINK-6120][Distributed Coordinator]Implement heartbeat ...

2017-03-29 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3645 @tillrohrmann , I have submitted the heartbeats between `JobManager` and `ResourceManager`, wish your reviews! Sorry for a little delay because of our team outing last week. --- If your project

[GitHub] flink issue #3645: [FLINK-6120][Distributed Coordinator]Implement heartbeat ...

2017-04-19 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3645 @tillrohrmann , just a kind reminder for this last heartbeat PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink issue #3645: [FLINK-6120][Distributed Coordinator]Implement heartbeat ...

2017-04-25 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3645 Thank you for reviews! @tillrohrmann --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request #3785: [FLINK-6337][network] Remove the buffer provider f...

2017-04-26 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/3785 [FLINK-6337][network] Remove the buffer provider from PartitionRequestServerHandler Currently, `PartitionRequestServerHandler` will create a `LocalBufferPool` when the channel is registered. The

[GitHub] flink issue #3785: [FLINK-6337][network] Remove the buffer provider from Par...

2017-04-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3785 Hey @uce , I have submitted the pull request for the issue we confirmed before. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] flink pull request #3785: [FLINK-6337][network] Remove the buffer provider f...

2017-04-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/3785#discussion_r113703927 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultPartition.java --- @@ -351,15 +351,15 @@ public void

[GitHub] flink pull request #3785: [FLINK-6337][network] Remove the buffer provider f...

2017-04-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/3785#discussion_r113706078 --- Diff: flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java --- @@ -68,7 +68,6 @@ public void

[GitHub] flink issue #3785: [FLINK-6337][network] Remove the buffer provider from Par...

2017-04-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3785 Hi @uce , I have submitted the modifications of `createReadView`, and the tests and IT have passed in my private travis. I checked the failed test `HistoryServerTest.testFullArchiveLifecycle` on

[GitHub] flink pull request #3785: [FLINK-6337][network] Remove the buffer provider f...

2017-04-28 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/3785#discussion_r113945018 --- Diff: flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java --- @@ -68,7 +68,6 @@ public void

[GitHub] flink pull request #3785: [FLINK-6337][network] Remove the buffer provider f...

2017-04-28 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/3785#discussion_r113946358 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/SpillableSubpartitionTest.java --- @@ -68,7 +68,11 @@ public static

[GitHub] flink issue #3785: [FLINK-6337][network] Remove the buffer provider from Par...

2017-04-28 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3785 Hi @uce, I have submitted the modifications for the above two issues. Welcome any comments! :-) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] flink pull request #3360: [FLINK-5830][Distributed Coordination] Handle OutO...

2017-02-20 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/3360 [FLINK-5830][Distributed Coordination] Handle OutOfMemory error during process async message in akka rpc actor If caught OOM error during process async messages in **AkkaRpcActor**, it will

[GitHub] flink issue #3360: [FLINK-5830][Distributed Coordination] Handle OutOfMemory...

2017-02-20 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3360 @StephanEwen , thank you for so quick reviews! That is a good idea to add the uniform way in the utils, so we can use that in anywhere. I will fix it as your suggestions later

[GitHub] flink issue #3360: [FLINK-5830][Distributed Coordination] Handle OutOfMemory...

2017-02-20 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3360 @StephanEwen , already submit the modifications. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] flink issue #3360: [FLINK-5830][Distributed Coordination] Handle OutOfMemory...

2017-02-21 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3360 Hi @tillrohrmann , thank you for reviews and positive suggestions! I try to explain the root case of this issue first: From JobMaster side, it sends the cancel rpc message and gets

[GitHub] flink issue #3360: [FLINK-5830][Distributed Coordination] Handle OutOfMemory...

2017-02-23 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3360 @StephanEwen , if the exception is bubbled out, and cause TaskExecutor to exit as a result, I think the JobMaster can be assumed in a sane state in final based on detection of TaskExecutor failure

[GitHub] flink pull request #3399: [FLINK-5893][ResourceManager]Fix the bug of race c...

2017-02-23 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/3399 [FLINK-5893][ResourceManager]Fix the bug of race condition for removing previous JobManagerRegistration in ResourceManager The map of **JobManagerRegistration** in ResourceManager is not thread

[GitHub] flink issue #3303: [FLINK-5133][core] Support to set resource for operator i...

2017-02-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3303 Hi @StephanEwen , thanks for detail reviews of this PR and I learnt a lot from your comments. I considered all your suggestions above and submitted the modifications, including

[GitHub] flink issue #3340: [FLINK-5703][runtime] ExecutionGraph recovery via reconci...

2017-02-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3340 @StephanEwen , thanks for professional reviews! I attached the design doc in this jira for the implementation consideration. [https://docs.google.com/document/d/1rm3BYQyC

[GitHub] flink issue #3151: [FLINK-4364][runtime] Implement TaskManager side of heart...

2017-02-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3151 Hi, @tillrohrmann , thanks for so detail reviews! I got all the points and may submit the modifications tomorrow, then I will ping you! --- If your project is set up for it, you can reply to this

[GitHub] flink issue #3303: [FLINK-5133][core] Support to set resource for operator i...

2017-02-28 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3303 @StephanEwen , sorry for my carelessness of **checkNotNull**, it is a low mistake. And I passed the "clean verify" in my local machine, thank you for merging! --- If your project is

[GitHub] flink issue #3151: [FLINK-4364][runtime] Implement TaskManager side of heart...

2017-02-28 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3151 @tillrohrmann , I have submitted the modifications that covered all your suggestions. Only one issue I am not confirmed is in order to reuse the **ScheduledExecutor** in **RpcService

[GitHub] flink pull request #3455: [FLINK-5134][runtime]Aggregate ResourceSpe for cha...

2017-03-01 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/3455 [FLINK-5134][runtime]Aggregate ResourceSpe for chained operators when generating JobGraph It is a part of fine-grained resource configuration in 'flip-6'. In 'JobGraph&#x

[GitHub] flink pull request #3457: [FLINK-5135][runtime] Expand the memory types in R...

2017-03-02 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/3457 [FLINK-5135][runtime] Expand the memory types in ResourceProfile related with ResourceSpec This is a part of fine-grained resource configuration in `flip6` The `JobManager` requests slot

[GitHub] flink issue #3151: [FLINK-4364] [runtime] [FLIP-6] Implement TaskManager sid...

2017-03-05 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3151 Hi @tillrohrmann , I have submitted the modifications as you suggest one week ago. Have you received it and any other issues? --- If your project is set up for it, you can reply to this email and

[GitHub] flink issue #3467: [FLINK-4545] preparations for removing the network buffer...

2017-03-06 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3467 Hi @NicoK , I am interested in this issue and I like the way of asserting hold lock in this PR. It is really necessary to manage network buffers by framework, because it is difficult to

[GitHub] flink issue #3455: [FLINK-5134] [runtime] [FLIP-6] Aggregate ResourceSpec fo...

2017-03-08 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/3455 @StephanEwen , thank you for reviews and suggestions. I would add the related tests for `JobGraphGenerator` and try to cover different chained modes. BTW, I will also remove the

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r139075946 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -155,6 +156,57 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-14 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r139076168 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/BufferPoolFactoryTest.java --- @@ -155,6 +156,57 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-15 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r139078792 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPool.java --- @@ -159,23 +159,20 @@ public void recycle

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-15 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r139080289 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-15 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r139080305 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-15 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r139083473 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPoolTest.java --- @@ -172,44 +178,117 @@ public void

[GitHub] flink pull request #4485: [FLINK-7378][core]Create a fix size (non rebalanci...

2017-09-15 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4485#discussion_r139083671 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/SingleInputGate.java --- @@ -333,7 +333,7 @@ public void

[GitHub] flink issue #4485: [FLINK-7378][core]Create a fix size (non rebalancing) buf...

2017-09-15 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4485 @NicoK , thank you for always helpful reviews! I am very busy improving the runtime for our singles day these days. I will submit the updates for this PR later today. And I also plan to

[GitHub] flink issue #4485: [FLINK-7378][core]Create a fix size (non rebalancing) buf...

2017-09-15 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4485 @NicoK , I have submitted the updates. ---

[GitHub] flink issue #4499: [FLINK-7394][core] Implement basic InputChannel for credi...

2017-09-17 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4499 @NicoK , thanks for your reviews! This PR mainly covers four processes related with credit-based feature in `RemoteInputChannel`. I think it is better to split this PR into several small

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-18 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r139589040 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -72,6 +79,21

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-18 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r139590215 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -100,7 +122,16 @@ public

[GitHub] flink pull request #4499: [FLINK-7394][core] Implement basic InputChannel fo...

2017-09-18 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4499#discussion_r139596400 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -183,18 +214,40 @@ void

[GitHub] flink issue #4499: [FLINK-7394][core] Implement basic InputChannel for credi...

2017-09-18 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4499 @NicoK , I have submitted the updates based on the latest first PR. And this PR only concerns about the exclusive buffers management. I would submit the other issues in separate PRs later. ---

[GitHub] flink pull request #4695: Flink 7661

2017-09-21 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/4695 Flink 7661 ## What is the purpose of the change *`PartitionRequest` message adds the credit field which corresponds to the number of exclusive segments in `InputChannel`. *This

[GitHub] flink issue #4485: [FLINK-7378][core]Create a fix size (non rebalancing) buf...

2017-09-21 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4485 @NicoK , sorry for the typos. I have submitted the updates. ---

[GitHub] flink pull request #4695: [FLINK-7661][network] Add credit field in Partitio...

2017-09-21 Thread zhijiangW
Github user zhijiangW closed the pull request at: https://github.com/apache/flink/pull/4695 ---

[GitHub] flink pull request #4698: [FLINK-7661][network] Add credit field in Partitio...

2017-09-21 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/4698 [FLINK-7661][network] Add credit field in PartitionRequest message ## What is the purpose of the change *`PartitionRequest` message adds the credit field which corresponds to the number

[GitHub] flink issue #4485: [FLINK-7378][core]Create a fix size (non rebalancing) buf...

2017-09-21 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4485 @NicoK , I missed that message of `verifyAllBuffersReturned()` issue before. I have submitted the modifications of it. :) ---

[GitHub] flink issue #4499: [FLINK-7394][core] Manage exclusive buffers in RemoteInpu...

2017-09-25 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4499 @NicoK , I have fixed the above issues you mentioned. I should double check the typos after submit next time. I agree and understand your feeling of `RemoteInputChannel

[GitHub] flink issue #4698: [FLINK-7661][network] Add credit field in PartitionReques...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4698 @NicoK , thanks for your reviews! I checked the travis failure before and thought it should be existing flaky ones. I will re-trigger the tests later and see the results. ---

[GitHub] flink issue #4509: [FLINK-7406][network] Implement Netty receiver incoming p...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4509 @NicoK , thanks for your reviews! Actually there are still two PRs under submitted before this PR. One is for floating buffer interaction between `RemoteInputChannel` and `BufferPool`, the

[GitHub] flink pull request #4735: [FLINK-7699][core] Define the BufferListener inter...

2017-09-27 Thread zhijiangW
GitHub user zhijiangW opened a pull request: https://github.com/apache/flink/pull/4735 [FLINK-7699][core] Define the BufferListener interface to replace EventlListener in BufferProvider ## What is the purpose of the change *Currently the EventListener is used in

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141376844 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141377486 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141377976 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141377249 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141378307 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141380052 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -0,0 +1,283

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141380483 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -169,95 +182,19 @@ public

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141380216 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java --- @@ -193,7 +193,8 @@ private void

[GitHub] flink pull request #4735: [FLINK-7699][core] Define the BufferListener inter...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4735#discussion_r141398906 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -87,6 +88,12

[GitHub] flink pull request #4735: [FLINK-7699][core] Define the BufferListener inter...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4735#discussion_r141400676 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPoolTest.java --- @@ -211,10 +210,15 @@ public void

[GitHub] flink issue #4698: [FLINK-7661][network] Add credit field in PartitionReques...

2017-09-27 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4698 @NicoK , it already passed travis tests this time. ---

[GitHub] flink issue #4735: [FLINK-7699][core] Define the BufferListener interface to...

2017-09-28 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4735 @NicoK , I have submitted the updates based on your comments! ---

[GitHub] flink issue #4509: [FLINK-7406][network] Implement Netty receiver incoming p...

2017-09-28 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4509 @NicoK , I have submitted the modifications based on your comments. I will add the description for `CreditBasedClientHandler` module as you suggested in next PR, because the description

[GitHub] flink issue #4735: [FLINK-7699][core] Define the BufferListener interface to...

2017-09-28 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4735 Already removed the tags and added the `verify` ---

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-28 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141794108 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -390,7 +390,63 @@ public

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-09-28 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r141794461 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -390,7 +390,63 @@ public

[GitHub] flink issue #4509: [FLINK-7406][network] Implement Netty receiver incoming p...

2017-09-29 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4509 @NicoK , thanks for reviews and talking about this key point. I think I understand your point and agree with that. I will submit the modifications before Monday. ---

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-10-02 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r142120202 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPool.java --- @@ -131,6 +136,63 @@ public void recycle

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-10-02 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r142142736 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/NettyMessage.java --- @@ -235,14 +240,15 @@ void releaseBuffer

[GitHub] flink issue #4509: [FLINK-7406][network] Implement Netty receiver incoming p...

2017-10-02 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4509 @NicoK , I have fixed the other code issues. For UT, I only added one test for verifying there is no race condition between requesting and recycling floating buffers. I am not sure

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-10 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r143671614 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -37,20 +43,31 @@ import

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-10-10 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r143680178 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -306,10 +311,27 @@ public

[GitHub] flink pull request #4509: [FLINK-7406][network] Implement Netty receiver inc...

2017-10-10 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4509#discussion_r143680495 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java --- @@ -306,10 +311,27 @@ public

[GitHub] flink issue #4698: [FLINK-7661][network] Add credit field in PartitionReques...

2017-10-10 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4698 @zentol , I have rebased the latest master codes and solved the conflicts. ---

[GitHub] flink issue #4533: [FLINK-7416][network] Implement Netty receiver outgoing p...

2017-10-11 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4533 @pnowojski , thanks for your reviews! I should explain the context beforehand. We present the separate `CreditBasedClientHandler` in order not to affect the current logic in

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-11 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r144181936 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandler.java --- @@ -330,6 +330,10 @@ else if

[GitHub] flink issue #4533: [FLINK-7416][network] Implement Netty receiver outgoing p...

2017-10-11 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4533 `notifyCreditAvailable` would be called by three logics in `RemoteInputChannel`. They are `RemoteInputChannel#recycle`, `RemoteInputChannel#notifyBufferAvailable` and `RemoteInputChannel

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-11 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r144186213 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -272,4 +316,53 @@ private void

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-12 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r144233625 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -272,4 +316,53 @@ private void

[GitHub] flink issue #4533: [FLINK-7416][network] Implement Netty receiver outgoing p...

2017-10-12 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4533 @pnowojski , I have submitted the modifications. ---

[GitHub] flink issue #4552: [FLINK-7456][network] Implement Netty sender incoming pip...

2017-10-13 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4552 @pnowojski , this PR is ready for review. It covers almost all the logics of credit-based on sender side. In addition, I replace the current `PartitionRequestClientHandler` with

[GitHub] flink issue #4559: [FLINK-7468][network] Implement sender backlog logic for ...

2017-10-13 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4559 @pnowojski , this PR is ready for review. :) ---

[GitHub] flink issue #4509: [FLINK-7406][network] Implement Netty receiver incoming p...

2017-10-18 Thread zhijiangW
Github user zhijiangW commented on the issue: https://github.com/apache/flink/pull/4509 @NicoK @pnowojski , this PR is ready now. I rebased the latest master code and updated the modifications based on previous comments. ---

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-19 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r145746120 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java --- @@ -304,12 +304,21 @@ public

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-19 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r145746280 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java --- @@ -378,32 +387,47 @@ public

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-19 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r145749291 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -208,6 +211,53 @@ public

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-19 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r145750093 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -37,18 +43,29 @@ import

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-19 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r145750939 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -37,18 +43,29 @@ import

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-19 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r145752342 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -269,4 +315,49 @@ private void

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-20 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r146011321 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -269,4 +315,49 @@ private void

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-20 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r146013922 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientHandlerTest.java --- @@ -208,6 +211,53 @@ public

[GitHub] flink pull request #4533: [FLINK-7416][network] Implement Netty receiver out...

2017-10-20 Thread zhijiangW
Github user zhijiangW commented on a diff in the pull request: https://github.com/apache/flink/pull/4533#discussion_r146014273 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedClientHandler.java --- @@ -37,18 +43,29 @@ import

  1   2   3   >