GitHub user cleaton opened a pull request:
https://github.com/apache/spark/pull/4338
[STREAMING] SPARK-4986 Wait for receivers to deregister and receiver job to
terminate
A slow receiver might not have enough time to shutdown cleanly even when
graceful shutdown is used. This PR extends graceful waiting to make sure all
receivers have deregistered and that the receiver job has terminated.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cleaton/spark stopreceivers
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4338.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4338
----
commit d179372ae1d93adf67f8cd7ea818c443c584a8a0
Author: Jesper Lundgren <[email protected]>
Date: 2015-01-25T14:31:19Z
Add graceful shutdown unit test covering slow receiver onStop
Signed-off-by: Jesper Lundgren <[email protected]>
commit 9a9ff884d878824548f9a08d724f60b0c14f8310
Author: Jesper Lundgren <[email protected]>
Date: 2015-01-30T07:34:03Z
wait for receivers to shutdown and receiver job to terminate
Signed-off-by: Jesper Lundgren <[email protected]>
commit 3d0bd3501d24a8ab5e10e1867b464d471b3bbb67
Author: Jesper Lundgren <[email protected]>
Date: 2015-01-30T07:41:00Z
switch boleans to match running status instead of terminated
Signed-off-by: Jesper Lundgren <[email protected]>
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
