GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/2991
[SPARK-4062][Streaming]Add ReliableKafkaReceiver in Spark Streaming Kafka
connector
Add ReliableKafkaReceiver in Kafka connector to prevent data loss if WAL in
Spark Streaming is enabled. Details and design doc can be seen in
[SPARK-4062](https://issues.apache.org/jira/browse/SPARK-4062).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark kafka-refactor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2991.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2991
----
commit c173cc1b018de65225509ddf9de6c12468450896
Author: jerryshao <[email protected]>
Date: 2014-10-23T08:19:35Z
Initial commit for reliable Kafka receiver
commit 46bb2db679478cf878ad4d1dedde3767cd69211e
Author: jerryshao <[email protected]>
Date: 2014-10-27T08:55:13Z
Code refactor and add some unit tests
commit a18c3e0286f99ea3803c5e743b78beff43d34cc0
Author: jerryshao <[email protected]>
Date: 2014-10-28T02:56:18Z
Add some comments
commit cd64678d4868b2bb53d375e27918f5f81aa651a9
Author: jerryshao <[email protected]>
Date: 2014-10-28T05:38:23Z
Change the ordering of imports
commit 5cc4cb198662cb35008d9a2e46e320f75ce35a71
Author: jerryshao <[email protected]>
Date: 2014-10-29T05:08:50Z
Address the comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
