You could very easily strip out the BlockGenerator code from the Spark source code and use it directly in the same way the Reliable Kafka Receiver uses it. BTW, you should know that we will be deprecating the receiver based approach for the Direct Kafka approach. That is quite flexible, can give exactly-once guarantee without WAL, and is more robust and performant. Consider using it.
On Wed, Aug 5, 2015 at 5:48 PM, Sourabh Chandak <sourabh3...@gmail.com> wrote: > Hi, > > I am trying to replicate the Kafka Streaming Receiver for a custom version > of Kafka and want to create a Reliable receiver. The current implementation > uses BlockGenerator which is a private class inside Spark streaming hence I > can't use that in my code. Can someone help me with some resources to > tackle this issue? > > > > Thanks, > Sourabh >
