When you say "done fetching documents", does it mean that you are stopping the streamingContext? (ssc.stop) or you meant completed fetching documents for a batch? If possible, you could paste your custom receiver code so that we can have a look at it.
Thanks Best Regards On Tue, Apr 7, 2015 at 8:46 AM, Hari Polisetty <hpoli...@icloud.com> wrote: > My application is running Spark in local mode and I have a Spark > Streaming Listener as well as a Custom Receiver. When the receiver is done > fetching all documents, it invokes “stop” on itself. > > I see the StreamingListener getting a callback on “onReceiverStopped” > where I stop the streaming context. > > > However, I see the following message in my logs: > > > 2015-04-06 16:41:51,193 WARN [Thread-66] > com.amazon.grcs.gapanalysis.spark.streams.ElasticSearchResponseReceiver.onStop > - Stopped receiver > > 2015-04-06 16:41:51,193 ERROR > [sparkDriver-akka.actor.default-dispatcher-17] > org.apache.spark.Logging$class.logError - Deregistered receiver for stream > 0: AlHURLEY > > 2015-04-06 16:41:51,202 WARN [Executor task launch worker-2] > org.apache.spark.Logging$class.logWarning - Stopped executor without error > > 2015-04-06 16:41:51,203 WARN [StreamingListenerBus] > org.apache.spark.Logging$class.logWarning - All of the receivers have not > deregistered, Map(0 -> > ReceiverInfo(0,ElasticSearchResponseReceiver-0,null,false,localhost,HURLEY)) > > > What am I missing or doing wrong? >
