Re: Parallelism of Custom receiver in spark

2015-07-26 Thread Michal Čizmazia
#1 see https://spark.apache.org/docs/latest/streaming-programming-guide.html#level-of-parallelism-in-data-receiving #2 By default, all input data and persisted RDDs generated by DStream transformations are automatically cleared. Spark Streaming decides when to clear the data based on the transform

Parallelism of Custom receiver in spark

2015-07-25 Thread anshu shukla
1 - How to increase the level of *parallelism in spark streaming custom RECEIVER* . 2 - Will ssc.receiverstream(/**anything //) will *delete the data stored in spark memory using store(s) * logic . -- Thanks & Regards, Anshu Shukla