Python : StreamingContext isn't created properly

[Paolo Patierno]

Hi all,

I wrote following function in Python :

def createStreamingContext():
    conf = (SparkConf().setMaster("local[2]").setAppName("my_app"))
    conf.set("spark.streaming.receiver.writeAheadLog.enable", "true")

    sc = SparkContext(conf=conf)
    ssc = StreamingContext(sc, 1)
    ssc.checkpoint("/tmp/spark-streaming")

    ....
    ....

    ssc
But I don't know why launching it with spark_submit gives me following error :

AttributeError: 'NoneType' object has no attribute 'checkpoint'

I can imagine that ssc wasn't created properly but ... why ?

Thanks,
Paolo.


Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience