Have you tried simply ssc.checkpoint("checkpoint”)? This should create it in
the local folder, has always worked for me when in development on local mode.
For the others (/tmp/..) make sure you have rights to write there.
-adrian
From: srungarapu vamsi
Date: Tuesday, September 22, 2015 at 7:59 AM
To: user
Subject: Invalid checkpoint url
I am using reduceByKeyAndWindow (with inverse reduce function) in my code.
In order to use this, it seems the checkpointDirectory which i have to use
should be hadoop compatible file system.
Does that mean that, i should setup hadoop on my system.
I googled about this and i found in a S.O answer that i need not setup hdfs but
the checkpoint directory should be HDFS copatible.
I am a beginner in this area. I am running my spark streaming application on
ubuntu 14.04, spark -1.3.1
If at all i need not setup hdfs and ext4 is hdfs compatible, then how does my
checkpoint directory look like?
i tried all these:
ssc.checkpoint("/tmp/checkpoint")
ssc.checkpoint("hdfs:///tmp/checkpoint")
ssc.checkpoint("file:///tmp/checkpoint")
But none of them worked for me.
--
/Vamsi
