I tried below code but still carrying out the action even though there
is no new data.
JavaPairInputDStream<LongWritable, Text> input =
ssc.fileStream(iFolder, LongWritable.class,Text.class,
TextInputFormat.class);
if(input != null){
//do some action if it is not empty
}
On 21 October 2015 at 18:00, diplomatic Guru <[email protected]>
wrote:
>
> Hello All,
>
> I have a Spark Streaming job that should do some action only if the RDD
> is not empty. This can be done easily with the spark batch RDD as I could
> .take(1) and check whether it is empty or not. But this cannot been done
> in Spark Streaming DStrem
>
>
> JavaPairInputDStream<LongWritable, Text> input = ssc.fileStream(iFolder,
> LongWritable.class,Text.class, TextInputFormat.class);
>
> if(inputLines!=null){
> //do some action if it is not empty
> }
>
> Any ideas please?
>
>
>
>
