Hi Sean,
Thanks for the response. I changed from map to flatMap and in the function
return a list as below
if (temp != "+" and re.match("[01459]", q)):
return [(year,temp)]
else:
return []
Thanks,
Praveen
On Mon, Sep 22, 2014 at 9:26 PM, Sean Owen wrote:
> If your map() sometimes
If your map() sometimes does not emit an element, then you need to
call flatMap() instead, and emit Some(value) (or any collection of
values) if there is an element to return, or None otherwise.
On Mon, Sep 22, 2014 at 4:50 PM, Praveen Sripati
wrote:
> During the map based on some conditions if s
During the map based on some conditions if some of the rows are ignored
(without any transformation) then then there is a record by None in the
output RDD for the ignored records. And reduceByKey is not able to handle
this type of None record and so the exception. I tried filter, but it is
also not
Hi,
I am writing a Spark program in Python to find the maximum temperature for
a year, given a weather dataset. The below program throws an error when I
try to execute the Spark program.
TypeError: 'NoneType' object is not iterable
org.apache.spark.api.python.PythonRDD$$anon$1.read(PythonRDD.sc
