i remember us having issues with joda classes not serializing property and
coming out null "on the other side" in tasks
On Thu, Nov 12, 2015 at 10:12 AM, Ted Yu wrote:
> Even if log4j didn't work, you can still get some clue by wrapping the
> following call with try block:
>
> currentDate
Even if log4j didn't work, you can still get some clue by wrapping the
following call with try block:
currentDate = currentDate.plusDays(1)
catching NPE and rethrowing with an exception that shows the value of
currentDate
Cheers
On Thu, Nov 12, 2015 at 1:56 AM, Romain Sagean
wrote:
>
I Still can't make the logger work inside a map function. I can use
"logInfo("")" in the main but not in the function. Anyway I rewrite my
program to use java.util.Date instead joda time and I don't have NPE
anymore.
I will stick with this solution for the moment even if I find java Date
ugly.
Th
In case you need to adjust log4j properties, see the following thread:
http://search-hadoop.com/m/q3RTtJHkzb1t0J66&subj=Re+Spark+Streaming+Log4j+Inside+Eclipse
Cheers
On Tue, Nov 10, 2015 at 1:28 PM, Ted Yu wrote:
> I took a look at
> https://github.com/JodaOrg/joda-time/blob/master/src/main/j
I took a look at
https://github.com/JodaOrg/joda-time/blob/master/src/main/java/org/joda/time/DateTime.java
Looks like the NPE came from line below:
long instant = getChronology().days().add(getMillis(), days);
Maybe catch the NPE and print out the value of currentDate to see if there
is more clue
see below a more complete version of the code.
the firstDate (previously minDate) should not be null, I even added an
extra "filter( _._2 != null)" before the flatMap and the error is
still there.
What I don't understand is why I have the error on
dateSeq.las.plusDays and not on dateSeq.last.isBef
Can you show the stack trace for the NPE ?
Which release of Spark are you using ?
Cheers
On Tue, Nov 10, 2015 at 8:20 AM, romain sagean
wrote:
> Hi community,
> I try to apply the function below during a flatMapValues or a map but I
> get a nullPointerException with the plusDays(1). What did I
