Vaguely makes sense. :) Wow that's an interesting corner case.
On Wed, Apr 22, 2015 at 1:57 PM, Jean-Pascal Billaud
wrote:
> I have now a fair understanding of the situation after looking at javap
> output. So as a reminder:
>
> dstream.map(tuple => {
> val w = StreamState.fetch[K,W](state.prefi
I have now a fair understanding of the situation after looking at javap
output. So as a reminder:
dstream.map(tuple => {
val w = StreamState.fetch[K,W](state.prefixKey, tuple._1)
(tuple._1, (tuple._2, w)) })
And StreamState being a very simple standalone object:
object StreamState {
def fetch[
It is kind of unexpected, i can imagine a real scenario under which it
should trigger. But obviously I am missing something :)
TD
On Tue, Apr 21, 2015 at 4:23 PM, Jean-Pascal Billaud
wrote:
> Sure. But in general, I am assuming this ""Graph is unexpectedly null
> when DStream is being serialize
Sure. But in general, I am assuming this ""Graph is unexpectedly null when
DStream is being serialized" must mean something. Under which
circumstances, such an exception would trigger?
On Tue, Apr 21, 2015 at 4:12 PM, Tathagata Das wrote:
> Yeah, I am not sure what is going on. The only way to f
Yeah, I am not sure what is going on. The only way to figure to take a look
at the disassembled bytecodes using javap.
TD
On Tue, Apr 21, 2015 at 1:53 PM, Jean-Pascal Billaud
wrote:
> At this point I am assuming that nobody has an idea... I am still going to
> give it a last shot just in case i
At this point I am assuming that nobody has an idea... I am still going to
give it a last shot just in case it was missed by some people :)
Thanks,
On Mon, Apr 20, 2015 at 2:20 PM, Jean-Pascal Billaud
wrote:
> Hey, so I start the context at the very end when all the piping is done.
> BTW a fore
Hey, so I start the context at the very end when all the piping is done.
BTW a foreachRDD will be called on the resulting dstream.map() right after
that.
The puzzling thing is why removing the context bounds solve the problem...
What does this exception mean in general?
On Mon, Apr 20, 2015 at 1:
When are you getting this exception? After starting the context?
TD
On Mon, Apr 20, 2015 at 10:44 AM, Jean-Pascal Billaud
wrote:
> Hi,
>
> I am getting this serialization exception and I am not too sure what
> "Graph is unexpectedly null when DStream is being serialized" means?
>
> 15/04/20 06:
