Re: ReduceGroup fails on server

2015-02-02 Thread Aljoscha Krettek
Hi, we have some bug fixes queued up. So a 0.8.1 bug fix release should be expected in the upcoming weeks. Cheers, Aljoscha On Mon, Feb 2, 2015 at 12:48 PM, Arvid Heise wrote: > OK, patch indeed worked for my workflow. Thank you very much. > > Any idea when this patch will be in a non-snapshot?

Re: ReduceGroup fails on server

2015-02-02 Thread Arvid Heise
OK, patch indeed worked for my workflow. Thank you very much. Any idea when this patch will be in a non-snapshot? Best, Arvid On Fri, Jan 30, 2015 at 6:04 PM, Arvid Heise wrote: > Sorry building the 0.9 branch took longer than expected. I will follow > that up on Monday. > > Alternatively I w

Re: ReduceGroup fails on server

2015-01-30 Thread Arvid Heise
Sorry building the 0.9 branch took longer than expected. I will follow that up on Monday. Alternatively I would be grateful for a 0.8.0 patch ;) Best, Arvid On Fri, Jan 30, 2015 at 5:09 PM, Aljoscha Krettek wrote: > Hi Arvid, > I have a fix that I hope fixes your problem: > https://github.com

Re: ReduceGroup fails on server

2015-01-30 Thread Aljoscha Krettek
Hi Arvid, I have a fix that I hope fixes your problem: https://github.com/aljoscha/flink/tree/serializer-factories-fix Could you try building it and running your example? Cheers, Aljoscha On Fri, Jan 30, 2015 at 3:30 PM, Aljoscha Krettek wrote: > We have a bit of a divide in how we handle TypeS

Re: ReduceGroup fails on server

2015-01-30 Thread Aljoscha Krettek
We have a bit of a divide in how we handle TypeSerializer and TypeComparator: TypeSerializer does not handle duplication but relies on outside code (RuntimeStatefulSerializerFactory) to perform the duplication. TypeComparator does duplication itself. There is also the RuntimeComparatorFactory. This

Re: ReduceGroup fails on server

2015-01-30 Thread Aljoscha Krettek
Maybe we can get rid of the serializer factories altogether. We could enhance the Serializers with a method duplicate() that does nothing for stateless serializers and does a deep copy for stateful serializers. This would also consolidate all the knowledge about stateful/stateless in one place. On

Re: ReduceGroup fails on server

2015-01-30 Thread Aljoscha Krettek
Of course it doesn't work. The ClassLoader is declared transient in the serializer factory, so it is Null once the factory has been serialized/deserialized once. I open a Jira issue: https://issues.apache.org/jira/browse/FLINK-1463 On Thu, Jan 29, 2015 at 9:43 PM, Stephan Ewen wrote: > Thanks for

Re: ReduceGroup fails on server

2015-01-29 Thread Stephan Ewen
Thanks for reporting this, Arvid. I remember that we saw something similar a while back. I will take a look at this later today (I am in PST zone now), can hopefully fix this. Greetings, Stephan

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
A quick and dirty hack revealed that the loader is null at the time of failure with and without your patch. On Thu, Jan 29, 2015 at 5:05 PM, Arvid Heise wrote: > Quickfix did not help :/ > Any other idea? > > On Thu, Jan 29, 2015 at 4:45 PM, Arvid Heise > wrote: > >> Nevermind, I'm going to bui

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
Quickfix did not help :/ Any other idea? On Thu, Jan 29, 2015 at 4:45 PM, Arvid Heise wrote: > Nevermind, I'm going to build it myself and try your patch. > > On Thu, Jan 29, 2015 at 4:39 PM, Arvid Heise > wrote: > >> No I'm using the maven builds, I could try a nightly if you like. >> >> On Th

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
Nevermind, I'm going to build it myself and try your patch. On Thu, Jan 29, 2015 at 4:39 PM, Arvid Heise wrote: > No I'm using the maven builds, I could try a nightly if you like. > > On Thu, Jan 29, 2015 at 4:34 PM, Aljoscha Krettek > wrote: > >> Hi Arvid, >> this indeed looks like a ClassLoad

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
No I'm using the maven builds, I could try a nightly if you like. On Thu, Jan 29, 2015 at 4:34 PM, Aljoscha Krettek wrote: > Hi Arvid, > this indeed looks like a ClassLoader problem. Are you running a > version that you compiled yourself? If yes, could you try replacing > line 49 in RuntimeState

Re: ReduceGroup fails on server

2015-01-29 Thread Aljoscha Krettek
Hi Arvid, this indeed looks like a ClassLoader problem. Are you running a version that you compiled yourself? If yes, could you try replacing line 49 in RuntimeStatefulSerializerFactory.java with this line: this.loader = Thread.currentThread().getContextClassLoader(); Cheers, Aljoscha On Thu, Ja

Re: ReduceGroup fails on server

2015-01-29 Thread Alexander Alexandrov
Forget what I just said, didn't realize that it's Scala :) 2015-01-29 16:24 GMT+01:00 Alexander Alexandrov < alexander.s.alexand...@gmail.com>: > have you tried declaring your UDF classes (e.g. TotalRankDistribution) as > static? > > 2015-01-29 16:14 GMT+01:00 Arvid Heise : > >> Hi Flinker, >> >>

Re: ReduceGroup fails on server

2015-01-29 Thread Alexander Alexandrov
have you tried declaring your UDF classes (e.g. TotalRankDistribution) as static? 2015-01-29 16:14 GMT+01:00 Arvid Heise : > Hi Flinker, > > I'm currently desparetely trying to get a workflow to run remotely on a > server. The workflow works fine in the local execution environment (both > with Ex