Re: Timeout while requesting InputSplit

2015-01-29 Thread Till Rohrmann
Yes actually the timeouts should not really matter. However, an exception in the InputSplitAssigner should happen in the actor thread and thus cause the actor to stop. This should be logged by the supervisor. I just checked and the method InputSplitAssigner.getNextInputSplit is not supposed to thr

Re: Timeout while requesting InputSplit

2015-01-29 Thread Bruecke, Christoph
@Till: Yes I’m running the job on cloud-11 or better to say I’m using the yarn cluster and the flink-yarn package. I’m using flink-0.9-SNAPSHOT from the following commit [1] together with Timos patch [2]. I’ll send you a separate email with instructions where you can find the jars on cloud-11.

[jira] [Created] (FLINK-1461) Add sortPartition operator

2015-01-29 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-1461: Summary: Add sortPartition operator Key: FLINK-1461 URL: https://issues.apache.org/jira/browse/FLINK-1461 Project: Flink Issue Type: New Feature Co

Fwd: TypeSerializerInputFormat cannot determine its type automatically

2015-01-29 Thread Alexander Alexandrov
The problem seems to be that the reflection analysis cannot determine the type of the TypeSerializerInputFormat. One possible solution is to add the ResultTypeQueryable interface and force clients to explicitly set the TypeInformation. This might break code which relies on automatic type inferenc

Re: Adding non-core API features to Flink

2015-01-29 Thread Robert Metzger
Hey guys, I think the JIRA associated to this discussion needs some attention: https://issues.apache.org/jira/browse/FLINK-1452 On Mon, Jan 26, 2015 at 2:24 PM, Max Michels wrote: > +1 for having an optional flink-contrib maven dependency and an > extension repository in the long run. > > On M

Re: TypeSerializerInputFormat cannot determine its type automatically

2015-01-29 Thread Alexander Alexandrov
As a quickfix I implmeented ResultTypeQueryable for the TypeSerializerInputFormat. A PR for the 0.8 branch can be found here https://github.com/apache/flink/pull/349 Please check and let me know if there is a way to fix the problem without breaking the 0.8 line API. 2015-01-29 14:54 GMT+01:00 Al

Re: Fwd: TypeSerializerInputFormat cannot determine its type automatically

2015-01-29 Thread Timo Walther
Hey Alexander, I have looked into your issue. You can simply use env.createInput(InputFormat,TypeInformation) instead of env.readFile() then you can pass TypeInformation manually without implementing ResultTypeQueryable. Regards, Timo On 29.01.2015 14:54, Alexander Alexandrov wrote: The

[jira] [Created] (FLINK-1462) Add documentation guide for the graph API

2015-01-29 Thread Vasia Kalavri (JIRA)
Vasia Kalavri created FLINK-1462: Summary: Add documentation guide for the graph API Key: FLINK-1462 URL: https://issues.apache.org/jira/browse/FLINK-1462 Project: Flink Issue Type: Task

Re: Fwd: TypeSerializerInputFormat cannot determine its type automatically

2015-01-29 Thread Alexander Alexandrov
Alight, thanks for the hint. I suggest to close PR 349 and refine the exception with a hint HOW exactly to pass the TypeInformation instance, e.g. The type returned by the input format could not be automatically determined. Please pass the TypeInformation of the produced type explicitly via 'env.

Re: ReduceGroup fails on server

2015-01-29 Thread Alexander Alexandrov
Forget what I just said, didn't realize that it's Scala :) 2015-01-29 16:24 GMT+01:00 Alexander Alexandrov < alexander.s.alexand...@gmail.com>: > have you tried declaring your UDF classes (e.g. TotalRankDistribution) as > static? > > 2015-01-29 16:14 GMT+01:00 Arvid Heise : > >> Hi Flinker, >> >>

Re: ReduceGroup fails on server

2015-01-29 Thread Alexander Alexandrov
have you tried declaring your UDF classes (e.g. TotalRankDistribution) as static? 2015-01-29 16:14 GMT+01:00 Arvid Heise : > Hi Flinker, > > I'm currently desparetely trying to get a workflow to run remotely on a > server. The workflow works fine in the local execution environment (both > with Ex

Re: Fwd: TypeSerializerInputFormat cannot determine its type automatically

2015-01-29 Thread Timo Walther
You don't have to close the PR. The change makes sense anyways. You are right, the exception message could be improved at this point. On 29.01.2015 16:21, Alexander Alexandrov wrote: Alight, thanks for the hint. I suggest to close PR 349 and refine the exception with a hint HOW exactly to pa

Re: ReduceGroup fails on server

2015-01-29 Thread Aljoscha Krettek
Hi Arvid, this indeed looks like a ClassLoader problem. Are you running a version that you compiled yourself? If yes, could you try replacing line 49 in RuntimeStatefulSerializerFactory.java with this line: this.loader = Thread.currentThread().getContextClassLoader(); Cheers, Aljoscha On Thu, Ja

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
No I'm using the maven builds, I could try a nightly if you like. On Thu, Jan 29, 2015 at 4:34 PM, Aljoscha Krettek wrote: > Hi Arvid, > this indeed looks like a ClassLoader problem. Are you running a > version that you compiled yourself? If yes, could you try replacing > line 49 in RuntimeState

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
Nevermind, I'm going to build it myself and try your patch. On Thu, Jan 29, 2015 at 4:39 PM, Arvid Heise wrote: > No I'm using the maven builds, I could try a nightly if you like. > > On Thu, Jan 29, 2015 at 4:34 PM, Aljoscha Krettek > wrote: > >> Hi Arvid, >> this indeed looks like a ClassLoad

Design Question in Expression API

2015-01-29 Thread Aljoscha Krettek
Hi, I have to decide whether to expose the implementation or hide it from the user and would like to hear some opinions about that. The expression operations operate on DataSet[Row], where Row is basically a wrapper for an array of elements of different types. The expression API system keeps track

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
Quickfix did not help :/ Any other idea? On Thu, Jan 29, 2015 at 4:45 PM, Arvid Heise wrote: > Nevermind, I'm going to build it myself and try your patch. > > On Thu, Jan 29, 2015 at 4:39 PM, Arvid Heise > wrote: > >> No I'm using the maven builds, I could try a nightly if you like. >> >> On Th

Re: ReduceGroup fails on server

2015-01-29 Thread Arvid Heise
A quick and dirty hack revealed that the loader is null at the time of failure with and without your patch. On Thu, Jan 29, 2015 at 5:05 PM, Arvid Heise wrote: > Quickfix did not help :/ > Any other idea? > > On Thu, Jan 29, 2015 at 4:45 PM, Arvid Heise > wrote: > >> Nevermind, I'm going to bui

Re: ReduceGroup fails on server

2015-01-29 Thread Stephan Ewen
Thanks for reporting this, Arvid. I remember that we saw something similar a while back. I will take a look at this later today (I am in PST zone now), can hopefully fix this. Greetings, Stephan

Community vote for Hadoop Summit result

2015-01-29 Thread Márton Balassi
Hi everyone, Thanks for your support for the Flink talks at the community choice for the next Hadoop Summit Europe. The results are out. [1] Our submission "Real-Time Stream Processing with Apache Flink” [2] has been selected as the winner in the Future Hadoop track by Community Choice voting tha

Re: Community vote for Hadoop Summit result

2015-01-29 Thread Henry Saputra
W00t! Congrats guys! On Thu, Jan 29, 2015 at 4:06 PM, Márton Balassi wrote: > Hi everyone, > > Thanks for your support for the Flink talks at the community choice for the > next Hadoop Summit Europe. The results are out. [1] > > Our submission "Real-Time Stream Processing with Apache Flink” [2] h