Re: Will to contribute

2015-10-07 Thread Stephan Ewen
It depends a bit what parts/components you like: APIs / Libraries (Graph / ML) / Runtime / SQL / Relational / ... ? In general, any JIRA issue not assigned is good to grab, or anything you notice that is not smooth when you use the system. If you have a favorite data store system and would like t

Re: Towards Flink 0.10

2015-10-07 Thread Stephan Ewen
Hey Alexey! I would like to get this in very soon, but I am not sure it will make it until the feature freeze fro 0.10. In that case, it should be coming up the next maintenance release after that... Greetings, Stephan On Tue, Oct 6, 2015 at 12:18 PM, Alexey Sapozhnikov wrote: > Hello everyo

Re: Cannot run Storm WordCountLocal

2015-10-07 Thread Stephan Ewen
The mini cluster is started with not enough slots. I would guess that it is luck that the IT case works because the test environment probably uses a simple default of 4 slots or so for the mini cluster. The LocalEnvironment in the DataSet API actually makes a pass over the operators to see how man

Cannot run Storm WordCountLocal

2015-10-07 Thread Matthias J. Sax
Hi, I cannot run WordCountLocal example of flink-storm-example in Eclipse. The ITCase for it works, but it is using a test environment. If I start WordCountLocal directly it uses FlinkMiniCluster and I get the following exception: > org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailabl

Test Failed in flink-streaming-core

2015-10-07 Thread Matthias J. Sax
Not sure which test failed. It might be Running > org.apache.flink.streaming.api.operators.windowing.WindowingITCase but I am not sure. https://travis-ci.org/apache/flink/jobs/84184270 -Matthias signature.asc Description: OpenPGP digital signature

[jira] [Created] (FLINK-2832) Failing test: RandomSamplerTest.testReservoirSamplerWithReplacement

2015-10-07 Thread Vasia Kalavri (JIRA)
Vasia Kalavri created FLINK-2832: Summary: Failing test: RandomSamplerTest.testReservoirSamplerWithReplacement Key: FLINK-2832 URL: https://issues.apache.org/jira/browse/FLINK-2832 Project: Flink

Re: Will to contribute

2015-10-07 Thread Dawid Wysakowicz
Thanks a lot Chiwan! Any suggestions for some issues I could work on? 2015-10-07 20:27 GMT+02:00 Chiwan Park : > Hi Dawid, > > Welcome to Flink community! :) > I left some comments for your pull request. > > > On Oct 7, 2015, at 7:29 PM, Dawid Wysakowicz > wrote: > > > > Hi all, > > > > I liked

Re: Will to contribute

2015-10-07 Thread Chiwan Park
Hi Dawid, Welcome to Flink community! :) I left some comments for your pull request. > On Oct 7, 2015, at 7:29 PM, Dawid Wysakowicz > wrote: > > Hi all, > > I liked the idea behind flink and I would be happy if I could help. On > project's github page you mentioned you can help find some star

Will to contribute

2015-10-07 Thread Dawid Wysakowicz
Hi all, I liked the idea behind flink and I would be happy if I could help. On project's github page you mentioned you can help find some starter tasks that I would appreciate. I tried to search some on my own I even created a PR for FLINK-2156, but I couldn't find any bigger one. Looking forward

[jira] [Created] (FLINK-2831) Failinf Test: WrapperSetupHelperTest

2015-10-07 Thread Vasia Kalavri (JIRA)
Vasia Kalavri created FLINK-2831: Summary: Failinf Test: WrapperSetupHelperTest Key: FLINK-2831 URL: https://issues.apache.org/jira/browse/FLINK-2831 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-2830) Add Fold Window Operation for new Windowing API

2015-10-07 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-2830: --- Summary: Add Fold Window Operation for new Windowing API Key: FLINK-2830 URL: https://issues.apache.org/jira/browse/FLINK-2830 Project: Flink Issue Typ

Re: [DISCUSS] Introducing a review process for pull requests

2015-10-07 Thread Fabian Hueske
I like the idea of meeting once a week to discuss about PRs as well. Regarding lingering PRs, you can simply sort the Flink PRs in Github by "least recently updated" Cheers, Fabian 2015-10-07 16:48 GMT+02:00 Theodore Vasiloudis < theodoros.vasilou...@gmail.com>: > > > > Could we maybe do a "PR

Re: [DISCUSS] Introducing a review process for pull requests

2015-10-07 Thread Theodore Vasiloudis
> > Could we maybe do a "PR overall status assessment" once per week or so, > where we find those problematic PRs and try to assign them / close them? I like this idea, as it would raise awareness about lingering PRs. Does anybody know if there is some way to integrate this into JIRA, so we can

[jira] [Created] (FLINK-2829) Confusing error message when Flink cannot create enough task threads

2015-10-07 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-2829: - Summary: Confusing error message when Flink cannot create enough task threads Key: FLINK-2829 URL: https://issues.apache.org/jira/browse/FLINK-2829 Project: Flink

Re: TM failure when deploying a large number of sources

2015-10-07 Thread Gyula Fóra
Alright, I am creating one. Stephan Ewen ezt írta (időpont: 2015. okt. 7., Sze, 15:44): > I think the error message could have been better, though... > > This actually warrants a JIRA issue... > > On Wed, Oct 7, 2015 at 2:44 PM, Gyula Fóra wrote: > > > Thanks! > > > > Yes, it was indeed a memor

Re: TM failure when deploying a large number of sources

2015-10-07 Thread Stephan Ewen
I think the error message could have been better, though... This actually warrants a JIRA issue... On Wed, Oct 7, 2015 at 2:44 PM, Gyula Fóra wrote: > Thanks! > > Yes, it was indeed a memory issue: > java.lang.OutOfMemoryError: unable to create new native thread > at java.lang.Thread.start0(Nat

Running the job in Flink webInterface

2015-10-07 Thread santosh_rajaguru
Hi all, I am experiencing some problem while writing to the jena-hbase. I am using a custom OutPutformat for writing. This reads a configuration file from the classpath. it works while running the job from eclipse. but it throws an exception (hbaserdf.ttl file cannot found) while running it from f

[jira] [Created] (FLINK-2828) Add interfaces for Table API input formats

2015-10-07 Thread Timo Walther (JIRA)
Timo Walther created FLINK-2828: --- Summary: Add interfaces for Table API input formats Key: FLINK-2828 URL: https://issues.apache.org/jira/browse/FLINK-2828 Project: Flink Issue Type: New Featur

Re: TM failure when deploying a large number of sources

2015-10-07 Thread Gyula Fóra
Thanks! Yes, it was indeed a memory issue: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at org.apache.flink.runtime.taskmanager.Task.startTaskThread(Task.java:415) at org.apache.flink.runtime.tas

Re: TM failure when deploying a large number of sources

2015-10-07 Thread Stephan Ewen
Any further information from teh log? If you create so many tasks (8 x 80) on one machine, the JVM often has not enough memory reserved for the stack space to create enough threads (1-2 threads per task)... On Wed, Oct 7, 2015 at 2:13 PM, Gyula Fóra wrote: > Hey guys, > > I am writing a job whi

TM failure when deploying a large number of sources

2015-10-07 Thread Gyula Fóra
Hey guys, I am writing a job which involves creating many different sources to read data from (in this case 80 sources wiht the parallelism of 8 each, running locally on my mac). I cannot create less unfortunately. The problem is that the job fails while deploying the tasks with the following exc

Re: streaming GroupBy + Fold

2015-10-07 Thread Aljoscha Krettek
Hi, I ran it using the attached TimeShift.java and I didn't get any key cross-talk. Could you please try my example, or verify that the problem still persists on your side? I replaced the source by a source that just creates random strings. On Tue, 6 Oct 2015 at 09:56 Martin Neumann wrote: >

Re: [DISCUSS] Introducing a review process for pull requests

2015-10-07 Thread Vasiliki Kalavri
Hey, I agree that we need to organize the PR process. A PR management tool would be great. However, it seems to me that the shepherding process described is -more or less- what we've already been doing. There is usually a person that reviews the PR and kind-of drives the process. Maybe making thi

[DISCUSS] flink-external

2015-10-07 Thread Matthias J. Sax
Hi, many people are building quite exiting stuff on top of Flink. It is hard to keep an good overview on what stuff is available and what not. What do you think about starting a second git repository "flink-external" that collects all those code? The ideas would be to collect stuff in a central p

Re: [DISCUSS] Introducing a review process for pull requests

2015-10-07 Thread Matthias J. Sax
Ok. That makes sense. So most people will need to change behavior and start discussions in JIRA and not over dev list. Furthermore, issues list must be monitored more carefully... (I personally, watch dev carefully and only skip over issues list right now) -Matthias On 10/07/2015 10:22 AM, Fabian

Re: [DISCUSS] Introducing a review process for pull requests

2015-10-07 Thread Márton Balassi
+1 One minor comment: I suppose you implicitly mean that a committer can shepherd her own PR. On Wed, Oct 7, 2015 at 10:22 AM, Fabian Hueske wrote: > @Matthias: That's a good point. Each PR should be backed by a JIRA issue. > If that's not the case, we have to make the contributor aware of that

Re: [DISCUSS] Introducing a review process for pull requests

2015-10-07 Thread Fabian Hueske
@Matthias: That's a good point. Each PR should be backed by a JIRA issue. If that's not the case, we have to make the contributor aware of that rule. I'll update the process to keep all discussions in JIRA (will be mirrored to issues ML), OK? @Theo: You are right. Adding this process won't be the

Re: Extending and improving our "How to contribute" page

2015-10-07 Thread Fabian Hueske
Thanks for the feedback everybody. I updated the PR and would like to merge it later today if there are no more comments. Cheers, Fabian 2015-10-05 14:09 GMT+02:00 Fabian Hueske : > Hi, > > I opened a PR with the discussed changes [1]. > Please review, give feedback, and suggest changes. > > Ch