Re: Running multiple streaming jobs in same cluster

2017-02-12 Thread Tzu-Li (Gordon) Tai
Hi Ozan, From your description, it seems like your original huge job can be broken down into smaller disconnected graphs, with only some of the graphs requiring checkpointing / snapshots. In general, it would be a good practice to split disconnected graphs of the execution graph into multiple j

Re: Using QueryableState inside Flink jobs (and Parameter Server implementation)

2017-02-12 Thread Jinkui Shi
hi,Gábor Hermann The online parameter server is a good proposal. PS’ paper [1] have a early implement [2], and now it’s mxnet [3]. I have some thought about online PS in Flink: 1. Whether support flexible and configurable update strategy? For example, in one iteration, computing serv

Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-12 Thread Kurt Young
Congratulations Stefan! Best, Kurt On Mon, Feb 13, 2017 at 3:42 AM, Kostas Kloudas wrote: > Congratulations Stefan! > > Kostas > > > On Feb 12, 2017, at 7:17 PM, Chen Qin wrote: > > > > Congratulations! > > > > Chen > > > >> On Feb 10, 2017, at 02:00, Ufuk Celebi wrote: > >> > >> Hey everyon

Re: Flink ML - NaN Handling

2017-02-12 Thread Stavros Kontopoulos
Btw I think we should add an Imputer if we follow scikit-learn as stated here for preparing the dataset: http://scikit-learn.org/stable/modules/preprocessing.html "Imputation of Missing Values" paragraph. What do you think? Should I add it as an issue on jira? The question for NaN also holds for g

Re: flink-ml test

2017-02-12 Thread Stavros Kontopoulos
Ok I missed the IT part, if it is an integration test class you could use mvn integration-test -DwildcardSuites= the mvn target is different. On Sat, Feb 11, 2017 at 12:20 AM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Weird for me it works. I can post the output here if you wan

Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-12 Thread Kostas Kloudas
Congratulations Stefan! Kostas > On Feb 12, 2017, at 7:17 PM, Chen Qin wrote: > > Congratulations! > > Chen > >> On Feb 10, 2017, at 02:00, Ufuk Celebi wrote: >> >> Hey everyone, >> >> I'm very happy to announce that the Flink PMC has accepted Stefan >> Richter to become a committer of the

Re: Flink ML - NaN Handling

2017-02-12 Thread Stavros Kontopoulos
Ok cool thnx Till. On Sun, Feb 12, 2017 at 4:59 PM, Till Rohrmann wrote: > Hi Stavros, > > so far we've sticked mainly to scikit-learn in terms of semantics. Thus, I > would recommend to follow scikit-learn's approach to handle NaNs. > > Cheers, > Till > > On Fri, Feb 10, 2017 at 11:48 PM, Stavr

Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-12 Thread Chen Qin
Congratulations! Chen > On Feb 10, 2017, at 02:00, Ufuk Celebi wrote: > > Hey everyone, > > I'm very happy to announce that the Flink PMC has accepted Stefan > Richter to become a committer of the Apache Flink project. > > Stefan is part of the community for almost a year now and worked on >

Re: Flink ML - NaN Handling

2017-02-12 Thread Till Rohrmann
Hi Stavros, so far we've sticked mainly to scikit-learn in terms of semantics. Thus, I would recommend to follow scikit-learn's approach to handle NaNs. Cheers, Till On Fri, Feb 10, 2017 at 11:48 PM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Hello guys, > > Is there a story for t

Re: [jira] [Commented] (FLINK-5316) Make the GenericWriteAheadSink backwards compatible.

2017-02-12 Thread Kostas Kloudas
I agree. This is also the reason I have not worked on that yet. On Feb 12, 2017 12:59, "Robert Metzger (JIRA)" wrote: > > [ https://issues.apache.org/jira/browse/FLINK-5316?page= > com.atlassian.jira.plugin.system.issuetabpanels:comment- > tabpanel&focusedCommentId=15862757#comment-15862757

Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-12 Thread Fabian Hueske
Congrats Stefan and welcome! Cheers, Fabian 2017-02-12 4:19 GMT+01:00 Jark Wu : > Congrats, and thank you for your great work! > > - Jark Wu > > > 在 2017年2月11日,上午8:16,Stavros Kontopoulos 写道: > > > > Congrats! > > > > On Fri, Feb 10, 2017 at 9:11 PM, Matthias J. Sax > wrote: > > > >> -BEGIN