Re: Job ID

2017-05-09 Thread Fritz Budiyanto
Hi Gordon, Please add job’s computed SHA-1 as well if possible, so I can have the client code to automate uploading / replacing job if the computed SHA-1 is different — Fritz > On May 8, 2017, at 10:18 PM, Tzu-Li (Gordon) Tai wrote: > > Hi Joe, > > AFAIK, this currently isn’t possible throu

Re: Joining on multiple row values produced by TableFunction

2017-05-09 Thread Samuel Doyle
Great, thanks for the information! Looking forward to the new release. S.D. On Tue, May 9, 2017 at 1:40 PM Fabian Hueske wrote: > Flink 1.2 does not support windows in SQL. > The upcoming Flink 1.3, which is currently in testing phase and should be > released in a few weeks, allows you define s

Re: Joining on multiple row values produced by TableFunction

2017-05-09 Thread Fabian Hueske
Flink 1.2 does not support windows in SQL. The upcoming Flink 1.3, which is currently in testing phase and should be released in a few weeks, allows you define several types of windows in SQL. There are three types of group windows [1] (note that the syntax for the time indicators changed a bit, c

Re: Joining on multiple row values produced by TableFunction

2017-05-09 Thread Samuel Doyle
Is it possible to define windows in such a way that I can use straight sql like with the table environment vs the sql 'like' syntax you use when using the table api? My project is based off Java. Thanks On Tue, May 9, 2017 at 10:34 AM Samuel Doyle wrote: > I was able to get it work, albeit prob

Re: Job ID

2017-05-09 Thread Joe Olson
That's good news! Do you have a JIRA for that feature, so I can keep an eye on it? _ From: Tzu-Li (Gordon) Tai mailto:tzuli...@apache.org>> Sent: Tuesday, May 9, 2017 12:18 AM Subject: Re: Job ID To: mailto:user@flink.apache.org>> Hi Joe, AFAIK, this currently isn’t

Re: Joining on multiple row values produced by TableFunction

2017-05-09 Thread Samuel Doyle
I was able to get it work, albeit probably not the best solution, by calling the table function twice and assigning different column aliases On Tue, May 9, 2017, 12:43 AM Fabian Hueske wrote: > Hi, > > it might be a problem, that your TableFunction is called like an attribute > (both are called

Re: ConnectedStream keyby issues

2017-05-09 Thread Aljoscha Krettek
Hi, Yes, if you use ProcessFunction or CoProcessFunction you can (and should) set a timer to clean up state. Best, Aljoscha > On 7. May 2017, at 09:28, yunfan123 wrote: > > But what happened if some data can't be merged forever ? > The state will be saved forever? > Can I set a timeout? > >

Re: AllWindowed vs Windowed with 1 key

2017-05-09 Thread Stefan Richter
That is interesting, because already in Flink 1.1.x, windowAll() is implemented as input.keyBy(new DummyKeySelector()).window(). Are you using event time or processing or event time, and most important, do the execution graphs in the web frontend look different in both variants? > Am 08.05.2017

Re: Programmatic management of Flink jobs

2017-05-09 Thread Aljoscha Krettek
Hi, Not if it doesn’t happen to frequently. Otherwise you would spend most of your time restarting jobs. Best, Aljoscha > On 9. May 2017, at 08:23, Moiz S Jinia wrote: > > Thanks. Do you foresee any problems with canceling/starting the jobs whenever > the patterns change? > > Moiz > > On We

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

2017-05-09 Thread Robert Metzger
Chesnay, I would propose to look into a CI solution for Windows. I'll try out appveyor.com for my Flink fork and report back how well it works (let me know if you want to look into it). On Tue, May 9, 2017 at 10:05 AM, Stephan Ewen wrote: > @Renjie What functionality are you looking for? Most st

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

2017-05-09 Thread Renjie Liu
I'm just looking forward basic functionalities, e.g. a single web port for all flink jobs. On Tue, May 9, 2017 at 4:05 PM Stephan Ewen wrote: > @Renjie What functionality are you looking for? Most stuff should work in > the normal master branch. > > > On Tue, May 9, 2017 at 10:02 AM, Renjie Liu

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

2017-05-09 Thread Stephan Ewen
@Renjie What functionality are you looking for? Most stuff should work in the normal master branch. On Tue, May 9, 2017 at 10:02 AM, Renjie Liu wrote: > Stephan: > This means that we can use FLIP 6 for basic functionality now? Any > document for setting up that? Will FLIP 6 be finished in rele

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

2017-05-09 Thread Renjie Liu
Stephan: This means that we can use FLIP 6 for basic functionality now? Any document for setting up that? Will FLIP 6 be finished in release 1.3? On Tue, May 9, 2017 at 3:43 PM Stephan Ewen wrote: > @Renjie Liu: A good part of the FLIP-6 code is in there, but not all. > > It does run well on Ya

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

2017-05-09 Thread Chesnay Schepler
This RC (and 1.4-SNAPSHOT for that matter) cannot be compiled on Windows due to the rat-plugin not detecting certain test savepoints as binary files. The files in question are the "savepoints" created by the StreamOperatorTestHarness. This is not a new problem and has happened before, but we n

Re: Joining on multiple row values produced by TableFunction

2017-05-09 Thread Fabian Hueske
Hi, it might be a problem, that your TableFunction is called like an attribute (both are called fields). If that's not the case, can you provide more information? - Is there an error message? - Yes: what's the error message - No: what's the output and what would you expect? - Is this for batc

Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)

2017-05-09 Thread Stephan Ewen
@Renjie Liu: A good part of the FLIP-6 code is in there, but not all. It does run well on Yarn, Mesos, Docker, etc. We need to finish the FLIP-6 work mainly for full elasticity. On Tue, May 9, 2017 at 5:24 AM, Renjie Liu wrote: > Hi, does this include the FLIP6? > > On Tue, May 9, 2017 at 2:2

Re: Re: ElasticsearchSink on DataSet

2017-05-09 Thread Flavio Pompermaier
Just one note: I took a look at your connector and it doesn't provide any failure handling mechanism that is very useful for us. Maybe it could worth to add ActionRequestFailureHandler as provided now by the current ES streaming connector and introduced by commit https://github.com/apache/flink/com