回复:How can I count the element in datastream

2018-01-14 Thread 邓俊华
Yes, I want count all the elements. But I can't do cumulative.eg:    distinctOrder.map(new MapFunction() { @Override public Object map(Order value) throws Exception { return null; } }).setParallelism(1).print(); -

Re: Questions about managed operator state

2018-01-14 Thread Boris Lublinsky
Thanks Fabian, After I switched to var it compiles, but its not initialized. @transient private var currentModels : MapState[String, Model] = _ Assignes null to MapState. Do I create an empty hashMap there? Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/ >

Re: Aggregation using event timestamp than clock window

2018-01-14 Thread Gary Yao
Hi Rohan, I am not sure if I fully understand your problem. For example, if you receive an event with a start time of 4:50 and an end time of 5:30, do you want the "usage" from 4:50 - 5:00 to be included in the 4:00 - 5:00 window? What if the event had an end time of 5:31? Do you then want to igno

Re: Flink support for Microsoft Windows

2018-01-14 Thread Chesnay Schepler
What i do know is that you can run a Flink cluster on a single machine, submit jobs etc, run the webUI and stuff. You can use the plain cmd, cygwin or WSL. You can develop on/against Flink on Windows, run the majority of tests and execute jobs in the IDE. I can't really tell how well it works

Re: CEP issue in 1.3.2. Does 1.4 fix this ?

2018-01-14 Thread Vishal Santoshi
Will do. On Sun, Jan 14, 2018 at 10:23 AM, Fabian Hueske wrote: > We don't have a schedule for bugfix releases but do them based on need. > AFAIK, a discussion about a 1.4.1 release has not been started yet. > > Would you like to kick that off by sending a mail to the dev mailing list? > > > 201

Re: Questions about managed operator state

2018-01-14 Thread Fabian Hueske
Hi Boris, the CheckpointedRestoring interface was removed in Flink 1.4.0 (and deprecated in an earlier version). Unfortunately, the docs have not been updated accordingly. I'll open a JIRA to fix this. The replacements for CheckpointedRestoring are the CheckpointedFunction or ListCheckpointed inte

Re: Keyed State

2018-01-14 Thread Fabian Hueske
Sure. A CoProcessFunction is executed in parallel by running multiple instances of the CoProcessFunction. Each instance runs in a separate TaskManager slot and is responsible for a subset of all keys. Keys are assigned by hash partitioning to function instances. All calls to methods of an individ

Re: CEP issue in 1.3.2. Does 1.4 fix this ?

2018-01-14 Thread Fabian Hueske
We don't have a schedule for bugfix releases but do them based on need. AFAIK, a discussion about a 1.4.1 release has not been started yet. Would you like to kick that off by sending a mail to the dev mailing list? 2018-01-12 16:41 GMT+01:00 Vishal Santoshi : > Thanks. We will. > >When is

cleaning yarn logs for long-running applications

2018-01-14 Thread Soheil Pourbafrani
Hi, I want to use Yarn as cluster manager for running Flink applications, but I'm worried about how Flink or Yarn handle local logs in each machine. Does they clean aged logs for a long-running application? If not, it's possible the local storage get full!!!