SQL Do Not Support Custom Trigger

2018-06-21 Thread YennieChen88
I found that flink SQL use the specific default trigger, which will not triggered until the window closes. But sometimes, we need to trigger before window closes. As the class *WindowAssigner *provides method *getDefaultTrigger *with parameter *StreamExecutionEnvironment*, how about passing a c

Slide Window Compute Optimization

2018-07-05 Thread YennieChen88
Hi, I want to use slide windows of 1 hour window size and 1 second step size. I found that once a element arrives, it will be processed in 3600 windows serially through one thread. It takes serveral seconds to finish one element processing,much more than my expection. Do I have any way to optim

Re: Slide Window Compute Optimization

2018-07-05 Thread YennieChen88
Hi Kostas and Rong, Thank you for your reply. As both of you ask for more info about my use case, I now reply in unison. My case is used for counting the number of successful login and failures within one hour, keyBy other login related attributes (e.g. ip, device, login type ...). Acco

Re: Slide Window Compute Optimization

2018-07-05 Thread YennieChen88
Hope this helps, >> Kostas >> >> [1] >> https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/stream/operators/process_function.html >> >> >> On Jul 5, 2018, at 12:12 PM, YennieChen88 < > chenyanying3@ > > wrote: >> >> H

Taskmanager process memory increasing always

2018-08-29 Thread YennieChen88
Hello, My case is counting the number of successful login and failures within 1 hour, 10 min, 5 min, 3 min, 1 min, 10 second and 1 second, keyBy login ip or device id. Based on previous counting results of different time dimensions, predict the complicance of the next login. After v

Re: Taskmanager process memory increasing always

2018-09-05 Thread YennieChen88
As far as I know, rocksdb mainly uses off-heap memory, which is hard to be controlled by JVM. Maybe you can monitor off-heap memory of taskmanager process by professional tools, such as gperftools... -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/