Manual trigger the window in fold operator or incremental aggregation

2018-10-17 Thread zhen li
Hi all: How can I trigger the window manually in fold operator or incremental aggregation? For example, when some conditions is meet,althouth the count window or time window is not reach

share the big data structure between slots

2018-08-29 Thread zhen li
Hi all: When I broadcast the big config stream,every parallel instance should store the data, cause waste of the momery and time. Is there some method to make the slots to share the big data structure within the taskmanager? Here is the doc: By adjusting the number of task slots, users can

How to set global config in the rich functions

2018-07-04 Thread zhen li
Hi: I want to set some configs in the source functions use getRuntimeContext().getExecutionConfig().setGlobalJobParameters(parameterTool) And used the configs in the downstream operators such as filter function through the getGlobalJobParameters, But it returns null pointer exception.Is

Re: How to use broadcast variables in data stream

2018-06-30 Thread zhen li
You can use DataStream.broadcast() in Flink 1.3 but it has a few limitations. For example, you cannot connect a keyed and a broadcasted stream. 2018-06-21 11:58 GMT+02:00 zhen li mailto:lizhenm...@hotmail.com>>: Thanks for your reply. But broadcast state seems not supported in Flink-1.3 . I found

Re: How to use broadcast variables in data stream

2018-06-21 Thread zhen li
Thanks for your reply. But broadcast state seems not supported in Flink-1.3 . I found this in Flink-1.3: Broadcasting DataStream → DataStream Broadcasts elements to every partition. dataStream.broadcast(); But I don’t know how to convert it to list and get it in stream context . 在 2018年6月21日,

How to use broadcast variables in data stream

2018-06-20 Thread zhen li
Hi,all: I want to use some other broadcast resources such as list or map in the flatmap function or customer triggers, but I don’t find some api to satisfy. Anyone can help? thanks