Hello
Thank you for the information, for some reason this KeyedProcessFunction is
not found in my Flink version 1.4.2 I can only find ProcessFunction and
work like this
public class TxnProcessFn extends ProcessFunction<Object,Object> {
public void open(Configuration parameters) throws Exception {
state1 = getRuntimeContext().getState(new ValueStateDescriptor<>("objState1",
Object.class));
state2 = getRuntimeContext().getState(new ValueStateDescriptor<>("objState2",
Object.class));
state3 = getRuntimeContext().getState(new ValueStateDescriptor<>("objState3",
Object.class));
}
@Override
public void processElement(
Object obj,
Context ctx,
Collector<Transaction> out) throws Exception {
// TODO Auto-generated method stub
Object current = state.value();
if (current == null) {
current = new Object();
current.id=obj.id();
}
}
El lun., 20 ago. 2018 a las 2:24, vino yang (<yanghua1...@gmail.com>)
escribió:
> Hi antonio,
>
> First, I suggest you use KeyedProcessFunction if you have an operation
> similar to keyBy.
> The implementation is similar to the Fixed window.
> You can create three state collections to determine whether the time of
> each element belongs to a state collection.
> At the time of the trigger, the elements in the collection are evaluated.
>
> Thanks, vino.
>
> antonio saldivar <ansal...@gmail.com> 于2018年8月20日周一 上午11:54写道：
>
>> Thank you fro the references
>>
>> I have now my processFunction and getting the state but now how can i do
>> for the threshold times to group the elements and also as this is a global
>> window, how to purge because if going to keep increasing
>>
>> El dom., 19 ago. 2018 a las 8:57, vino yang (<yanghua1...@gmail.com>)
>> escribió:
>>
>>> Hi antonio,
>>>
>>> Regarding your scenario, I think maybe you can consider using the
>>> ProcessFunction (or keyed ProcessFunction) function directly on the Stream.
>>> [1]
>>> It can handle each of your elements with a Timer, and you can combine
>>> Flink's state API[2] to store your data.
>>>
>>> [1]:
>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/process_function.html#process-function-low-level-operations
>>> [2]:
>>> https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/state/state.html#working-with-state
>>>
>>> Thanks, vino.
>>>
>>> antonio saldivar <ansal...@gmail.com> 于2018年8月19日周日 上午10:18写道：
>>>
>>>> hi Vino
>>>>
>>>> it is possible to use global window, then set the trigger onElement
>>>> comparing the element that has arrived with for example 10 mins, 20 mins
>>>> and 60 mins of data?
>>>>
>>>> I have rules evaluating sum of amount for 10,20 or 60 mins for the same
>>>> keyed element if the same id sum like $200 total within those thresholds
>>>> and count more or equals to 3 I need to be able to set some values to the
>>>> object if the object does not reach those thresholds i do not set the
>>>> values and keep sending the output with or without those value.
>>>>
>>>> just processing the object on the fly and send output
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> El vie., 17 ago. 2018 a las 22:14, vino yang (<yanghua1...@gmail.com>)
>>>> escribió:
>>>>
>>>>> Hi antonio,
>>>>>
>>>>> Yes, ProcessWindowFunction is a very low level window function.
>>>>> It allows you to access the data in the window and allows you to
>>>>> customize the output of the window.
>>>>> So if you use it, while giving you flexibility, you need to think
>>>>> about other things, which may require you to write more processing logic.
>>>>>
>>>>> Generally speaking, sliding windows usually have some data that is
>>>>> repeated, but a common mode is to apply a reduce function on it to get
>>>>> your
>>>>> calculation results.
>>>>> If you only send data, there will definitely be some duplication.
>>>>>
>>>>> Thanks, vino.
>>>>>
>>>>> antonio saldivar <ansal...@gmail.com> 于2018年8月17日周五 下午12:01写道：
>>>>>
>>>>>> Hi Vino
>>>>>> thank you for the information, actually I am using a trigger alert
>>>>>> and processWindowFunction to send my results, but when my window slides
>>>>>> or
>>>>>> ends it sends again the objects and I an getting duplicated data
>>>>>>
>>>>>> El jue., 16 ago. 2018 a las 22:05, vino yang (<yanghua1...@gmail.com>)
>>>>>> escribió:
>>>>>>
>>>>>>> Hi Antonio,
>>>>>>>
>>>>>>> What results do not you want to get when creating each window?
>>>>>>> Examples of the use of ProcessWindowFunction are included in many
>>>>>>> test files in Flink's project, such as SideOutputITCase.scala or
>>>>>>> WindowTranslationTest.scala.
>>>>>>>
>>>>>>> For more information on ProcessWindowFunction, you can refer to the
>>>>>>> official website.[1]
>>>>>>>
>>>>>>> [1]:
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/windows.html#processwindowfunction
>>>>>>>
>>>>>>> Thanks, vino.
>>>>>>>
>>>>>>> antonio saldivar <ansal...@gmail.com> 于2018年8月17日周五 上午6:24写道：
>>>>>>>
>>>>>>>> Hello
>>>>>>>>
>>>>>>>> I am implementing a data stream where I use sliding windows but I
>>>>>>>> am stuck because I need to set values to my object based on some if
>>>>>>>> statements in my process function and send the object to the next
>>>>>>>> step but
>>>>>>>> I don't want results every time a window is creating
>>>>>>>>
>>>>>>>> if anyone has a good example on this that can help me
>>>>>>>>
>>>>>>>
