Maybe the usage of that function change, now I have to use it as this [1]

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/process_function.html#the-keyedprocessfunction

El lun., 20 ago. 2018 a las 5:56, vino yang (<yanghua1...@gmail.com>)
escribió:

> Hi antonio,
>
> Oh, if you can't use KeyedProcessFunction, then this would be a pity.
> Then you can use MapState, where Key is used to store the key of your
> partition.
> But I am not sure if this will achieve the effect you want.
>
> Thanks, vino.
>
> antonio saldivar <ansal...@gmail.com> 于2018年8月20日周一 下午4:32写道:
>
>> Hello
>>
>> Thank you for the information, for some reason this KeyedProcessFunction
>> is not found in my Flink version 1.4.2 I can only find ProcessFunction and
>> work like this
>>
>> public class TxnProcessFn extends ProcessFunction<Object,Object> {
>>
>>  public void open(Configuration parameters) throws Exception {
>>
>> state1 = getRuntimeContext().getState(new ValueStateDescriptor<>(
>> "objState1", Object.class));
>>
>> state2 = getRuntimeContext().getState(new ValueStateDescriptor<>(
>> "objState2", Object.class));
>>
>> state3 = getRuntimeContext().getState(new ValueStateDescriptor<>(
>> "objState3", Object.class));
>>
>> }
>>
>> @Override
>>
>> public void processElement(
>>
>> Object obj,
>>
>> Context ctx,
>>
>> Collector<Transaction> out) throws Exception {
>>
>> // TODO Auto-generated method stub
>>
>> Object current = state.value();
>>
>> if (current == null) {
>>
>>             current = new Object();
>>
>>             current.id=obj.id();
>>
>>
>>
>>         }
>>
>> }
>>
>> El lun., 20 ago. 2018 a las 2:24, vino yang (<yanghua1...@gmail.com>)
>> escribió:
>>
>>> Hi antonio,
>>>
>>> First, I suggest you use KeyedProcessFunction if you have an operation
>>> similar to keyBy.
>>> The implementation is similar to the Fixed window.
>>> You can create three state collections to determine whether the time of
>>> each element belongs to a state collection.
>>> At the time of the trigger, the elements in the collection are evaluated.
>>>
>>> Thanks, vino.
>>>
>>> antonio saldivar <ansal...@gmail.com> 于2018年8月20日周一 上午11:54写道:
>>>
>>>> Thank you fro the references
>>>>
>>>> I have now my processFunction and getting the state but now how can i
>>>> do for the threshold times to group the elements and also as this is a
>>>> global window, how to purge because if going to keep increasing
>>>>
>>>> El dom., 19 ago. 2018 a las 8:57, vino yang (<yanghua1...@gmail.com>)
>>>> escribió:
>>>>
>>>>> Hi antonio,
>>>>>
>>>>> Regarding your scenario, I think maybe you can consider using the
>>>>> ProcessFunction (or keyed ProcessFunction) function directly on the 
>>>>> Stream.
>>>>> [1]
>>>>> It can handle each of your elements with a Timer, and you can combine
>>>>> Flink's state API[2] to store your data.
>>>>>
>>>>> [1]:
>>>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/process_function.html#process-function-low-level-operations
>>>>> [2]:
>>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/state/state.html#working-with-state
>>>>>
>>>>> Thanks, vino.
>>>>>
>>>>> antonio saldivar <ansal...@gmail.com> 于2018年8月19日周日 上午10:18写道:
>>>>>
>>>>>> hi Vino
>>>>>>
>>>>>> it is possible to use global window, then set the trigger onElement
>>>>>> comparing the element that has arrived with for example 10 mins, 20 mins
>>>>>> and 60 mins of data?
>>>>>>
>>>>>> I have rules evaluating sum of amount for 10,20 or 60 mins for the
>>>>>> same keyed element if the same id sum like $200 total within those
>>>>>> thresholds and count more or equals to 3 I need to be able to set some
>>>>>> values to the object if the object does not reach those thresholds i do 
>>>>>> not
>>>>>> set the values and keep sending the output with or without those value.
>>>>>>
>>>>>> just processing the object on the fly and send output
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> El vie., 17 ago. 2018 a las 22:14, vino yang (<yanghua1...@gmail.com>)
>>>>>> escribió:
>>>>>>
>>>>>>> Hi antonio,
>>>>>>>
>>>>>>> Yes, ProcessWindowFunction is a very low level window function.
>>>>>>> It allows you to access the data in the window and allows you to
>>>>>>> customize the output of the window.
>>>>>>> So if you use it, while giving you flexibility, you need to think
>>>>>>> about other things, which may require you to write more processing 
>>>>>>> logic.
>>>>>>>
>>>>>>> Generally speaking, sliding windows usually have some data that is
>>>>>>> repeated, but a common mode is to apply a reduce function on it to get 
>>>>>>> your
>>>>>>> calculation results.
>>>>>>> If you only send data, there will definitely be some duplication.
>>>>>>>
>>>>>>> Thanks, vino.
>>>>>>>
>>>>>>> antonio saldivar <ansal...@gmail.com> 于2018年8月17日周五 下午12:01写道:
>>>>>>>
>>>>>>>> Hi Vino
>>>>>>>> thank you for the information, actually I am using a trigger alert
>>>>>>>> and processWindowFunction to send my results, but when my window 
>>>>>>>> slides or
>>>>>>>> ends it sends again the objects and I an getting duplicated data
>>>>>>>>
>>>>>>>> El jue., 16 ago. 2018 a las 22:05, vino yang (<
>>>>>>>> yanghua1...@gmail.com>) escribió:
>>>>>>>>
>>>>>>>>> Hi Antonio,
>>>>>>>>>
>>>>>>>>> What results do not you want to get when creating each window?
>>>>>>>>> Examples of the use of ProcessWindowFunction are included in many
>>>>>>>>> test files in Flink's project, such as SideOutputITCase.scala or
>>>>>>>>> WindowTranslationTest.scala.
>>>>>>>>>
>>>>>>>>> For more information on ProcessWindowFunction, you can refer to
>>>>>>>>> the official website.[1]
>>>>>>>>>
>>>>>>>>> [1]:
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/windows.html#processwindowfunction
>>>>>>>>>
>>>>>>>>> Thanks, vino.
>>>>>>>>>
>>>>>>>>> antonio saldivar <ansal...@gmail.com> 于2018年8月17日周五 上午6:24写道:
>>>>>>>>>
>>>>>>>>>> Hello
>>>>>>>>>>
>>>>>>>>>> I am implementing a data stream where I use sliding windows but I
>>>>>>>>>> am stuck because I need to set values to my object based on some if
>>>>>>>>>> statements in my process function  and send the object to the next 
>>>>>>>>>> step but
>>>>>>>>>> I don't want results every time a window is creating
>>>>>>>>>>
>>>>>>>>>> if anyone has a good example on this that can help me
>>>>>>>>>>
>>>>>>>>>

Reply via email to