Re: [Statefun] Dynamic behavior

2021-02-24 Thread Miguel Araújo
Thanks Seth. I understood Igal's suggestion. My concern was about maintaining a separate service (outside flink/statefun) when this control stream might be an incremental stream as well (think, rules in fraud detection - although this is not a fraud detection application, but the example is good).

Re: [Statefun] Dynamic behavior

2021-02-23 Thread Seth Wiesman
I don't think there is anything statefun specific here and I would follow Igals advice. Let's say you have a state value called `Behavior` that describes the behavior of an instance. There is a default behavior but any given instance may have a customized behavior. What I would do is the following

Re: [Statefun] Dynamic behavior

2021-02-23 Thread Miguel Araújo
Hi Seth, Thanks for your comment. I've seen that repository in the past and it was really helpful to "validate" that this was the way to go. I think my question is not being addressed there though: how could one add dynamic behavior to your TransactionManager? In this case, state that is available

Re: [Statefun] Dynamic behavior

2021-02-23 Thread Seth Wiesman
Hey Miguel, What you are describing is exactly what is implemented in this repo. The TransactionManager function acts as an orchestrator to work with the other functions. The repo is structured as an exercise but the full solution exists on the branch `advanced-solution`. https://github.com/verve

Re: [Statefun] Dynamic behavior

2021-02-23 Thread Miguel Araújo
Another possibility I am considering is handling this in Flink using a broadcast and adding all the information needed to the event itself. I'm a little concerned about the amount of data that will be serialized and sent on every request though, as I'll need to include information about all availab

Re: [Statefun] Dynamic behavior

2021-02-23 Thread Miguel Araújo
Hi Gordon, Igal, Thanks for your replies. PubSub would be a good addition, I have a few scenarios where that would be useful. However, after reading your answers I realized that your proposed solutions (which address the most obvious interpretation of my question) do not necessarily solve my prob

Re: [Statefun] Dynamic behavior

2021-02-22 Thread Igal Shilman
Hi Miguel, I think that there are a couple of ways to achieve this, and it really depends on your specific use case, and the trade-offs that you are willing to accept. For example, one way to approach this: - Suppose you have an external service somewhere that returns a representation of the logi

Re: [Statefun] Dynamic behavior

2021-02-21 Thread Tzu-Li (Gordon) Tai
Hi, FWIW, there is this JIRA that is tracking a pubsub / broadcast messaging primitive in StateFun: https://issues.apache.org/jira/browse/FLINK-16319 This is probably what you are looking for. And I do agree, in the case that the control stream (which updates the application logic) is high volume

[Statefun] Dynamic behavior

2021-02-21 Thread Miguel Araújo
Hi everyone, What is the recommended way of achieving the equivalent of a broadcast in Flink when using Stateful Functions? For instance, assume we are implementing something similar to Flink's demo fraud detection but in Stat