Re: question about Async IO

2022-11-04 Thread David Anderson
Yes, that will work as you expect. So long as you don't put another shuffle or rebalance in between, the keyed partitioning that's already in place will carry through the async i/o operator, and beyond. In most cases you can even use reinterpretAsKeyedStream on the output (so long as you haven't do

Re: question about Async IO

2022-11-02 Thread Galen Warren
Thanks, that makes sense and matches my understanding of how it works. In my case, I don't actually need access to keyed *state*; I just want to make sure that all elements with the same key are routed to the same instance of the async function. (Without going into too much detail, the reason for

Re: question about Async IO

2022-11-02 Thread Filip Karnicki
Hi Galen I was thinking about the same thing recently and reached a point where I see that async io does not have access to the keyed state because: "* State related apis in [[org.apache.flink.api.common.functions.RuntimeContext]] are not supported * yet because the key may get changed while acc

Re: question about Async IO

2022-10-14 Thread Krzysztof Chmielewski
Hi Galen, i will tell from my experience as a Flink user and developer of Flink jobs. *"if the input to an AsyncFunction is a keyed stream, can I assume that all input elements with the same key will be handled by the same instance of the async operator"* >From what I know (and someone can corre

question about Async IO

2022-10-14 Thread Galen Warren
I have a question about Flink's Async IO support: Async I/O | Apache Flink . I understand that access to state is not supported in an AsyncFunction. However, if the input to an AsyncFunction is a keyed st