Thanks for the clarification. I'll dig in then! On Mon, 16 Mar 2020, 3:47 pm Piotr Nowojski, <pi...@ververica.com> wrote:
> Hi, > > We are not maintaining internal docs. We have design docs for newly > proposed features (previously informal design docs published on dev mailing > list and recently as FLIP documents [1]), but keyed state is such an old > concept that dates back so much into the past, that I’m pretty sure it pre > dates any of that. So you would have to digg through the code if you want > to understand it. > > Piotrek > > [1] > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > > On 13 Mar 2020, at 16:14, Padarn Wilson <pad...@gmail.com> wrote: > > Thanks Piotr, > > Conceptually I understand (and use) the key'ed state quite a lot, but the > implementation details are what I was looking for. > > It looks like > `org.apache.flink.streaming.api.operators.AbstractStreamOperator#setKeyContextElement1` > is what I'm looking for though. It would be cool if there were some > internals design doc however? Quite hard to dig through the code as there > is a log tied to how the execution of the job actually happens. > > Padarn > > On Fri, Mar 13, 2020 at 9:43 PM Piotr Nowojski <pi...@ververica.com> > wrote: > >> Hi, >> >> Please take a look for example here: >> >> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#keyed-state >> And the example in particular >> >> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#using-managed-keyed-state >> >> The part about "there is a specific key implicitly in context” might be >> referring to the fact, that for every instance of `CountWindowAverage` that >> will be running in the cluster, user doesn’t have to set the key context >> explicility. Flink will set the the key context automatically for the >> `ValueState<Tuple2<Long, Long>> sum;` before any invocation of >> `CountWindowAverage#flatMap` method. >> >> In other words, one parallel instance of `CountWindowAverage` function, >> for two consecutive invocations of `CountWindowAverage#flatMap` can be >> referring to different underlying value of `CountWindowAverage#sum` field. >> For details you could take a look at >> `org.apache.flink.streaming.api.operators.AbstractStreamOperator#setKeyContextElement1` >> method and how it’s being used/implemented. >> >> I hope that helps. >> >> Piotrek >> >> On 13 Mar 2020, at 08:20, Padarn Wilson <pad...@gmail.com> wrote: >> >> Hi Users, >> >> I am trying to understand the details of how some aspects of Flink work. >> >> While understanding `keyed state` I kept coming up against a claim that >> `there >> is a specific key implicitly in context` I would like to understand how >> this works, which I'm guessing means understanding the details of the >> runtime context: Is there any documentation or FLIP someone can recommend >> on this? >> >> >> >
