Re: ValueState is null; checkpointing issues

2021-07-13 Thread Chesnay Schepler
When your operator process a value with the key, then the ValueState is implicitly scoped to that key. So, If all your payloads have a unique ID (and hence unique key) then the ValueState will initially always be null. Only if 2 payloads have the same ID will be ValueState return something non-n

Re: ValueState is null; checkpointing issues

2021-07-12 Thread Marzi K
Hi, > 1) The ValueState can only return a non-null value if a prior value with the > same key (in your case, "x.id") has been received. Have you double-checked > that this is the case? I use the payloads in each operator to make post requests. So the payloads are not null and I keyBy using the

Re: ValueState is null; checkpointing issues

2021-07-12 Thread Chesnay Schepler
1) The ValueState can only return a non-null value if a prior value with the same key (in your case, "x.id") has been received. Have you double-checked that this is the case? 2) Checkpointing does not alleviate the need to restart all operators, it alleviates having to reprocess all data. It i