The reason I ask is that I have a "Process Window Function" on that
Session Window and I keep key scoped Global State. I maintain a TTL on
that state ( that is outside the Window state ) that is roughly the
current WM + lateness.
I would imagine that keys for that custom state are *roughly* equal to the
number of keys in the "merging-window-set" . It seems twice that number but
does follow the slope. I am trying to figure out why this deviation.
public void process(KEY key,
ProcessWindowFunction<KeyedSession<KEY, VALUE>, KeyedSessionWithSessionID<
KEY, VALUE>, KEY, TimeWindow>.Context context,
Iterable<KeyedSession<KEY, VALUE>> elements, Collector<
KeyedSessionWithSessionID<KEY, VALUE>> out)
throws Exception {
// scoped to the key
if (state.value() == null) {
this.newKeysInState.inc();
state.update(new IntervalList());
}else{
this.existingKeysInState.inc();
}
On Sun, Mar 14, 2021 at 3:32 PM Vishal Santoshi <vishal.santo...@gmail.com>
wrote:
> Hey folks,
>
> Was looking at this very specific metric
> "session_aggregate.merging-window-set.rocksdb_estimate-num-keys". Does
> this metric also represent session windows ( it is a session window ) that
> have lateness on them ? In essence if the session window was closed but has
> a lateness of a few hours would those keys still be counted against this
> metric.
>
> I think they should as it is an estimate keys for the Column Family for
> the operator and if the window has not been GCed then the key for those
> Windows should be in RocksDB but wanted to be sure.
>
> Regards.
>
>
>
