Hi Giannis, I'm curious, what tool did you use for this analysis (what the screenshot shows)? Is it something custom?
Thank you. On Wed, Apr 26, 2023 at 10:38 PM Giannis Polyzos <ipolyzos...@gmail.com> wrote: > This is really helpful, > > Thanks > > On Thu, Apr 27, 2023 at 5:46 AM Yanfei Lei <fredia...@gmail.com> wrote: > >> Hi Giannis, >> >> Except “default” Colume Family(CF), all other CFs represent the state >> in rocksdb state backend, the name of a CF is the name of a >> StateDescriptor. >> >> - deduplicate-state is a value state, you can find it in >> DeduplicateFunctionBase.java and >> MiniBatchDeduplicateFunctionBase.java, they are used for >> deduplication. >> - _timer_state/event_user-timers, _timer_state/event_timers , >> _timer_state/processing_timers and _timer_state/processing_user-timers >> are created by internal time service, which can be found in >> InternalTimeServiceManagerImpl.java. Here is a blog post[1] on best >> practices for using timers. >> - timer, next-index, left and right can be found in >> TemporalRowTimeJoinOperator.java, TemporalRowTimeJoinOperator >> implements the logic of temporal join, this post[2] might be helpful >> in understanding what happened to temporal join. >> >> [1] >> https://www.alibabacloud.com/help/en/realtime-compute-for-apache-flink/latest/datastream-timer >> [2] >> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/#temporal-joins >> >> Giannis Polyzos <ipolyzos...@gmail.com> 于2023年4月26日周三 23:19写道： >> > >> > I have two input kafka topics - a compacted one (with upsert-kafka) and >> a normal one. >> > When I perform a temporal join I notice the following state being >> created in rocksdb and was hoping someone could help me better understand >> what everything means >> > >> > >> > > deduplicate-state: does it refer to duplicate keys found by the >> kafka-upsert-connector? >> > > timers: what timer and _timer_state/event_timers refer to and whats >> their difference? Is it to keep track on when the join results need to be >> materialised or state to be expired? >> > > next-index: what does it refer to? >> > > left: also I'm curious why the left cf has 407 entries. Is it records >> that are being buffered because there is no match on the right table? >> > >> > Thanks >> >> >> >> -- >> Best, >> Yanfei >> >
