Hi Chengzhi, Broadcast State is checkpointed like any other state and will be restored in all failure cases (including the ones you mentioned). We added the warning to inform users that Broadcast state will also be stored in the JVM memory, even if the RocksDB StateBackend was configured (which stores state on disk). This warning is only about the size of the state, not about the consistency guarantees.
Best, Fabian Am Mo., 22. Okt. 2018 um 19:26 Uhr schrieb Chengzhi Zhao < w.zhaocheng...@gmail.com>: > Hey folks, > > We are trying to use the broadcast state as "Shared Rule" state to filter > test data in our stream pipeline, the broadcast will be connected with > other streams in the pipeline. > I noticed on broadcast_state[1] important consideration page, it is > mentioned *No RocksDB state backend* and state would be kept in in-memory > at runtime. > > I am trying to figure out how it works, for example, > 1. If a node goes down, will broadcast state lost the entire state for > that node and then sync from other nodes? > 2. In case of the entire job fail or savepoint been triggered, how > broadcast state get its state back or additional bootstrapping logic needs > to be added ourselves? > > Thanks for your help! > > Best regards, > Chengzhi > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html#important-considerations >
