Hi Vasily, Currently, Flink did not do the coordination between a general stream and broadcast stream, they are both streams. Your scene of using the broadcast state is a special one. In a more general scene, the states need to be broadcasted is an unbounded stream, the state events may be broadcasted to the downstream at any time. So it can not be wait to be done before playing the usual stream events.
For your scene: - you can change your storage about dimension table, e.g. Redis or MySQL and so on to do the stream and dimension table join; - you can inject some control event in your broadcast stream to mark the stream is end and let the fact stream wait until receiving the control event. Or you can introduce a thrid-party coordinator e.g. ZooKeeper to coordinate them, however, it would make your solution more complex. Best, Vino Vasily Melnik <vasily.mel...@glowbyteconsulting.com> 于2019年11月14日周四 下午1:28写道： > Hi all. > > In our task we have two Kafka topics: > - one with fact stream (web traffic) > - one with dimension > > We would like to put dimension data into broadcast state and lookup on int > with facts. But we see that not all dimension records are put into state > before first fact record is processed, so lookup gives no data. > > The question is: how could we read fact topic with some "delay" to give > dimension enough time to initialize state? > > > С уважением, > Василий Мельник >
