Re: Advice or best practices on adding metadata to stream events

2018-01-31 Thread Hung
So there are three ways. 1. make your model as stream source 2. let master read the model once, distribute it via constructor, and update it periodically 3. let worker read the model and update it periodically(you mentioned) option 3 would be problematic if you scale a lot and use many parallelis

Advice or best practices on adding metadata to stream events

2018-01-29 Thread Hauke Hans
Hi everyone, I am fairly new to the world of stream processing and I was wondering about best practices when needing to add metadata to a stream in Flink (or stream processing in general). Searching for examples/discussions of this topic did not yield the results I was hoping for, so I figured