Is there a way to accomplish this for the batch operations? On Thu, Jul 13, 2017 at 4:59 AM, Timo Walther <twal...@apache.org> wrote:
> Hi Mohit, > > do you plan to implement a batch or streaming job? If it is a streaming > job: You can use a connected stream (see [1], Slide 34). The static data is > one side of the stream that could be updated from time to time and will > always propagated (using a broadcast()) to all workers that do filtering, > augmentation etc. > > [1] http://training.data-artisans.com/dataStream/1-intro.html > > I hope this helps. > > Timo > > > Am 13.07.17 um 02:16 schrieb Mohit Anchlia: > > What is the best way to read a map of lookup data? This lookup data is >> like a small short lived data that is available in transformation to do >> things like filtering, additional augmentation of data etc. >> > > >
