Re: Flink redshift table lookup and updates

2016-08-19 Thread Robert Metzger
Hi Harshith, Welcome to the Flink community ;) I would recommend using approach 2. Keeping the state in Flink and just sending updates to the dashboard store should give you better performance and consistency. I don't know whether its better to download the full state snapshot from redshift in th

Flink redshift table lookup and updates

2016-08-18 Thread Harshith Chennamaneni
Hi, I've very recently come upon flink and I'm trying to use it to solve a problem that I have. I have a stream of User Settings updates coming through kafka queue. I need to store the most recent settings along with a history of settings for each user in redshift which then feeds into analytics