Re: Streaming Application Design with Database Lookups

2016-07-26 Thread Guozhang Wang
Hello David, There is another remote store IO saving idea from a recent paper: https://scontent.xx.fbcdn.net/t39.2365-6/13331599_975087972607457_1796386216_n/Realtime_Data_Processing_at_Facebook.pdf "A monoid is an algebraic structure that has an identity element and is associative. When a monoi

Streaming Application Design with Database Lookups

2016-07-26 Thread David Garcia
Hello, we are working on designs for several streaming applications and a common consideration is the need for occasional external database updates/lookups. For example…we would be processing a stream of events with some kind of local-id, and we occasionally need to resolve the local-id to a g