Hi Peter, About Q1: The DSL has the "branch" API, where one stream is branched to several streams, based on a predicate. I think that could help.
About Q2: I'm not entirely sure I understand the problem space. What is the definition of a "full image"? Thanks Eno > On 19 Jan 2017, at 12:07, Peter Kopias <kopias.pe...@gmail.com> wrote: > > Greetings Everyone, > > I'm just getting into the kafka world with a sample project, and I've got > two conceptional issues, you might have a trivial answer already at hand to. > > Scenario: multiuser painting webapp, with N user working on M images > simultaneously. The "brush" events go to one single kafka topic, in a > format: imageid,x,y -> brushevent , that I aggregate to imageid,x,y > > Q1: > It would be nice to separate the stream to M output topics, so that would > work nice as "partitioning", and also we could just subscribe to update > events of a specific image maybe. How can I fan out the records to > different (maybe not yet existing) topics by using DSL? > > Is that a good idea? (If I can solve every processing in a common > processing graph that would be the best, but then I'd need some high > performance solution of filtering out the noise, as the subscribers are > only interested in a very small subset of the soup.) > > Q2: > - When a new user comes, I'd like give him the latest full image? > (I could do a "fullimages" output topic, but then also comes the problem > of serious overhead on each incoming update, and also the newcomer should > somehow only get the image he's interested in, not read all the images, and > ignore the others.) > > I know I'm still new to this, but I'd like to learn the best practices you > might already tried. > > Thank you, > > Peter
