Q1: Thank you, the branch() is what I'm looking for, I just missed it somehow.
Q2: I receive something like "imageid,x,y" as key, and a color as value. I aggregate this to something like average color for example. So technically I do not have images, I have colored pixels with 3 dimensions one being the image... And then my newcomer user wants to join the fun, so we'd need to serve him an image with the latest current state (all pixels of imageid=x), and of course everything that comes later (updates on the output of the aggregate, but filtered to that imageid). Problem is that how to get the stream api node to "export" part of it's current state on demand. (All imageid=x keys with values). There could be a "request topic" that I could join together with the aggregated ktable maybe? Other problem is following the updates without getting the details of other images (99% of records are not interesting for the specific user). Thanks, Peter On Thu, Jan 19, 2017 at 4:55 PM, Eno Thereska <eno.there...@gmail.com> wrote: > Hi Peter, > > About Q1: The DSL has the "branch" API, where one stream is branched to > several streams, based on a predicate. I think that could help. > > About Q2: I'm not entirely sure I understand the problem space. What is > the definition of a "full image"? > > Thanks > Eno > > On 19 Jan 2017, at 12:07, Peter Kopias <kopias.pe...@gmail.com> wrote: > > > > Greetings Everyone, > > > > I'm just getting into the kafka world with a sample project, and I've got > > two conceptional issues, you might have a trivial answer already at hand > to. > > > > Scenario: multiuser painting webapp, with N user working on M images > > simultaneously. The "brush" events go to one single kafka topic, in a > > format: imageid,x,y -> brushevent , that I aggregate to imageid,x,y > > > > Q1: > > It would be nice to separate the stream to M output topics, so that would > > work nice as "partitioning", and also we could just subscribe to update > > events of a specific image maybe. How can I fan out the records to > > different (maybe not yet existing) topics by using DSL? > > > > Is that a good idea? (If I can solve every processing in a common > > processing graph that would be the best, but then I'd need some high > > performance solution of filtering out the noise, as the subscribers are > > only interested in a very small subset of the soup.) > > > > Q2: > > - When a new user comes, I'd like give him the latest full image? > > (I could do a "fullimages" output topic, but then also comes the problem > > of serious overhead on each incoming update, and also the newcomer should > > somehow only get the image he's interested in, not read all the images, > and > > ignore the others.) > > > > I know I'm still new to this, but I'd like to learn the best practices > you > > might already tried. > > > > Thank you, > > > > Peter > >
