Greetings Everyone, I'm just getting into the kafka world with a sample project, and I've got two conceptional issues, you might have a trivial answer already at hand to.
Scenario: multiuser painting webapp, with N user working on M images simultaneously. The "brush" events go to one single kafka topic, in a format: imageid,x,y -> brushevent , that I aggregate to imageid,x,y Q1: It would be nice to separate the stream to M output topics, so that would work nice as "partitioning", and also we could just subscribe to update events of a specific image maybe. How can I fan out the records to different (maybe not yet existing) topics by using DSL? Is that a good idea? (If I can solve every processing in a common processing graph that would be the best, but then I'd need some high performance solution of filtering out the noise, as the subscribers are only interested in a very small subset of the soup.) Q2: - When a new user comes, I'd like give him the latest full image? (I could do a "fullimages" output topic, but then also comes the problem of serious overhead on each incoming update, and also the newcomer should somehow only get the image he's interested in, not read all the images, and ignore the others.) I know I'm still new to this, but I'd like to learn the best practices you might already tried. Thank you, Peter
