Re: Merging multiple streams into one

2020-05-12 Thread Alessandro Tagliapietra
Hi Bill, thank you for replying. Yes keys are all the same type (machine ID string) Btw, your solution sounds great, but it'll only work if al the 3 streams have the same number of partitions, right? Otherwise there's no guarantee that all the data of the same machine (the topic keys are the ma

Re: Merging multiple streams into one

2020-05-12 Thread Bill Bejeck
Hi Alessandro, For merging the three streams, have you considered the `KStream.merge` method? If the values are of different types, you'll need to map them into a common type first, but I think something like this will work: KStream mappedOne = orignalStreamOne.mapValues(...); KStream mappedTwo =

Merging multiple streams into one

2020-05-11 Thread Alessandro Tagliapietra
Hello everyone, we currently use 3 streams (metrics, events, states) and I need to implement a keepalive mechanism so that if the machine doesn't send any data (from a specific list of variables) it'll emit a value that changes the machine state. For example, in machine 1 the list of keepalive va