Re: KStreams Proposal

2016-03-14 Thread Guozhang Wang
Thanks Bill for the detailed description of your use case. I think part of the issue is that today KStream does not allow non-keyed aggregation, and hence users need to call a `map` function if they want to do aggregations on other fields. We did this by design for KStream since if users write the

Re: KStreams Proposal

2016-03-13 Thread Bill Bejeck
Hi Guozhang, Possibly, but the use case I'm working with is having a of collector object , for aggregate statistics for example, that would output results intermittently (via punctuate). The issue for me is that 'transform(..)' returns a key-value pair for each message, possibly of a different

Re: KStreams Proposal

2016-03-13 Thread Guozhang Wang
Hello Bill, We added transform() together with process() to support any user-customized stateful processor that can still concatenate to another KStream. So for your case, would `transform(...).to(topic)` provide the same functionality as "processTo(topic, ...)"? Guozhang On Sat, Mar 12, 2016