Re: Re: Multiple streaming aggregations in structured streaming

2016-11-20 Thread Reynold Xin
Can you use the approximate count distinct? On Sun, Nov 20, 2016 at 11:51 PM, Xinyu Zhang wrote: > > MapWithState is also very useful. > I want to calculate UV in real time, but "distinct count" and "multiple > streaming aggregations" are not supported. > Is there any method to calculate real-t

Re:Re: Multiple streaming aggregations in structured streaming

2016-11-20 Thread Xinyu Zhang
MapWithState is also very useful. I want to calculate UV in real time, but "distinct count" and "multiple streaming aggregations" are not supported. Is there any method to calculate real-time UV in the current version? At 2016-11-19 06:01:45, "Michael Armbrust" wrote: Doing this generally

Re: Multiple streaming aggregations in structured streaming

2016-11-18 Thread Michael Armbrust
Doing this generally is pretty hard. We will likely support algebraic aggregate eventually, but this is not currently slotted for 2.2. Instead I think we will add something like mapWithState that lets users compute arbitrary stateful things. What is your use case? On Wed, Nov 16, 2016 at 6:58

Multiple streaming aggregations in structured streaming

2016-11-16 Thread wszxyh
Hi Multiple streaming aggregations are not yet supported. When will it be supported? Is it in the plan? Thanks