The only way to do this today is to write it as a Hive UDAF. We hope to improve the window functions to use our native aggregation in a future release.
On Thu, Sep 10, 2015 at 12:26 AM, xander92 <[email protected]> wrote: > While testing out the new UserDefinedAggregateFunction in Spark 1.5.0, I > successfully implemented a simple function to compute an average. I then > tried to test this function by applying it over a simple window and I got > an > error saying that my function is not supported over window operation. > > So, is applying custom UDAFs over windows possible in Spark 1.5.0 and I > simply have a mistake somewhere?. If it is not possible, are there patches > that make this sort of thing possible that are simply not included in the > new release or is this functionality something that will hopefully come > soon > in a later release? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Custom-UDAF-Evaluated-Over-Window-tp24637.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
