thanks thats helpful
On Sun, Jan 29, 2017 at 12:54 PM, Anton Okolnychyi <
anton.okolnyc...@gmail.com> wrote:
> Hi,
>
> I recently extended the Spark SQL programming guide to cover user-defined
> aggregations, where I modified existing variables and returned them back in
> reduce and merge. This a
Hi,
I recently extended the Spark SQL programming guide to cover user-defined
aggregations, where I modified existing variables and returned them back in
reduce and merge. This approach worked and it was approved by people who
know the context.
Hope that helps.
2017-01-29 17:17 GMT+01:00 Koert K
anyone?
it not i will follow the trail and try to deduce it myself
On Mon, Jan 23, 2017 at 2:31 PM, Koert Kuipers wrote:
> looking at the docs for org.apache.spark.sql.expressions.Aggregator it
> says for reduce method: "For performance, the function may modify `b` and
> return it instead of con
looking at the docs for org.apache.spark.sql.expressions.Aggregator it says
for reduce method: "For performance, the function may modify `b` and return
it instead of constructing new object for b.".
it makes no such comment for the merge method.
this is surprising to me because i know that for
Pa
