回复: array_sort function behaviour

2020-05-19 Thread Liu Genie
I will extract the element I want to sort, then combine it with the old struct as a new struct whose first element is what I want to sort. 发件人: neeraj bhadani 发送时间: 2020年5月19日 19:09 收件人: user 主题: array_sort function behaviour Hi All, I need to sort the array

Re: [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen

2020-02-26 Thread Liu Genie
n Mon, 24 Feb 2020 22:06:40 -0500 jianneng...@workday.com<mailto:jianneng...@workday.com> wrote Thanks Genie. Unfortunately, the joins I'm doing in this case are large, so UDF likely won't work. Jianneng From: Liu Genie mailto:genie_...@outlo

Re: [Spark SQL] Memory problems with packing too many joins into the same WholeStageCodegen

2020-02-24 Thread Liu Genie
I have encountered too many joins problem before. Since the joined dataframe is small enough, I convert join to udf operation, which is much faster and didn’t generate out of memory problem. 2020年2月25日 10:15,Jianneng Li mailto:jianneng...@workday.com>> 写道: Hello everyone, WholeStageCodegen ge