Re: Need help with HashAggregateExec, TungstenAggregationIterator and UnsafeFixedWidthAggregationMap

2018-09-08 Thread Jacek Laskowski
Hi Herman, Right. No @deprecated, but something that would tell people who review the code "be extra careful since you're reading code that is no longer in use" for SparkPlans that do support WSCG. That would help a lot as I got tricked few times already while trying to understand something that I

Re: [DISCUSS] PySpark Window UDF

2018-09-08 Thread Wes McKinney
hi Li, These results are very cool. I'm excited to see you continuing to push this effort forward. - Wes On Wed, Sep 5, 2018 at 5:52 PM Li Jin wrote: > > Hello again! > > I recently implemented a proof-of-concept implementation of proposal above. I > think the results are pretty exciting so I w

Re: Need help with HashAggregateExec, TungstenAggregationIterator and UnsafeFixedWidthAggregationMap

2018-09-08 Thread Jacek Laskowski
Thanks Russ! That helps a lot. On the other hand makes reviewing the codebase of Spark SQL slightly harder since Java code generation is so much about string concatenation :( p.s. Should all the code in doExecute be considered and marked @deprecated? Pozdrawiam, Jacek Laskowski https://abou