Re: Tungsten's Vectorized Execution

2015-05-25 Thread Reynold Xin
Yes that's exactly the reason. On Sat, May 23, 2015 at 12:37 AM, Yijie Shen wrote: > Davies and Reynold, > > Glad to hear about the status. > > I’ve seen [SPARK-7813](https://issues.apache.org/jira/browse/SPARK-7813) > and watching it now. > > If I understand correctly, it’s aimed at moving Cod

Re: Tungsten's Vectorized Execution

2015-05-23 Thread Yijie Shen
Davies and Reynold, Glad to hear about the status. I’ve seen [SPARK-7813](https://issues.apache.org/jira/browse/SPARK-7813) and watching it now. If I understand correctly, it’s aimed at moving CodeGenerator’s expressionEvaluator’s code-gen logic into each expressions’ eval() and eliminating t

Re: Tungsten's Vectorized Execution

2015-05-22 Thread Reynold Xin
Yijie, As Davies said, it will take us a while to get to vectorized execution. However, before that, we are going to refactor code generation to push it into each expression: https://issues.apache.org/jira/browse/SPARK-7813 Once this one is in (probably in the next 2 or 3 weeks), there will be lo

Re: Tungsten's Vectorized Execution

2015-05-21 Thread Davies Liu
We have not start to prototype the vectorized one yet, will evaluated in 1.5 and may targeted for 1.6. We're glad to hear some feedback/suggestions/comments from your side! On Thu, May 21, 2015 at 9:37 AM, Yijie Shen wrote: > Hi all, > > I’ve seen the Blog of Project Tungsten here, it sounds awe

Tungsten's Vectorized Execution

2015-05-21 Thread Yijie Shen
Hi all, I’ve seen the Blog of Project Tungsten here, it sounds awesome to me! I’ve also noticed there is a plan to change the code generation from record-at-a-time evaluation to a vectorized one, which interests me most. What’s the status of vectorized evaluation?  Is this an inner effort of D