Re: [SQL] Unresolved reference with chained window functions.

This is definitely a bug in the CollapseWindow optimizer rule. I think we can use SPARK-20086 to track this. On Fri, Mar 24, 2017 at 9:28 PM, Maciej Szymkiewicz wrote: > Forwarded from SO (http://stackoverflow.com/q/43007433). Looks like > regr

[SQL] Unresolved reference with chained window functions.

Forwarded from SO (http://stackoverflow.com/q/43007433). Looks like regression compared to 2.0.2. scala> import org.apache.spark.sql.expressions.Window import org.apache.spark.sql.expressions.Window scala> val win_spec_max = Window.partitionBy("x").orderBy("AmtPaid").rowsBetween(Window.unboundedP

Re: RFC: deprecate SparkStatusTracker, remove JobProgressListener

On Fri, Mar 24, 2017 at 1:18 PM, Ryan Blue wrote: > For the status tracker, I'd like to see it replaced with something better > before deprecating it. I've been looking at it for implementing better > feedback in notebooks, and it looks sufficient for that at the moment. Is > there a reason to rem

Re: RFC: deprecate SparkStatusTracker, remove JobProgressListener

+1 for deprecating JobProgressListener and removing it eventually. Thanks for working toward a better back-end for the UI. For the status tracker, I'd like to see it replaced with something better before deprecating it. I've been looking at it for implementing better feedback in notebooks, and it

Re: RFC: deprecate SparkStatusTracker, remove JobProgressListener

On Fri, Mar 24, 2017 at 1:13 PM, Josh Rosen wrote: > Let's not deprecate SparkStatusTracker (at least not until there's a > suitable replacement that's just as easy to use). Deprecating it now would > leave users with an unactionable warning and that's not great for users who > try to keep their b

Re: RFC: deprecate SparkStatusTracker, remove JobProgressListener

Let's not deprecate SparkStatusTracker (at least not until there's a suitable replacement that's just as easy to use). Deprecating it now would leave users with an unactionable warning and that's not great for users who try to keep their build warnings clean. If you want to be really proactive in

Re: RFC: deprecate SparkStatusTracker, remove JobProgressListener

On Fri, Mar 24, 2017 at 12:07 PM, Josh Rosen wrote: > I think that it should be safe to remove JobProgressListener but I'd like to > keep the SparkStatusTracker API. Thanks Josh. I can work with that. My main concern would be keeping the listener around. Is it worth it to add a deprecated annota

Re: RFC: deprecate SparkStatusTracker, remove JobProgressListener

I think that it should be safe to remove JobProgressListener but I'd like to keep the SparkStatusTracker API. SparkStatusTracker was originally developed to provide a stable programmatic status API for use by Hive on Spark. SparkStatusTracker predated the Spark REST APIs for status tracking which

Re: planning & discussion for larger scheduler changes

On Fri, Mar 24, 2017 at 4:41 PM, Imran Rashid wrote: > Kay and I were discussing some of the bigger scheduler changes getting > proposed lately, and realized there is a broader discussion to have with > the community, outside of any single jira. I'll start by sharing my > initial thoughts, I kn

planning & discussion for larger scheduler changes

Kay and I were discussing some of the bigger scheduler changes getting proposed lately, and realized there is a broader discussion to have with the community, outside of any single jira. I'll start by sharing my initial thoughts, I know Kay has thoughts on this too, but it would be good to input

Re: Fwd: [SparkSQL] Project using NamedExpression

Hi , Can you please point me on how to resolve the expression ? I was looking into LogicalPlan.Resolve expression() that takes a Partial Function but I am not sure how to use that. Thanks, Aviral Agarwal On Mar 24, 2017 09:20, "Liang-Chi Hsieh" wrote: Hi, You need to resolve the expressions b