@Wenchen Fan, understood that the mapping of query plan to application code is very hard. I was wondering if we might be able to instead just handle the mapping from the final physical plan to the stage graph. So for example you’d be able to tell what part of the plan generated which stages. I feel this would provide the most benefit without having to worry about several optimisation steps.
The main issue as I see it is that currently, if there’s a failing stage, it’s almost impossible to track down the part of the plan that generated the stage. Would this be possible? If not, do you have any other suggestions for this kind of debugging? Best, Matt -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
