GitHub user JDrit opened a pull request:
https://github.com/apache/spark/pull/7478
[SPARK-8007][SQL] Added virtual column support to Spark
Added virtual column support by adding a new resolution role to the query
analyzer. Additional virtual columns can be added by adding case expressions to
[the new
rule](https://github.com/JDrit/spark/blob/virt_columns/sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala#L1026)
and my modifying the [logical
plan](https://github.com/JDrit/spark/blob/virt_columns/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala#L216)
to resolve them.
This also solves
[SPARK-8003](https://issues.apache.org/jira/browse/SPARK-8003)
This allows you to perform queries such as:
```sql
select spark__partition__id, count(*) as c from table group by
spark__partition__id;
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JDrit/spark virt_columns
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7478.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7478
----
commit 7b46e7de6f98df98480fa34c85248aa2d90bc635
Author: JD <[email protected]>
Date: 2015-07-16T04:33:29Z
Added virtual column resolution by adding a new resolution rule.
commit 0a6a0a508a18e19edd1a802d91682a991b772242
Author: Joseph Batchik <[email protected]>
Date: 2015-07-17T19:58:36Z
cleaned up test cases
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
