[
https://issues.apache.org/jira/browse/HIVE-8352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HIVE-8352:
------------------------------
Status: Patch Available (was: Open)
Attached a patch that enabled windowing.q for spark. The test output is about
the same as that of MapReduce, except some counter related exceptions:
{noformat}
[Error 30017]: Skipping stats aggregation by error org.apache.hadoop.h <
[Error 30017]: Skipping stats aggregation by error org.apache.hadoop.h <
[Error 30017]: Skipping stats aggregation by error org.apache.hadoop.h <
{noformat}
> Enable windowing.q for spark
> ----------------------------
>
> Key: HIVE-8352
> URL: https://issues.apache.org/jira/browse/HIVE-8352
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Brock Noland
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: hive-8385.patch
>
>
> We should enable windowing.q for basic windowing coverage. After checking out
> the spark branch, we would build:
> {noformat}
> $ mvn clean install -DskipTests -Phadoop-2
> $ cd itests/
> $ mvn clean install -DskipTests -Phadoop-2
> {noformat}
> Then generate the windowing.q.out file:
> {noformat}
> $ cd qtest-spark/
> $ mvn test -Dtest=TestSparkCliDriver -Dqfile=windowing.q -Phadoop-2
> -Dtest.output.overwrite=true
> {noformat}
> Compare the output against MapReduce:
> {noformat}
> $ diff -y -W 150
> ../../ql/src/test/results/clientpositive/spark/windowing.q.out
> ../../ql/src/test/results/clientpositive/windowing.q.out| less
> {noformat}
> And if everything looks good, add it to {{spark.query.files}} in
> {{./itests/src/test/resources/testconfiguration.properties}}
> then submit the patch including the .q file
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
