[
https://issues.apache.org/jira/browse/HIVE-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anishek Agarwal resolved HIVE-25159.
------------------------------------
Resolution: Fixed
+1 , patch committed to master, Thanks for the patch [~ShubhamChaurasia] and
review [~harishjp]
> Remove support for ordered results in llap external client library
> ------------------------------------------------------------------
>
> Key: HIVE-25159
> URL: https://issues.apache.org/jira/browse/HIVE-25159
> Project: Hive
> Issue Type: Bug
> Components: Clients, Hive
> Reporter: Shubham Chaurasia
> Assignee: Shubham Chaurasia
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-25159.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently when querying via llap external client framework, in case of order
> by queries -
> 1. Due to the fact that spark-llap used to wrap actual query in a subquery as
> mentioned in [HIVE-19794|https://issues.apache.org/jira/browse/HIVE-19794]
> a) We had to detect order by like -
> {code}
> orderByQuery = plan.getQueryProperties().hasOrderBy() ||
> plan.getQueryProperties().hasOuterOrderBy();
> {code}
> Due to this we recently saw an exception like below for one of the queries
> that did not have an outer order by (It was having an order by in a subquery)
> {code}
> org.apache.hive.service.cli.HiveSQLException: java.io.IOException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.lang.IllegalStateException: Requested to generate single split. Paths
> and fileStatuses are expected to be 1. Got paths: 1 fileStatuses: 7
> {code}
> b) Also we had to disable following optimization -
> {code}
> HiveConf.setBoolVar(conf, ConfVars.HIVE_REMOVE_ORDERBY_IN_SUBQUERY, false);
> {code}
> 2. By default we have
> {{hive.llap.external.splits.order.by.force.single.split=true}} which forces
> us to generate single split leading to performance bottleneck.
> We should remove ordering support altogether from llap external client repo
> and let clients handle it at their end.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
