[
https://issues.apache.org/jira/browse/HIVE-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493725#comment-13493725
]
Thejas M Nair commented on HIVE-3693:
-------------------------------------
There is more change required than just adding a Shell.windows check in one
place, because the code has been changed to use Path instead of Strings
representation of paths. This was done because Path handles issues like some
string representations have paths that start with "/C:" (result of
Path.toUri().getPath()), while others have "C:" (Path.toString()). But once
when Path comparison were introduced, I saw issues because given path in
CombineFilter.accept(path) didn't have scheme but paths
CombineFilter.filterPaths had scheme, and hence the change to make given path
fully qualified in checkFilterPathContains.
Though using Path makes the code more extensible (eg in case of query against
files on different file system, scheme should be considered), I think it needs
a more holistic change is required to use it (HIVE-3616).
As a quick fix,I think the code can be changed back to using strings instead of
path, and change the way path in accept(Path path) is converted to string -
(replace Path.toString() with Path.toUri().toString().)
> Performance regression introduced by HIVE-3483
> ----------------------------------------------
>
> Key: HIVE-3693
> URL: https://issues.apache.org/jira/browse/HIVE-3693
> Project: Hive
> Issue Type: Bug
> Reporter: Gang Tim Liu
> Priority: Critical
>
> https://issues.apache.org/jira/browse/HIVE-3483 introduced a performance
> regression in the client side during split computation.
> The client side spends a lot more time in the split computation phase. The
> problem is checkFilterPathContains method.
> While investigating, can you create a config to disable it by default?
> thanks
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
