[
https://issues.apache.org/jira/browse/HIVE-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753254#comment-15753254
]
Chao Sun commented on HIVE-15441:
---------------------------------
Thanks [~sershe] for taking a look. Regarding 1), yes it doesn't have to - the
only purpose is to quit the loop when {{shouldStop}} is true, but I guess I can
just interrupt the thread in that case. Will change it.
2) it won't kill, but just interrupt. Yes in CLI it's the main thread while in
HS2 it's the handler thread. Both cases it will recover. However, the error
message is a little misleading since it's wrapped with potentially
many other exceptions on top of it. I'll see if we can improve the message.
I do have one concern about this patch though: the interrupted exception could
literally happen at ANY point during the compilation process, and I'm not sure
if it can be handled gracefully in all the places. Any thought on that?
> Provide a config to timeout long compiling queries
> --------------------------------------------------
>
> Key: HIVE-15441
> URL: https://issues.apache.org/jira/browse/HIVE-15441
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
> Reporter: Chao Sun
> Assignee: Chao Sun
> Attachments: HIVE-15441.1.patch
>
>
> Sometimes Hive users have long compiling queries which may need to scan
> thousands or even more partitions (perhaps by accident). The compilation
> process may take a very long time, especially in {{getInputSummary}} where it
> need to make NN calls to get info about each input path.
> This is bad because it may block many other queries. Parallel compilation may
> be useful but still {{getInputSummary}} has a global lock. In this case, it
> makes sense to provide Hive admin with a config to put a timeout limit for
> compilation, so that these "bad" queries can be blocked.
> Note https://issues.apache.org/jira/browse/HIVE-12431 also tries to address
> similar issue. However it cancels those queries that are waiting for the
> compile lock, which I think is not so useful for our case since the *query
> under compile is the one to be blamed.*
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
