[
https://issues.apache.org/jira/browse/HIVE-17300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622140#comment-16622140
]
Karen Coppage commented on HIVE-17300:
--------------------------------------
[~szehon], thank you so much for taking a look! I really appreciate your taking
the time.
To answer your questions:
# The method I use to get the log4j logs is just refactored code from
HadoopJobExecHelper. My initial thought is that since HadoopJobExecHelper is
the main source of information for MapRedStats, getting the log file from here
is a good idea; plus, OperationLog seems to be quite distant, i.e. accessible
through SessionManager only. But I'll look into this further.
# org.apache.commons.lang.StringUtils is already imported in MapRedTask,
however here I am using hadoop.util.StringUtils. As far as I can tell, this is
how errors are logged in Hive, for example in Driver and SemanticAnalyzer.
> WebUI query plan graphs
> -----------------------
>
> Key: HIVE-17300
> URL: https://issues.apache.org/jira/browse/HIVE-17300
> Project: Hive
> Issue Type: Sub-task
> Components: Web UI
> Affects Versions: 4.0.0
> Reporter: Karen Coppage
> Assignee: Karen Coppage
> Priority: Major
> Labels: beginner, features, patch
> Attachments: HIVE-17300.3.patch, HIVE-17300.4.patch,
> HIVE-17300.5.patch, HIVE-17300.6.patch, HIVE-17300.7.patch,
> HIVE-17300.7.patch, HIVE-17300.8.patch, HIVE-17300.8.patch,
> HIVE-17300.8.patch, HIVE-17300.patch, complete_success.png,
> full_mapred_stats.png, graph_with_mapred_stats.png, last_stage_error.png,
> last_stage_running.png, non_mapred_task_selected.png
>
>
> Hi all,
> I’m working on a feature of the Hive WebUI Query Plan tab that would provide
> the option to display the query plan as a nice graph (scroll down for
> screenshots). If you click on one of the graph’s stages, the plan for that
> stage appears as text below.
> Stages are color-coded if they have a status (Success, Error, Running), and
> the rest are grayed out. Coloring is based on status already available in the
> WebUI, under the Stages tab.
> There is an additional option to display stats for MapReduce tasks. This
> includes the job’s ID, tracking URL (where the logs are found), and mapper
> and reducer numbers/progress, among other info.
> The library I’m using for the graph is called vis.js (http://visjs.org/). It
> has an Apache license, and the only necessary file to be included from this
> library is about 700 KB.
> I tried to keep server-side changes minimal, and graph generation is taken
> care of by the client. Plans with more than a given number of stages
> (default: 25) won't be displayed in order to preserve resources.
> I’d love to hear any and all input from the community about this feature: do
> you think it’s useful, and is there anything important I’m missing?
> Thanks,
> Karen Coppage
> Review request: https://reviews.apache.org/r/61663/
> Any input is welcome!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
