Async logging was added to hive 2.1 version onwards via https://issues.apache.org/jira/browse/HIVE-13027
Thanks Prasanth _____________________________ From: Xionghua Hu <huasanye...@gmail.com<mailto:huasanye...@gmail.com>> Sent: Thursday, July 7, 2016 10:54 PM Subject: Re: hive 2.1.0 beeline cannot show verbose log To: <user@hive.apache.org<mailto:user@hive.apache.org>> Yes, works well. Logging initialized using configuration in file:/data/hadoopbin/apache-hive-2.1.0-bin/conf/hive-log4j2.properties Async: true Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases. hive> set hive.async.log.enabled=false; hive> some_sql; Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 2016-07-08 13:48:02,871 Stage-1 map = 0%, reduce = 0% 2016-07-08 13:48:14,902 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.59 sec 2016-07-08 13:48:28,101 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 6.3 sec However there is no need to set hive.async.log.enabled=false; for Hive 2. In Hive 2 CLI, the verbose log show normally with hive.async.log.enabled=true. 2016-07-08 12:01 GMT+08:00 Mich Talebzadeh <mich.talebza...@gmail.com<mailto:mich.talebza...@gmail.com>>: Hi Is this available in Hive 2? hive> set hive.async.log.enabled=false; Query returned non-zero code: 1, cause: hive configuration hive.async.log.enabled does not exists. Thanks Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw http://talebzadehmich.wordpress.com Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destructionof data or any other property which may arise from relying on this email's technical content is explicitly disclaimed.The author will in no case be liable for any monetary damages arising from suchloss, damage or destruction. On 8 July 2016 at 04:26, Xionghua Hu <huasanye...@gmail.com<mailto:huasanye...@gmail.com>> wrote: Appologize for my careless. hive.async.log.enabled=false; can resolve the issue. the verbose log can show normally. Thanks ! 2016-07-08 10:48 GMT+08:00 Xionghua Hu <huasanye...@gmail.com<mailto:huasanye...@gmail.com>>: I have tried the set : hive.async.log.enabled=false; The same issue happens. The verbose log canno show. 2016-07-08 0:15 GMT+08:00 Prasanth Jayachandran <pjayachand...@hortonworks.com<mailto:pjayachand...@hortonworks.com>>: Hi Can you try disabling async logging in HS2 and see if that helps? set hive.async.log.enabled=false; Thanks Prasanth On Thu, Jul 7, 2016 at 3:37 AM -0700, "Mich Talebzadeh" <mich.talebza...@gmail.com<mailto:mich.talebza...@gmail.com>> wrote: Well this works in Hive 2 hive --hiveconf hive.root.logger=DEBUG,console Logging initialized using configuration in file:/usr/lib/hive/conf/hive-log4j2.properties 16/07/07 11:36:22 [main]: INFO SessionState: Logging initialized using configuration in file:/usr/lib/hive/conf/hive-log4j2.properties 16/07/07 11:36:22 [main]: DEBUG conf.VariableSubstitution: Substitution is on: hive HTH Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw http://talebzadehmich.wordpress.com Disclaimer: Use it at your own risk.Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 7 July 2016 at 09:46, Xionghua Hu <huasanye...@gmail.com<mailto:huasanye...@gmail.com>> wrote: Dear all, In Hive 1.2.1 , config the log verbose , the beeline client will show the verbose log like this: 16/07/07 13:29:33 INFO mapreduce.Job: The url to track the job: http://host:8088/proxy/application_1467708727273_0035/ 16/07/07 13:29:33 INFO exec.Task: Starting Job = job_1467708727273_0035, Tracking URL =http://host:8088/proxy/application_1467708727273_0035/ 16/07/07 13:29:33 INFO exec.Task: Kill Command = /hadoop/hadoop-2.7.2/bin/hadoop job -kill job_1467708727273_0035 16/07/07 13:30:07 INFO exec.Task: Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 16/07/07 13:30:07 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 16/07/07 13:30:07 INFO exec.Task: 2016-07-07 13:30:07,905 Stage-1 map = 0%, reduce = 0% 16/07/07 13:30:17 INFO exec.Task: 2016-07-07 13:30:17,757 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 2.39 sec 16/07/07 13:30:28 the verbose config: <property> <name>hive.server2.logging.operation.enabled</name> <value>true</value> <description>When true, HS2 will save operation logs and make them available for clients</description> </property> <property> <name>hive.server2.logging.operation.log.location</name> <value>/hadooplog/apache-hive-1.2.1-bin/operation_logs</value> <description>Top level directory where operation logs are stored if logging functionality is enabled</description> </property> <property> <name>hive.server2.logging.operation.level</name> <value>VERBOSE</value> <description> Expects one of [none, execution, performance, verbose]. HS2 operation logging mode available to clients to be set at session level. For this to work, hive.server2.logging.operation.enabled should be set to true. NONE: Ignore any logging EXECUTION: Log completion of tasks PERFORMANCE: Execution + Performance logs VERBOSE: All logs </description> </property> However, when upgrade to hive 2.1.0, with the same verbose configure, the verbose log cannot show.(Also, hue 3.10.0 cannot display the verbose log) so , how to enable the verbose log in hive 2.1.0? Any advise is appreciated. Thanks in advance!
