Hi,
Does anyone know how I can reduce the level of logging of the Hive History
file which are created per Hive statement executed in the (default) location
of /tmp/${user.name}/ hive_job_log_hdfs_201109122109_675419678.txt. The
level of detail is very high by default but I can't find a way to tune it
down.
It looks like this:
-------------------------------
SessionStart SESSION_ID="hdfs_201109122110" TIME="1315854630381"
QueryStart QUERY_STRING="show partitions dart_facts"
QUERY_ID="hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4"
TIME="1315854630789"
Counters
plan="{"queryId":"hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4","queryType":null,"queryAttributes":{"queryString":"show
partitions
dart_facts"},"queryCounters":"null","stageGraph":{"nodeType":"STAGE","roots":"null","adjacencyList":"]"},"stageList":[{"stageId":"Stage-0","stageType":"DDL","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-0_OTHER","taskType":"OTHER","taskAttributes":"null","taskCounters":"null","operatorGraph":"null","operatorList":"]","done":"false","started":"false"}],"done":"false","started":"false"}],"done":"false","started":"true"}"
TIME="1315854630797"
TaskStart TASK_NAME="org.apache.hadoop.hive.ql.exec.DDLTask"
TASK_ID="Stage-0"
QUERY_ID="hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4"
TIME="1315854630805"
Counters
plan="{"queryId":"hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4","queryType":null,"queryAttributes":{"queryString":"show
partitions
dart_facts"},"queryCounters":"null","stageGraph":{"nodeType":"STAGE","roots":"null","adjacencyList":"]"},"stageList":[{"stageId":"Stage-0","stageType":"DDL","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-0_OTHER","taskType":"OTHER","taskAttributes":"null","taskCounters":"null","operatorGraph":"null","operatorList":"]","done":"false","started":"true"}],"done":"false","started":"true"}],"done":"false","started":"true"}"
TIME="1315854630806"
-----------------------------------------
It doesn't look like standard log4j logging to me and this logging does not
respond to the log4j properties in :
-/usr/lib/hive/conf/hive-log4j (these settings influence
the detail in HIVE_LOG_DIR/hive.log)
-/usr/lib/hive/conf/hive-exec-log4j (don't know where these
settings are meant for. Anyone ? )
I have to find a way to turn the logging in /tmp/${user.name}/ down. I ran a
large Hive job once that generated 8GB of logging this way which stalled the
server.
Thanks
--
Kind Regards,
