[
https://issues.apache.org/jira/browse/HIVE-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126643#comment-16126643
]
Hive QA commented on HIVE-17100:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12881838/HIVE-17100.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6391/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6391/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6391/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-08-14 23:56:04.494
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-6391/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-08-14 23:56:04.497
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 06d9a6b HIVE-16873: Remove Thread Cache From Logging (BELUGA
BEHR reviewed by Aihua Xu)
+ git clean -f -d
Removing common/src/java/org/apache/hadoop/hive/common/ndv/fm/FMSketch.java
Removing metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/cache/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 06d9a6b HIVE-16873: Remove Thread Cache From Logging (BELUGA
BEHR reviewed by Aihua Xu)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-08-14 23:56:11.579
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p1
patching file metastore/src/java/org/apache/hadoop/hive/metastore/TableType.java
patching file ql/if/queryplan.thrift
patching file ql/src/gen/thrift/gen-cpp/queryplan_types.cpp
patching file ql/src/gen/thrift/gen-cpp/queryplan_types.h
patching file
ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/StageType.java
patching file ql/src/gen/thrift/gen-php/Types.php
patching file ql/src/gen/thrift/gen-py/queryplan/ttypes.py
patching file ql/src/gen/thrift/gen-rb/queryplan_types.rb
patching file ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java
patching file ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplStateLogTask.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplStateLogWork.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/ReplLoadTask.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/ReplLoadWork.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/events/filesystem/BootstrapEventsIterator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/events/filesystem/DatabaseEventsIterator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/LoadFunction.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java
patching file ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSpec.java
patching file ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/Utils.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/AbstractMessageHandler.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/CreateFunctionHandler.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/logger/BootstrapDumpLogger.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/logger/BootstrapLoadLogger.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/logger/IncrementalDumpLogger.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/logger/IncrementalLoadLogger.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/logger/ReplLogger.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/AbstractReplLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapDumpBeginLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapDumpEndLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapDumpFunctionLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapDumpTableLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapLoadBeginLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapLoadEndLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapLoadFunctionLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/BootstrapLoadTableLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/IncrementalDumpBeginLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/IncrementalDumpEndLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/IncrementalDumpEventLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/IncrementalLoadBeginLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/IncrementalLoadEndLog.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/log/message/IncrementalLoadEventLog.java
patching file ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java
patching file standalone-metastore/src/gen/thrift/gen-py/__init__.py (renamed
from metastore/src/gen/thrift/gen-py/__init__.py)
patch: **** Can't get file attributes of file
standalone-metastore/src/gen/thrift/gen-py/__init__.py : No such file or
directory
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12881838 - PreCommit-HIVE-Build
> Improve HS2 operation logs for REPL commands.
> ---------------------------------------------
>
> Key: HIVE-17100
> URL: https://issues.apache.org/jira/browse/HIVE-17100
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, repl
> Affects Versions: 2.1.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Labels: DR, replication
> Fix For: 3.0.0
>
> Attachments: HIVE-17100.01.patch
>
>
> It is necessary to log the progress the replication tasks in a structured
> manner as follows.
> *+Bootstrap Dump:+*
> * At the start of bootstrap dump, will add one log with below details.
> {color:#59afe1}* Database Name
> * Dump Type (BOOTSTRAP)
> * (Estimated) Total number of tables/views to dump
> * (Estimated) Total number of functions to dump.
> * Dump Start Time{color}
> * After each table dump, will add a log as follows
> {color:#59afe1}* Table/View Name
> * Type (TABLE/VIEW/MATERIALIZED_VIEW)
> * Table dump end time
> * Table dump progress. Format is Table sequence no/(Estimated) Total number
> of tables and views.{color}
> * After each function dump, will add a log as follows
> {color:#59afe1}* Function Name
> * Function dump end time
> * Function dump progress. Format is Function sequence no/(Estimated) Total
> number of functions.{color}
> * After completion of all dumps, will add a log as follows to consolidate the
> dump.
> {color:#59afe1}* Database Name.
> * Dump Type (BOOTSTRAP).
> * Dump End Time.
> * (Actual) Total number of tables/views dumped.
> * (Actual) Total number of functions dumped.
> * Dump Directory.
> * Last Repl ID of the dump.{color}
> *Note:* The actual and estimated number of tables/functions may not match if
> any table/function is dropped when dump in progress.
> *+Bootstrap Load:+*
> * At the start of bootstrap load, will add one log with below details.
> {color:#59afe1}* Database Name
> * Dump directory
> * Load Type (BOOTSTRAP)
> * Total number of tables/views to load
> * Total number of functions to load.
> * Load Start Time{color}
> * After each table load, will add a log as follows
> {color:#59afe1}* Table/View Name
> * Type (TABLE/VIEW/MATERIALIZED_VIEW)
> * Table load completion time
> * Table load progress. Format is Table sequence no/Total number of tables and
> views.{color}
> * After each function load, will add a log as follows
> {color:#59afe1}* Function Name
> * Function load completion time
> * Function load progress. Format is Function sequence no/Total number of
> functions.{color}
> * After completion of all dumps, will add a log as follows to consolidate the
> load.
> {color:#59afe1}* Database Name.
> * Load Type (BOOTSTRAP).
> * Load End Time.
> * Total number of tables/views loaded.
> * Total number of functions loaded.
> * Last Repl ID of the loaded database.{color}
> *+Incremental Dump:+*
> * At the start of database dump, will add one log with below details.
> {color:#59afe1}* Database Name
> * Dump Type (INCREMENTAL)
> * (Estimated) Total number of events to dump.
> * Dump Start Time{color}
> * After each event dump, will add a log as follows
> {color:#59afe1}* Event ID
> * Event Type (CREATE_TABLE, DROP_TABLE, ALTER_TABLE, INSERT etc)
> * Event dump end time
> * Event dump progress. Format is Event sequence no/ (Estimated) Total number
> of events.{color}
> * After completion of all event dumps, will add a log as follows.
> {color:#59afe1}* Database Name.
> * Dump Type (INCREMENTAL).
> * Dump End Time.
> * (Actual) Total number of events dumped.
> * Dump Directory.
> * Last Repl ID of the dump.{color}
> *Note:* The estimated number of events can be terribly inaccurate with actual
> number as we don’t have the number of events upfront until we read from
> metastore NotificationEvents table.
> *+Incremental Load:+*
> * At the start of incremental load, will add one log with below details.
> {color:#59afe1}* Target Database Name
> * Dump directory
> * Load Type (INCREMENTAL)
> * Total number of events to load
> * Load Start Time{color}
> * After each event load, will add a log as follows
> {color:#59afe1}* Event ID
> * Event Type (CREATE_TABLE, DROP_TABLE, ALTER_TABLE, INSERT etc)
> * Event load end time
> * Event load progress. Format is Event sequence no/ Total number of
> events.{color}
> * After completion of all event loads, will add a log as follows to
> consolidate the load.
> {color:#59afe1}* Target Database Name.
> * Load Type (INCREMENTAL).
> * Load End Time.
> * Total number of events loaded.
> * Last Repl ID of the loaded database.{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
