[ https://issues.apache.org/jira/browse/HIVE-14815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15631499#comment-15631499 ]
Hive QA commented on HIVE-14815: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12836718/HIVE-14815.4.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1935/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1935/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1935/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2016-11-03 04:14:52.111 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + MAVEN_OPTS='-Xmx1g -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-1935/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2016-11-03 04:14:52.114 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 345353c HIVE-15039: A better job monitor console output for HoS (Rui reviewed by Xuefu and Ferdinand) + git clean -f -d Removing common/src/java/org/apache/hadoop/hive/common/ValidWriteIds.java Removing common/src/java/org/apache/hadoop/hive/conf/HiveConf.java.orig Removing common/src/test/org/apache/hive/common/util/MockFileSystem.java Removing metastore/scripts/upgrade/derby/037-HIVE-14637.derby.sql Removing metastore/scripts/upgrade/mssql/022-HIVE-14637.mssql.sql Removing metastore/scripts/upgrade/mysql/037-HIVE-14637.mysql.sql Removing metastore/scripts/upgrade/oracle/037-HIVE-14637.oracle.sql Removing metastore/scripts/upgrade/postgres/036-HIVE-14637.postgres.sql Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FinalizeWriteIdRequest.java Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FinalizeWriteIdResult.java Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetNextWriteIdRequest.java Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetNextWriteIdResult.java Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetValidWriteIdsRequest.java Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/GetValidWriteIdsResult.java Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatWriteIdRequest.java Removing metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/HeartbeatWriteIdResult.java Removing metastore/src/java/org/apache/hadoop/hive/metastore/MmCleanerThread.java Removing metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableWrite.java Removing ql/src/java/org/apache/hadoop/hive/ql/exec/ImportCommitTask.java Removing ql/src/java/org/apache/hadoop/hive/ql/exec/ImportCommitWork.java Removing ql/src/test/queries/clientnegative/mm_concatenate.q Removing ql/src/test/queries/clientnegative/mm_truncate_cols.q Removing ql/src/test/queries/clientpositive/mm_all.q Removing ql/src/test/queries/clientpositive/mm_all2.q Removing ql/src/test/queries/clientpositive/mm_conversions.q Removing ql/src/test/queries/clientpositive/mm_current.q Removing ql/src/test/queries/clientpositive/mm_insertonly_acid.q Removing ql/src/test/results/clientnegative/mm_concatenate.q.out Removing ql/src/test/results/clientnegative/mm_truncate_cols.q.out Removing ql/src/test/results/clientpositive/llap/mm_all.q.out Removing ql/src/test/results/clientpositive/llap/mm_all2.q.out Removing ql/src/test/results/clientpositive/llap/mm_conversions.q.out Removing ql/src/test/results/clientpositive/llap/mm_current.q.out Removing ql/src/test/results/clientpositive/mm_insertonly_acid.q.out + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 345353c HIVE-15039: A better job monitor console output for HoS (Rui reviewed by Xuefu and Ferdinand) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2016-11-03 04:14:53.221 + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch Going to apply patch with: patch -p0 patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ParquetRecordReaderBase.java patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/VectorizedParquetInputFormat.java patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/timestamp/NanoTimeUtils.java patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedColumnReader.java patching file ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java patching file ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestVectorizedColumnReader.java patching file ql/src/test/queries/clientpositive/parquet_types_vectorization.q patching file ql/src/test/results/clientpositive/llap/vectorized_parquet.q.out patching file ql/src/test/results/clientpositive/llap/vectorized_parquet_types.q.out patching file ql/src/test/results/clientpositive/parquet_types_vectorization.q.out patching file ql/src/test/results/clientpositive/vectorized_parquet_types.q.out + [[ maven == \m\a\v\e\n ]] + rm -rf /data/hiveptest/working/maven/org/apache/hive + mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven ANTLR Parser Generator Version 3.4 org/apache/hadoop/hive/metastore/parser/Filter.g DataNucleus Enhancer (version 4.1.6) for API "JDO" DataNucleus Enhancer : Classpath >> /usr/share/maven/boot/plexus-classworlds-2.x.jar ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MDatabase ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MFieldSchema ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MType ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MTable ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MConstraint ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MSerDeInfo ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MOrder ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MColumnDescriptor ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MStringList ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MStorageDescriptor ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MPartition ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MIndex ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MRole ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MRoleMap ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MGlobalPrivilege ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MDBPrivilege ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MTablePrivilege ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MPartitionPrivilege ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MTableColumnPrivilege ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MPartitionColumnPrivilege ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MPartitionEvent ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MMasterKey ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MDelegationToken ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MTableColumnStatistics ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MVersionTable ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MResourceUri ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MFunction ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MNotificationLog ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MNotificationNextId DataNucleus Enhancer completed with success for 30 classes. Timings : input=175 ms, enhance=218 ms, total=393 ms. Consult the log for full details Generating vector expression code Generating vector expression test code ANTLR Parser Generator Version 3.4 org/apache/hadoop/hive/ql/parse/HiveLexer.g org/apache/hadoop/hive/ql/parse/HiveParser.g [ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on project hive-exec: Error resolving project artifact: Could not transfer artifact org.pentaho:pentaho-aggdesigner-algorithm:pom:5.1.5-jhyde from/to datanucleus (http://www.datanucleus.org/downloads/maven2): Connect to localhost:3128 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused for project org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hive-exec + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12836718 - PreCommit-HIVE-Build > Implement Parquet vectorization reader for Primitive types > ----------------------------------------------------------- > > Key: HIVE-14815 > URL: https://issues.apache.org/jira/browse/HIVE-14815 > Project: Hive > Issue Type: Sub-task > Reporter: Ferdinand Xu > Assignee: Ferdinand Xu > Attachments: HIVE-14815.1.patch, HIVE-14815.2.patch, > HIVE-14815.3.patch, HIVE-14815.4.patch, HIVE-14815.patch > > > Parquet doesn't provide a vectorized reader which can be used by Hive > directly. Also for Decimal Column batch, it consists of a batch of > HiveDecimal which is a Hive type which is unknown for Parquet. To support > Hive vectorization execution engine in Hive, we have to implement the > vectorized Parquet reader in Hive side. To limit the performance impacts, we > need to implement a page level vectorized reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)