[ 
https://issues.apache.org/jira/browse/HIVE-22971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17054199#comment-17054199
 ] 

Hive QA commented on HIVE-22971:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12995882/HIVE-22971.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 28 failed/errored test(s), 18103 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testOperationsOnCompletedTxnComponentsForMmTable
 (batchId=317)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testOperationsOnCompletedTxnComponentsForMmTable
 (batchId=339)
org.apache.hadoop.hive.ql.TestTxnConcatenate.testConcatenateMM (batchId=327)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
 (batchId=270)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths 
(batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTable (batchId=255)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableBucketed 
(batchId=255)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableOpenWriteId 
(batchId=255)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableOriginalsOrc 
(batchId=255)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableOriginalsText 
(batchId=255)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTablePartitioned 
(batchId=255)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMinorMmCompactionRemovesAbortedDirs
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMajorCompactionAfterMinor
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompaction10DeltaDirs
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionAfterMajor
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionNotPartitionedWithBuckets
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionNotPartitionedWithoutBuckets
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithBucketsAvro
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithBucketsJsonFile
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithBucketsOrc
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithBucketsParquet
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithBucketsRcFile
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithBucketsSequenceFile
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithBucketsTextFile
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionPartitionedWithoutBuckets
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionWithSchemaEvolutionAndBuckets
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMmMinorCompactionWithSchemaEvolutionNoBucketsMultipleReducers
 (batchId=270)
org.apache.hadoop.hive.ql.txn.compactor.TestMmCompactorOnTez.testMultipleMmMinorCompactions
 (batchId=270)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/21003/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21003/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21003/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 28 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12995882 - PreCommit-HIVE-Build

> Eliminate file rename in insert-only compactor
> ----------------------------------------------
>
>                 Key: HIVE-22971
>                 URL: https://issues.apache.org/jira/browse/HIVE-22971
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Karen Coppage
>            Assignee: Karen Coppage
>            Priority: Major
>              Labels: ACID, compaction
>         Attachments: HIVE-22971.01.patch
>
>
> File rename is expensive for object stores, so MM (insert-only) compaction 
> should skip that step when committing and write directly to base_x_cZ or 
> delta_x_y_cZ.
> This also fixes the issue that for MM QB compaction the temp tables were 
> stored under the table directory, and these temp dirs were never cleaned up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to