[
https://issues.apache.org/jira/browse/HIVE-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351233#comment-16351233
]
Hive QA commented on HIVE-18350:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909069/HIVE-18350.12.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 40 failed/errored test(s), 12966 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_cttas] (batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.org.apache.hadoop.hive.cli.TestContribNegativeCliDriver
(batchId=245)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_cttas]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.TestReplChangeManager.testRecycleNonPartTable
(batchId=223)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testTableOps
(batchId=213)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.TestBeeLineWithArgs.testEscapeCRLFOffInDSVOutput
(batchId=231)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=231)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testMulti (batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchAbort
(batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchCommitPartitioned
(batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchCommitUnpartitioned
(batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchEmptyAbortPartitioned
(batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchEmptyAbortUnartitioned
(batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchEmptyCommitPartitioned
(batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testTransactionBatchEmptyCommitUnpartitioned
(batchId=202)
org.apache.hive.hcatalog.streaming.mutate.TestMutations.testUpdatesAndDeletes
(batchId=202)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8994/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8994/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8994/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 40 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909069 - PreCommit-HIVE-Build
> load data should rename files consistent with insert statements
> ---------------------------------------------------------------
>
> Key: HIVE-18350
> URL: https://issues.apache.org/jira/browse/HIVE-18350
> Project: Hive
> Issue Type: Bug
> Reporter: Deepak Jaiswal
> Assignee: Deepak Jaiswal
> Priority: Major
> Attachments: HIVE-18350.1.patch, HIVE-18350.10.patch,
> HIVE-18350.11.patch, HIVE-18350.12.patch, HIVE-18350.2.patch,
> HIVE-18350.3.patch, HIVE-18350.4.patch, HIVE-18350.5.patch,
> HIVE-18350.6.patch, HIVE-18350.7.patch, HIVE-18350.8.patch, HIVE-18350.9.patch
>
>
> Insert statements create files of format ending with 0000_0, 0001_0 etc.
> However, the load data uses the input file name. That results in inconsistent
> naming convention which makes SMB joins difficult in some scenarios and may
> cause trouble for other types of queries in future.
> We need consistent naming convention.
> For non-bucketed table, hive renames all the files regardless of how they
> were named by the user.
> For bucketed table, hive relies on user to name the files matching the
> bucket in non-strict mode. Hive assumes that the data belongs to same bucket
> in a file. In strict mode, loading bucketed table is disabled.
> This will likely affect most of the tests which load data which is pretty
> significant due to which it is further divided into two subtasks for smoother
> merge.
> For existing tables in customer database, it is recommended to reload
> bucketed tables otherwise if customer tries to run SMB join and there is a
> bucket for which there is no split, then there is a possibility of getting
> incorrect results. However, this is not a regression as it would happen even
> without the patch.
> With this patch however, and reloading data, the results should be correct.
> For non-bucketed tables and external tables, there is no difference in
> behavior and reloading data is not needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
