[
https://issues.apache.org/jira/browse/HIVE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551922#comment-16551922
]
Hive QA commented on HIVE-19267:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12932563/HIVE-19267.02-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 96 failed/errored test(s), 14406 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=271)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=226)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=271)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=229)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=234)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=231)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestListPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestMarkPartition - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestMarkPartitionRemote - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMetaStoreConnectionUrlHook - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestMetaStoreEndFunctionListener - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestMetaStoreEventListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=232)
TestMetaStoreEventListenerOnlyOnCommit - did not produce a TEST-*.xml file
(likely timed out) (batchId=234)
TestMetaStoreEventListenerWithOldConf - did not produce a TEST-*.xml file
(likely timed out) (batchId=236)
TestMetaStoreInitListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestMetaStoreListenersError - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestMetaStoreSchemaFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestMetaStoreSchemaInfo - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestMetaStoreUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestMetastoreConf - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestMetastoreSchemaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestMetrics - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=271)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=271)
TestNotNullConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestObjectStoreInitRetry - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestObjectStoreSchemaMethods - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestOldSchema - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestPartitionNameWhitelistValidation - did not produce a TEST-*.xml file
(likely timed out) (batchId=228)
TestPrimaryKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestRawStoreProxy - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestRemoteHiveMetaStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestRemoteHiveMetaStoreIpAddress - did not produce a TEST-*.xml file (likely
timed out) (batchId=224)
TestRemoteUGIHiveMetaStoreIpAddress - did not produce a TEST-*.xml file (likely
timed out) (batchId=233)
TestRetriesInRetryingHMSHandler - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestRetryingHMSHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestRuntimeStats - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestSchemaToolForMetastore - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestSetUGIOnBothClientServer - did not produce a TEST-*.xml file (likely timed
out) (batchId=227)
TestSetUGIOnOnlyClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=225)
TestSetUGIOnOnlyServer - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestSparseEncodeHash - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestStats - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestStatsSetupConst - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestTableIterable - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestTablesCreateDropAlterTruncate - did not produce a TEST-*.xml file (likely
timed out) (batchId=226)
TestTablesGetExists - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestTablesList - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=271)
TestTxnHandlerNegative - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestTxnUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestUniqueConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_with_masking]
(batchId=174)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths
(batchId=248)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
(batchId=256)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=323)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12774/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12774/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12774/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 96 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12932563 - PreCommit-HIVE-Build
> Replicate ACID/MM tables write operations.
> ------------------------------------------
>
> Key: HIVE-19267
> URL: https://issues.apache.org/jira/browse/HIVE-19267
> Project: Hive
> Issue Type: Sub-task
> Components: repl, Transactions
> Affects Versions: 3.0.0
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: ACID, DR, pull-request-available, replication
> Fix For: 4.0.0
>
> Attachments: HIVE-19267.01-branch-3.patch, HIVE-19267.01.patch,
> HIVE-19267.02-branch-3.patch, HIVE-19267.02.patch, HIVE-19267.03.patch,
> HIVE-19267.04.patch, HIVE-19267.05.patch, HIVE-19267.06.patch,
> HIVE-19267.07.patch, HIVE-19267.08.patch, HIVE-19267.09.patch,
> HIVE-19267.10.patch, HIVE-19267.11.patch, HIVE-19267.12.patch,
> HIVE-19267.13.patch, HIVE-19267.14.patch, HIVE-19267.15.patch,
> HIVE-19267.16.patch, HIVE-19267.17.patch, HIVE-19267.18.patch,
> HIVE-19267.19.patch, HIVE-19267.20.patch, HIVE-19267.21.patch,
> HIVE-19267.22.patch
>
>
>
> h1. Replicate ACID write Events
> * Create new EVENT_WRITE event with related message format to log the write
> operations with in a txn along with data associated.
> * Log this event when perform any writes (insert into, insert overwrite,
> load table, delete, update, merge, truncate) on table/partition.
> * If a single MERGE/UPDATE/INSERT/DELETE statement operates on multiple
> partitions, then need to log one event per partition.
> * DbNotificationListener should log this type of event to special metastore
> table named "MTxnWriteNotificationLog".
> * This table should maintain a map of txn ID against list of
> tables/partitions written by given txn.
> * The entry for a given txn should be removed by the cleaner thread that
> removes the expired events from EventNotificationTable.
> h1. Replicate Commit Txn operation (with writes)
> Add new EVENT_COMMIT_TXN to log the metadata/data of all tables/partitions
> modified within the txn.
> *Source warehouse:*
> * This event should read the EVENT_WRITEs from "MTxnWriteNotificationLog"
> metastore table to consolidate the list of tables/partitions modified within
> this txn scope.
> * Based on the list of tables/partitions modified and table Write ID, need
> to compute the list of delta files added by this txn.
> * Repl dump should read this message and dump the metadata and delta files
> list.
> *Target warehouse:*
> * Ensure snapshot isolation at target for on-going read txns which shouldn't
> view the data replicated from committed txn. (Ensured with open and allocate
> write ID events).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
