[
https://issues.apache.org/jira/browse/HIVE-20225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599533#comment-16599533
]
Hive QA commented on HIVE-20225:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12937988/HIVE-20225.17-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 107 failed/errored test(s), 14445 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=273)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=273)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=233)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestListPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestMarkPartition - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMarkPartitionRemote - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestMetaStoreConnectionUrlHook - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestMetaStoreEndFunctionListener - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestMetaStoreEventListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestMetaStoreEventListenerOnlyOnCommit - did not produce a TEST-*.xml file
(likely timed out) (batchId=236)
TestMetaStoreEventListenerWithOldConf - did not produce a TEST-*.xml file
(likely timed out) (batchId=238)
TestMetaStoreInitListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestMetaStoreListenersError - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestMetaStoreSchemaFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestMetaStoreSchemaInfo - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMetaStoreServerUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMetastoreConf - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestMetastoreSchemaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMetrics - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=273)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=273)
TestNotNullConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestObjectStoreInitRetry - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestObjectStoreSchemaMethods - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestOldSchema - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestPartitionNameWhitelistValidation - did not produce a TEST-*.xml file
(likely timed out) (batchId=230)
TestPreUpgradeTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=331)
TestPrimaryKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestRawStoreProxy - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestRemoteHiveMetaStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestRemoteHiveMetaStoreIpAddress - did not produce a TEST-*.xml file (likely
timed out) (batchId=226)
TestRemoteUGIHiveMetaStoreIpAddress - did not produce a TEST-*.xml file (likely
timed out) (batchId=235)
TestRetriesInRetryingHMSHandler - did not produce a TEST-*.xml file (likely
timed out) (batchId=238)
TestRetryingHMSHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestRuntimeStats - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestSchemaToolForMetastore - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestSetUGIOnBothClientServer - did not produce a TEST-*.xml file (likely timed
out) (batchId=229)
TestSetUGIOnOnlyClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=227)
TestSetUGIOnOnlyServer - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestSparseEncodeHash - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestStats - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
TestStatsSetupConst - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestTableIterable - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestTablesCreateDropAlterTruncate - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestTablesGetExists - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestTablesList - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=273)
TestTxnHandlerNegative - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestTxnUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestUniqueConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_mv] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_all] (batchId=71)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_partitioned_2]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_partitioned_3]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_part_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_with_masking]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[constprog_semijoin]
(batchId=189)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket_map_join_tez2]
(batchId=114)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths
(batchId=250)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
(batchId=258)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=325)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13546/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13546/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13546/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 107 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12937988 - PreCommit-HIVE-Build
> SerDe to support Teradata Binary Format
> ---------------------------------------
>
> Key: HIVE-20225
> URL: https://issues.apache.org/jira/browse/HIVE-20225
> Project: Hive
> Issue Type: New Feature
> Components: Serializers/Deserializers
> Reporter: Lu Li
> Assignee: Lu Li
> Priority: Major
> Attachments: HIVE-20225.1.patch, HIVE-20225.10.patch,
> HIVE-20225.11.patch, HIVE-20225.12.patch, HIVE-20225.13.patch,
> HIVE-20225.14-branch-2.patch, HIVE-20225.15.patch,
> HIVE-20225.16-branch-2.patch, HIVE-20225.17-branch-3.patch,
> HIVE-20225.2.patch, HIVE-20225.3.patch, HIVE-20225.4.patch,
> HIVE-20225.5-branch-2.patch, HIVE-20225.6.patch, HIVE-20225.7.patch,
> HIVE-20225.8.patch, HIVE-20225.9.patch
>
>
> When using TPT/BTEQ to export/import Data from Teradata, Teradata will
> generate/require binary files based on the schema.
> A Customized SerDe is needed in order to directly read these files from Hive
> or write these files in order to load back to TD.
> {code:java}
> CREATE EXTERNAL TABLE `TABLE1`(
> ...)
> PARTITIONED BY (
> ...)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.contrib.serde2.TeradataBinarySerde'
> STORED AS INPUTFORMAT
>
> 'org.apache.hadoop.hive.contrib.fileformat.teradata.TeradataBinaryFileInputFormat'
> OUTPUTFORMAT
>
> 'org.apache.hadoop.hive.contrib.fileformat.teradata.TeradataBinaryFileOutputFormat'
> LOCATION ...;
> SELECT * FROM `TABLE1`;{code}
> Problem Statement:
> Right now the fast way to export/import data from Teradata is using TPT.
> However, the Hive could not directly utilize/generate these binary format
> because it doesn't have a SerDe for these files.
> Result:
> Provided with the SerDe, Hive can operate upon/generate the exported Teradata
> Binary Format file transparently
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
