-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68099/
-----------------------------------------------------------
(Updated July 30, 2018, 10:22 p.m.)
Review request for hive and Carl Steinbach.
Changes
-------
fix the issues found by Hive QA
Bugs: HIVE-20225
https://issues.apache.org/jira/browse/HIVE-20225
Repository: hive-git
Description
-------
When using TPT/BTEQ to export Data from Teradata, Teradata will export binary
files based on the schema.
A Customized SerDe is needed in order to directly read these files from Hive.
CREATE EXTERNAL TABLE `TABLE1`(
...)
PARTITIONED BY (
...)
ROW FORMAT SERDE
'org.apache.hadoop.hive.contrib.serde2.TeradataBinarySerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.contrib.fileformat.teradata.TeradataBinaryFileInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.contrib.fileformat.teradata.TeradataBinaryFileOutputFormat'
LOCATION ...;
SELECT * FROM `TABLE1`;
Problem Statement:
Right now the fast way to export data from Teradata is using TPT. However, the
Hive could not directly utilize these exported binary format because it doesn't
have a SerDe for these files.
Result:
Provided with the SerDe, Hive can operate upon the exported Teradata Binary
Format file transparently.
Diffs (updated)
-----
contrib/src/java/org/apache/hadoop/hive/contrib/fileformat/teradata/TeradataBinaryFileInputFormat.java
PRE-CREATION
contrib/src/java/org/apache/hadoop/hive/contrib/fileformat/teradata/TeradataBinaryFileOutputFormat.java
PRE-CREATION
contrib/src/java/org/apache/hadoop/hive/contrib/fileformat/teradata/TeradataBinaryRecordReader.java
PRE-CREATION
contrib/src/java/org/apache/hadoop/hive/contrib/serde2/teradata/TeradataBinaryDataInputStream.java
PRE-CREATION
contrib/src/java/org/apache/hadoop/hive/contrib/serde2/teradata/TeradataBinaryDataOutputStream.java
PRE-CREATION
contrib/src/java/org/apache/hadoop/hive/contrib/serde2/teradata/TeradataBinarySerde.java
PRE-CREATION
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestGeneralFunctions.java
PRE-CREATION
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeForDate.java
PRE-CREATION
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeForDecimal.java
PRE-CREATION
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeForTimeStamp.java
PRE-CREATION
contrib/src/test/org/apache/hadoop/hive/contrib/serde2/TestTeradataBinarySerdeGeneral.java
PRE-CREATION
Diff: https://reviews.apache.org/r/68099/diff/2/
Changes: https://reviews.apache.org/r/68099/diff/1-2/
Testing
-------
Junit tests have been added for Serialization and Deserialization functions
Thanks,
Lu Li
