> On Feb. 3, 2014, 11:53 p.m., Xuefu Zhang wrote: > > ql/src/java/parquet/hive/DeprecatedParquetInputFormat.java, line 22 > > <https://reviews.apache.org/r/17061/diff/5/?file=454363#file454363line22> > > > > I don't quite get how this provides backward compatibility. Please > > explain. > > Brock Noland wrote: > The issue is that users til now specified the create table statement > below which puts those class names in the metastore. The same is true for all > parquet.hive.* classes included in this change. > > CREATE TABLE parquet_test (column1 string) > ROW FORMAT SERDE 'parquet.hive.serde.ParquetHiveSerDe' > STORED AS > INPUTFORMAT 'parquet.hive.DeprecatedParquetInputFormat' > OUTPUTFORMAT 'parquet.hive.DeprecatedParquetOutputFormat'; > > > Xuefu Zhang wrote: > OKay. These are interesting class names acutually used.
The reason I believe is that a long time ago someone named the "org.apache.hadoop.mapred" version of the LZO inputformat "Deprecated...InputFormat". That has percolated to many other projects. After some discussions with the Parquet folks we created MapredParquetInputFormat to replace DepcreatedParquetInputFormat. - Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17061/#review33473 ----------------------------------------------------------- On Feb. 4, 2014, 4:52 p.m., Brock Noland wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/17061/ > ----------------------------------------------------------- > > (Updated Feb. 4, 2014, 4:52 p.m.) > > > Review request for hive. > > > Bugs: HIVE-5783 > https://issues.apache.org/jira/browse/HIVE-5783 > > > Repository: hive-git > > > Description > ------- > > Adds native parquet support hive > > > Diffs > ----- > > data/files/parquet_create.txt PRE-CREATION > data/files/parquet_partitioned.txt PRE-CREATION > pom.xml 41f5337 > ql/pom.xml 7087a4c > ql/src/java/org/apache/hadoop/hive/ql/io/IOConstants.java PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetOutputFormat.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ProjectionPusher.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/DataWritableGroupConverter.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/DataWritableRecordConverter.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveGroupConverter.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/AbstractParquetMapInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/DeepParquetHiveMapInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveArrayInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/StandardParquetHiveMapInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetByteInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetPrimitiveInspectorFactory.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetShortInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetStringInspector.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/writable/BigDecimalWritable.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/writable/BinaryWritable.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriteSupport.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java 2bc7e86 > ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java > 13d0a56 > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f83c15d > ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 17f3552 > ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 538b2b0 > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 719b496 > ql/src/java/parquet/hive/DeprecatedParquetInputFormat.java PRE-CREATION > ql/src/java/parquet/hive/DeprecatedParquetOutputFormat.java PRE-CREATION > ql/src/java/parquet/hive/MapredParquetInputFormat.java PRE-CREATION > ql/src/java/parquet/hive/MapredParquetOutputFormat.java PRE-CREATION > ql/src/java/parquet/hive/serde/ParquetHiveSerDe.java PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestMapredParquetInputFormat.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestMapredParquetOutputFormat.java > PRE-CREATION > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetSerDe.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestAbstractParquetMapInspector.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestDeepParquetHiveMapInspector.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestParquetHiveArrayInspector.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestStandardParquetHiveMapInspector.java > PRE-CREATION > ql/src/test/queries/clientpositive/parquet_create.q PRE-CREATION > ql/src/test/queries/clientpositive/parquet_partitioned.q PRE-CREATION > ql/src/test/results/clientpositive/parquet_create.q.out PRE-CREATION > ql/src/test/results/clientpositive/parquet_partitioned.q.out PRE-CREATION > > Diff: https://reviews.apache.org/r/17061/diff/ > > > Testing > ------- > > > Thanks, > > Brock Noland > >
