----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65478/#review196867 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/ParquetDataColumnReaderFactory.java Lines 114 (patched) <https://reviews.apache.org/r/65478/#comment276743> nit, It should probably say "Implementation is consistent with .." Same with TypesFromInt64PageReader and TypesFromFloatPageReader below. Thanks ql/src/test/queries/clientpositive/schema_evol_par_vec_table.q Lines 9 (patched) <https://reviews.apache.org/r/65478/#comment276745> Does this test work when dictionary encoding is both enabled/disabled? You can change parquet dictionary encoding by setting table property "parquet.enable.dictionary"="true" or "false". Can you please modify the test to include that as well? ql/src/test/queries/clientpositive/schema_evol_par_vec_table.q Lines 49 (patched) <https://reviews.apache.org/r/65478/#comment276742> I think this comment can be removed now so that there is no confusion in the future. ql/src/test/queries/clientpositive/schema_evol_par_vec_table.q Lines 71 (patched) <https://reviews.apache.org/r/65478/#comment276744> Can you please test using timestamps as well. Specifically, the following should work. drop table test_alter2; create table test_alter2 (ts timestamp) stored as parquet; insert into test_alter2 values ('2018-01-01 13:14:15.123456'), ('2018-01-02 14:15:16.123456'), ('2018-01-03 16:17:18.123456'); select * from test_alter2; alter table test_alter2 replace columns (ts string); select * from test_alter2; drop table test_alter2; create table test_alter2 (ts timestamp) stored as parquet; insert into test_alter2 values ('2018-01-01 13:14:15.123456'), ('2018-01-02 14:15:16.123456'), ('2018-01-03 16:17:18.123456'); select * from test_alter2; alter table test_alter2 replace columns (ts varchar(19)); -- this should truncate the microseconds select * from test_alter2; drop table test_alter2; create table test_alter2 (ts timestamp) stored as parquet; insert into test_alter2 values ('2018-01-01 13:14:15.123456'), ('2018-01-02 14:15:16.123456'), ('2018-01-03 16:17:18.123456'); select * from test_alter2; alter table test_alter2 replace columns (ts char(25); select * from test_alter2; - Vihang Karajgaonkar On Feb. 5, 2018, 8:46 a.m., cheng xu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/65478/ > ----------------------------------------------------------- > > (Updated Feb. 5, 2018, 8:46 a.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > ------- > > VectorizedParquetReader throws an exception when trying to reading from a > parquet table on which new columns are added. > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/BaseVectorizedColumnReader.java > 907a9b8 > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/ParquetDataColumnReaderFactory.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedDummyColumnReader.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedParquetRecordReader.java > 08ac57b > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestVectorizedColumnReader.java > 9e414dc > > ql/src/test/org/apache/hadoop/hive/ql/io/parquet/VectorizedColumnReaderTestBase.java > 5d3ebd6 > ql/src/test/queries/clientpositive/schema_evol_par_vec_table.q PRE-CREATION > ql/src/test/results/clientpositive/schema_evol_par_vec_table.q.out > PRE-CREATION > > > Diff: https://reviews.apache.org/r/65478/diff/2/ > > > Testing > ------- > > Newly added UT passed and qtest passed locally. > > > Thanks, > > cheng xu > >
