----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35950/#review89860 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java (line 63) <https://reviews.apache.org/r/35950/#comment142719> shall we keep this as 'final'? Nice refactor. The change looks good. Thanks - Dong Chen On June 28, 2015, 12:29 a.m., Sergio Pena wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/35950/ > ----------------------------------------------------------- > > (Updated June 28, 2015, 12:29 a.m.) > > > Review request for hive, Ryan Blue, cheng xu, and Dong Chen. > > > Bugs: HIVE-11131 > https://issues.apache.org/jira/browse/HIVE-11131 > > > Repository: hive-git > > > Description > ------- > > Implemented data type writers that will be created before the first Hive row > is written to Parquet. These writers contain information about object > inspectors and schema of a specific data type, and calls the specific > addXXXX() method used by Parquet for each data type. > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java > c195c3ec3ddae19bf255fc2c9633f8bf4390f428 > > Diff: https://reviews.apache.org/r/35950/diff/ > > > Testing > ------- > > Tests from TestDataWritableWriter run OK. > > I run other tests with micro-becnhmarks, and I got some better results from > this new implemntation: > > Using repeated rows across the file, this is the throughput increase using 1 > million records: > > bigint boolean double float int string > 7.598 7.491 7.488 7.588 7.53 0.270 (before) > 10.137 11.511 10.155 10.297 10.242 0.286 (after) > > Using random rows across the file, the is the throughput increase using 1 > million records: > > bigint boolean double float int string > 5.268 7.723 4.107 4.173 4.729 0.20 (before) > 6.236 10.466 5.944 4.749 5.234 0.22 (after) > > > Thanks, > > Sergio Pena > >
