> On Aug. 24, 2018, 7:49 a.m., Szabolcs Vasas wrote: > > src/test/org/apache/sqoop/testutil/ParquetFileTestUtils.java > > Lines 33 (patched) > > <https://reviews.apache.org/r/68475/diff/3/?file=2076727#file2076727line33> > > > > I think you could use org.apache.sqoop.util.ParquetReader instead of > > this new class in your tests. ParquetReader already has the capability to > > read the content of a directory and you don't need to specify the file > > name, so your test cases would get simpler.
Good catch, thank you! I have created SQOOP-3372 for introducing similar reader util classes for other file formats too as it is oiut of the scope of this change and would require a bigger refactor. - Boglarka ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68475/#review207854 ----------------------------------------------------------- On Aug. 24, 2018, 2:41 p.m., Boglarka Egyed wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68475/ > ----------------------------------------------------------- > > (Updated Aug. 24, 2018, 2:41 p.m.) > > > Review request for Sqoop, daniel voros, Fero Szabo, Nguyen Truong, and > Szabolcs Vasas. > > > Bugs: SQOOP-3363 > https://issues.apache.org/jira/browse/SQOOP-3363 > > > Repository: sqoop-trunk > > > Description > ------- > > * Added new test cases for Parquet import into S3 as it was still missing > * Added new test cases for incremental append import into S3 in Text, Avro, > Sequence and Parquet file format > * Added new test cases for incremental merge import into S3 in Text and > Parquet file format > * Updated some previously added logic in S3 util and test classes > > > Diffs > ----- > > src/java/org/apache/sqoop/util/FileSystemUtil.java > 96ec21233d6378865195008f5ab67a74370fa2ed > src/test/org/apache/sqoop/s3/TestS3AvroImport.java > e130c42104b86e854d45babc009a5f1409a74a48 > src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java > PRE-CREATION > src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java > PRE-CREATION > src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java > PRE-CREATION > src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java > PRE-CREATION > src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java > PRE-CREATION > src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java > PRE-CREATION > src/test/org/apache/sqoop/s3/TestS3ParquetImport.java PRE-CREATION > src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java > c17c1c54918df0b4d1ecbaef4e381975d72756ae > src/test/org/apache/sqoop/s3/TestS3TextImport.java > 60e2cd3025e67ecd43bdfb6b30d1b8d69a50da86 > src/test/org/apache/sqoop/testutil/AvroTestUtils.java > 04a8494a5d1d8a5020d5a3b629bbab62d3c09ffd > src/test/org/apache/sqoop/testutil/BaseSqoopTestCase.java > 1730698e80cc77395f8a296b7bf01c104533e10b > src/test/org/apache/sqoop/testutil/S3TestUtils.java > ceaff3b3a2bfd031b9772c9b43afdfa670c23718 > src/test/org/apache/sqoop/testutil/SequenceFileTestUtils.java > ad7576dbb2447423c677429f24163031a9d39b5f > src/test/org/apache/sqoop/testutil/TextFileTestUtils.java > df19cb8be7a633a6f1e1e3f9bc7d0dbc268aa90a > > > Diff: https://reviews.apache.org/r/68475/diff/4/ > > > Testing > ------- > > ant clean test -Ds3.bucket.url=<bucket-url> > -Ds3.generator.command=<credential-generator-command> > ./gradlew test -Ds3.bucket.url=<bucket-url> > -Ds3.generator.command=<credential-generator-command> > > > Thanks, > > Boglarka Egyed > >
