> On March 23, 2018, 4:33 p.m., Szabolcs Vasas wrote: > > Hi Anna, > > > > Awesome, thanks for the quick update. > > > > I have tried running the third party tests today, it was successful, > > however there is a small problem with the cleanup. > > If I run ./gradlew test first and then the third party tests then > > org.apache.sqoop.manager.sqlserver.SQLServerHiveImportTest.testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema > > fails. This test case succeeds if I run the third party tests only after a > > ./gradlew clean. I think the problem is that SQLServerHiveImportTest > > extends HiveImportTest which does not do a proper Hive metastore database > > cleanup, so a table which has to be created by the test already exists. > > Anyway, changing the Hive metastore DB to an in-memory DB solves the > > problem so I think can incorporate it into your patch > > (https://github.com/szvasas/sqoop/blob/gradletest2/testdata/hcatalog/conf/hive-site.xml#L37). > > I am not sure why but I did not experience this issue when running the > > tests with ant. > > > > An interesting thing I have noticed is that when I try to execute one test > > case Gradle seems to start up lots of JVMs even after the test has > > successfully finished. > > > > ./gradlew test --tests org.apache.sqoop.hive.TestHiveImport > > :test > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[10180]: Class JavaLaunchHelper is implemented in both > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java > > and > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib. > > One of the two will be used. Which one is undefined. > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[10183]: Class JavaLaunchHelper is implemented in both > > ... > > ... > > ... > > ... > > > > org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testDecimalMapColumnHive PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelims PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testGenerateOnly STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testGenerateOnly PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testHiveExitFails STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testHiveExitFails PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testDate STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testDate PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement > > STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testFieldWithHiveDelimsReplacement > > PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCustomDelimiters PASSED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveDropAndReplaceOptionValidation STARTED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveDropAndReplaceOptionValidation PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOverwriteHiveImport PASSED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema STARTED > > > > org.apache.sqoop.hive.TestHiveImport > > > testHiveImportAsParquetWhenTableExistsWithIncompatibleSchema PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testCreateOnlyHiveImport PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testAppendHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImport PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testNormalHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testImportWithBadPartitionKey PASSED > > > > org.apache.sqoop.hive.TestHiveImport > > > testCreateOverwriteHiveImportAsParquet STARTED > > > > org.apache.sqoop.hive.TestHiveImport > > > testCreateOverwriteHiveImportAsParquet PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testImportHiveWithPartitions PASSED > > > > org.apache.sqoop.hive.TestHiveImport > testNumeric STARTED > > > > org.apache.sqoop.hive.TestHiveImport > testNumeric PASSED > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[11118]: Class JavaLaunchHelper is implemented in both > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java > > and > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib. > > One of the two will be used. Which one is undefined. > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; > > support was removed in 8.0 > > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option > > MaxPermSize=512m; support was removed in 8.0 > > objc[11119]: Class JavaLaunchHelper is implemented in both > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/bin/java > > and > > /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre/lib/libinstrument.dylib. > > One of the two will be used. Which one is undefined. > > ... > > ... > > > > BUILD SUCCESSFUL > > > > Total time: 2 mins 54.654 secs > > > > > > I am not sure what happens here, my guess is that a new JVM is started for > > every test case even if they are not executed. Have you experienced this > > problem too?
I've had similar issues running with the --tests option, I usually use -Dtest.single, unfortunately if you have a buildSrc folder in your project that seems to mess everything up for test.single, so I can rename that directory to solve this for now. Oddly enough it doesn't affect thirdParty.single. Maybe in the long run we can also try to make test runs independent, so we don't have to use forkevery 1... - Anna ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66067/#review199896 ----------------------------------------------------------- On March 23, 2018, 10:28 a.m., Anna Szonyi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66067/ > ----------------------------------------------------------- > > (Updated March 23, 2018, 10:28 a.m.) > > > Review request for Sqoop. > > > Bugs: Sqoop-3052 > https://issues.apache.org/jira/browse/Sqoop-3052 > > > Repository: sqoop-trunk > > > Description > ------- > > SQOOP-3052: Introduce gradle based build for Sqoop to make it more developer > friendly / open > > > Diffs > ----- > > .gitignore 68cbe28 > COMPILING.txt 3b82250 > build.gradle PRE-CREATION > buildSrc/customUnixStartScript.txt PRE-CREATION > buildSrc/customWindowsStartScript.txt PRE-CREATION > buildSrc/sqoop-package.gradle PRE-CREATION > buildSrc/sqoop-version-gen.gradle PRE-CREATION > config/checkstyle/checkstyle-java-header.txt PRE-CREATION > config/checkstyle/checkstyle-noframes.xsl PRE-CREATION > config/checkstyle/checkstyle.xml PRE-CREATION > gradle.properties PRE-CREATION > gradle/wrapper/gradle-wrapper.jar PRE-CREATION > gradle/wrapper/gradle-wrapper.properties PRE-CREATION > gradlew PRE-CREATION > gradlew.bat PRE-CREATION > settings.gradle PRE-CREATION > > > Diff: https://reviews.apache.org/r/66067/diff/6/ > > > Testing > ------- > > ran all new tasks, except for internal maven publishing > > Notes: > - To try it out you can call ./gradlew tasks --all to see all the tasks and > compare them to current tasks/artifacts. > - Replaced cobertura with jacoco, as it's easier/cleaner to configure, easier > to combine all test results into a single report. > - Generated pom.xml now has correct dependencies/versions > - Script generation is currently hardcoded and not based on sqoop help, as > previously - though added the possiblity of hooking it in later > > > Thanks, > > Anna Szonyi > >
