> On Sept. 29, 2017, 10:27 a.m., Boglarka Egyed wrote: > > Hi Chris, > > > > Thanks for your patch! > > > > I ran unit tests with your patch and TestMainframeDatasetInputFormat test > > case failed for me with the following error message: > > > > [ERROR - > > org.apache.sqoop.util.MainframeFTPClientUtils.listSequentialDatasets(MainframeFTPClientUtils.java:64)] > > MainframeDatasetPath helper class incorrectly initialised > > java.lang.NullPointerException > > at > > org.apache.sqoop.mapreduce.mainframe.MainframeDatasetPath.setMainframeDatasetType(MainframeDatasetPath.java:110) > > at > > org.apache.sqoop.mapreduce.mainframe.MainframeDatasetPath.<init>(MainframeDatasetPath.java:54) > > at > > org.apache.sqoop.util.MainframeFTPClientUtils.listSequentialDatasets(MainframeFTPClientUtils.java:61) > > at > > org.apache.sqoop.mapreduce.mainframe.MainframeDatasetInputFormat.retrieveDatasets(MainframeDatasetInputFormat.java:93) > > at > > org.apache.sqoop.mapreduce.mainframe.MainframeDatasetInputFormat.getSplits(MainframeDatasetInputFormat.java:67) > > at > > org.apache.sqoop.mapreduce.mainframe.TestMainframeDatasetInputFormat.testRetrieveDatasets(TestMainframeDatasetInputFormat.java:106) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > > at > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > > at > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > > at > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > > at > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > > at > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > > at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:535) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1182) > > at > > org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1033) > > > > Please review your changes there. Also, please avoid any reformatting of > > any file as it makes difficult to review your change. > > > > Thanks, > > Bogi
Thanks for picking this up. I have updated the code so the unit test passes. - Chris ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62653/#review186658 ----------------------------------------------------------- On Oct. 2, 2017, 11:12 p.m., Chris Teoh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62653/ > ----------------------------------------------------------- > > (Updated Oct. 2, 2017, 11:12 p.m.) > > > Review request for Sqoop. > > > Bugs: SQOOP-3225 > https://issues.apache.org/jira/browse/SQOOP-3225 > > > Repository: sqoop-trunk > > > Description > ------- > > There were some cases where the FTP listing parser was not used for mainframe > datasets, only the default was used. This caused some datasets to not be seen > by Sqoop as the default parser couldn't find them. This patch addresses this > behaviour where it is used to parse FTP listings for sequential and gdg > datasets on disk and tape, only partitioned datasets are excluded as their > FTP listing looks very different and is handled by the default FTP listing > parser. > > > Diffs > ----- > > src/java/org/apache/sqoop/util/MainframeFTPClientUtils.java f61b9838 > src/test/org/apache/sqoop/util/TestMainframeFTPClientUtils.java d87c75df > > > Diff: https://reviews.apache.org/r/62653/diff/5/ > > > Testing > ------- > > Unit testing. > Functional testing on a real mainframe with a previous combined patch. > > > Thanks, > > Chris Teoh > >
