Robert, thanks for fixing the MacOS build! Building on my Ubuntu VM is still failing though. Will send you the Maven output.
Cheers, Fabian 2015-01-26 15:49 GMT+01:00 Robert Metzger <rmetz...@apache.org>: > With the help of Fabian and Ufuk, I was able to identify the issue for OS > X. I already pushed the fix (see JIRA: > https://issues.apache.org/jira/browse/FLINK-1453). > > Fabian is currently running the tests on the Ubuntu VM again. Lets see if > that one is fixed as well or if its a different issue. > > On Sun, Jan 25, 2015 at 1:10 PM, Fabian Hueske <fhue...@gmail.com> wrote: > > > Its a 14.04 Ubuntu Server VM with 6GB Ram and a 6GB swap partition. > > > > 2015-01-25 13:00 GMT+01:00 Robert Metzger <rmetz...@apache.org>: > > > > > Can you give me the exact version of the Ubuntu VM? > > > Do you have a swapfile on the Ubuntu VM? The tests might currently > > allocate > > > up to 8 GB of memory, so maybe your VM is starting to swap and thats > why > > > the test runs out of the timeout. (That would not explain the failure > on > > > OSX) > > > > > > I've tried it with a 3 GB (w/o swap file) 14.04 Ubuntu and only one > test > > > failed. > > > > > > On Sun, Jan 25, 2015 at 12:55 PM, Fabian Hueske <fhue...@gmail.com> > > wrote: > > > > > > > My VM is configured with 6GB and the OS X host has 16GB. In both > setups > > > the > > > > error was identical (with and without cleared .m2). > > > > If the tests pass on a 3GB Travis host, I doubt that my errors are > > caused > > > > by lack of memory. > > > > > > > > 2015-01-25 12:28 GMT+01:00 Robert Metzger <rmetz...@apache.org>: > > > > > > > > > I also had errors when running the tests in an Ubuntu virtual > > machine, > > > > > caused by limited memory resources. > > > > > > > > > > Do you think its okay to assume that we have at least 3 GB of main > > > memory > > > > > available for running the tests? Thats what we have with Travis. > The > > > YARN > > > > > tests are very memory intensive, because YARN itself has a bunch of > > > > > services (resource manager, 2 node managers) and we allocate some > > > > > containers there, which all need memory. > > > > > > > > > > The main problem is that the amount of memory per node manager is > > > > > hard-coded to 4GB. I've already opened an issue in the Hadoop > project > > > and > > > > > will fix the issue there... > > > > > > > > > > But I think I have some ideas how to work around the memory > > limitations > > > > for > > > > > now. > > > > > > > > > > > > > > > > > > > > On Sat, Jan 24, 2015 at 11:47 PM, Vasiliki Kalavri < > > > > > vasilikikala...@gmail.com> wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > "mvn clean verify" fails for me on Ubuntu with deleted .m2 > > > repository. > > > > > > I'm getting the following: > > > > > > > > > > > > Results : > > > > > > > > > > > > Failed tests: > > > > > > > > > YARNSessionFIFOITCase.setup:56->YarnTestBase.startYARNWithConfig:249 > > > > > null > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > YARNSessionCapacitySchedulerITCase.setup:42->YarnTestBase.startYARNWithConfig:249 > > > > > > null > > > > > > > > > > > > Tests run: 2, Failures: 2, Errors: 0, Skipped: 0 > > > > > > > > > > > > -V. > > > > > > > > > > > > On 24 January 2015 at 23:03, Fabian Hueske <fhue...@gmail.com> > > > wrote: > > > > > > > > > > > > > The build fails also after the .m2 repository was deleted. > > > > > > > > > > > > > > Does anybody else have this problem? > > > > > > > > > > > > > > 2015-01-24 21:31 GMT+01:00 Stephan Ewen <se...@apache.org>: > > > > > > > > > > > > > > > Is this reproducible on a machine when you delete the > > > > .m2/repository > > > > > > > > directory (local maven cache) ? > > > > > > > > > > > > > > > > (I currently cannot try that because I am behind a rather > > > > > low-bandwith > > > > > > > > connection and would take very long to re-download all > > dependency > > > > > > > > artifacts) > > > > > > > > > > > > > > > > On Sat, Jan 24, 2015 at 5:54 AM, Fabian Hueske < > > > fhue...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > I just tried to build ("mvn clean install") on a fresh > Ubuntu > > > VM. > > > > > > Fails > > > > > > > > > with the same exception as natively on MacOS. > > > > > > > > > Something strange is going on... > > > > > > > > > > > > > > > > > > 2015-01-24 11:19 GMT+01:00 Fabian Hueske < > fhue...@gmail.com > > >: > > > > > > > > > > > > > > > > > > > Thanks Robert! Sounds indeed like an environment problem. > > > > > > > > > > Will run the tests again and send you the output. > > > > > > > > > > > > > > > > > > > > 2015-01-24 11:11 GMT+01:00 Robert Metzger < > > > rmetz...@apache.org > > > > >: > > > > > > > > > > > > > > > > > > > >> Okay, the tests have finished on my local machine, and > > they > > > > > > passed. > > > > > > > So > > > > > > > > > it > > > > > > > > > >> looks like an environment specific issue. > > > > > > > > > >> Maybe the log helps me already to figure out whats the > > > issue. > > > > > > > > > >> We should make sure that our tests are passing on all > > > > platforms > > > > > ;) > > > > > > > > > >> > > > > > > > > > >> On Sat, Jan 24, 2015 at 11:06 AM, Robert Metzger < > > > > > > > rmetz...@apache.org > > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > >> > > > > > > > > > >> > Hi, > > > > > > > > > >> > > > > > > > > > > >> > the tests are passing on travis. Maybe its a issue > with > > > your > > > > > > > > > >> environment. > > > > > > > > > >> > I'm currently running the tests on my machine as well, > > > just > > > > to > > > > > > > make > > > > > > > > > >> sure. > > > > > > > > > >> > I haven't ran the tests on OS X, maybe that's causing > > the > > > > > > issues. > > > > > > > > > >> > > > > > > > > > > >> > Can you send me (privately) the full output of the > > tests? > > > > > > > > > >> > > > > > > > > > > >> > Best, > > > > > > > > > >> > Robert > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > On Sat, Jan 24, 2015 at 11:00 AM, Fabian Hueske < > > > > > > > fhue...@gmail.com> > > > > > > > > > >> wrote: > > > > > > > > > >> > > > > > > > > > > >> >> Hi Henry, > > > > > > > > > >> >> > > > > > > > > > >> >> running "mvn -DskipTests clean install" before "mvn > > clean > > > > > > > install" > > > > > > > > > did > > > > > > > > > >> not > > > > > > > > > >> >> fix the build for me. > > > > > > > > > >> >> The failing tests are also integration tests > (*ITCase) > > > > which > > > > > > are > > > > > > > > only > > > > > > > > > >> >> executed in Maven's verify phase which is not > triggered > > > if > > > > > you > > > > > > > run > > > > > > > > > "mvn > > > > > > > > > >> >> clean test". > > > > > > > > > >> >> If I run "mvn test" without "mvn install" it fails > for > > me > > > > as > > > > > > well > > > > > > > > > with > > > > > > > > > >> the > > > > > > > > > >> >> error you posted. > > > > > > > > > >> >> > > > > > > > > > >> >> So there seem to be at least two build issues with > the > > > > > current > > > > > > > > > master. > > > > > > > > > >> >> > > > > > > > > > >> >> 2015-01-24 1:47 GMT+01:00 Henry Saputra < > > > > > > henry.sapu...@gmail.com > > > > > > > >: > > > > > > > > > >> >> > > > > > > > > > >> >> > Hmm, I think there could be some weird dependencies > > to > > > > get > > > > > > the > > > > > > > > > Flink > > > > > > > > > >> >> > YARN uber jar. > > > > > > > > > >> >> > > > > > > > > > > >> >> > If you do "mvn clean install -DskipTests" then call > > > "mvn > > > > > > test" > > > > > > > > all > > > > > > > > > >> the > > > > > > > > > >> >> > tests passed. > > > > > > > > > >> >> > > > > > > > > > > >> >> > But if you directly call "mvn clean test" then you > > see > > > > the > > > > > > > stack > > > > > > > > I > > > > > > > > > >> >> > have seen before. > > > > > > > > > >> >> > > > > > > > > > > >> >> > - Henry > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > > >> >> > On Fri, Jan 23, 2015 at 3:35 PM, Henry Saputra < > > > > > > > > > >> henry.sapu...@gmail.com > > > > > > > > > >> >> > > > > > > > > > > >> >> > wrote: > > > > > > > > > >> >> > > Did not see that trace but do see this: > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > ------------------------------------------------------- > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > T E S T S > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > ------------------------------------------------------- > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > Running org.apache.flink.yarn.UtilsTest > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > log4j:WARN No such property [append] in > > > > > > > > > >> >> org.apache.log4j.ConsoleAppender. > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, > > > Time > > > > > > > elapsed: > > > > > > > > > >> 0.476 > > > > > > > > > >> >> > > sec <<< FAILURE! - in > > org.apache.flink.yarn.UtilsTest > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > testUberjarLocator(org.apache.flink.yarn.UtilsTest) > > > > Time > > > > > > > > > elapsed: > > > > > > > > > >> >> > > 0.405 sec <<< FAILURE! > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > java.lang.AssertionError: null > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > at org.junit.Assert.fail(Assert.java:86) > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > at org.junit.Assert.assertTrue(Assert.java:41) > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > at > org.junit.Assert.assertNotNull(Assert.java:621) > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > at > org.junit.Assert.assertNotNull(Assert.java:631) > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > at > > > > > > > > > >> >> > > > > > > > > > > > > org.apache.flink.yarn.UtilsTest.testUberjarLocator(UtilsTest.java:32) > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > Results : > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > Failed tests: > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > UtilsTest.testUberjarLocator:32 null > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > - Henry > > > > > > > > > >> >> > > > > > > > > > > > >> >> > > On Fri, Jan 23, 2015 at 2:16 PM, Fabian Hueske < > > > > > > > > > fhue...@gmail.com> > > > > > > > > > >> >> > wrote: > > > > > > > > > >> >> > >> Hi all, > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> I tried to build the current master (mvn clean > > > > install) > > > > > > and > > > > > > > > some > > > > > > > > > >> >> tests > > > > > > > > > >> >> > in > > > > > > > > > >> >> > >> the flink-yarn-tests module fail: > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> Failed tests: > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > YARNSessionCapacitySchedulerITCase.testClientStartup:50->YarnTestBase.runWithArgs:314 > > > > > > > > > >> >> > >> During the timeout period of 60 seconds the > > expected > > > > > > string > > > > > > > > did > > > > > > > > > >> not > > > > > > > > > >> >> > show up > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > > > > > > > > > > > > > > > YARNSessionCapacitySchedulerITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > > >> >> > >> There is at least one application on the cluster > > is > > > > not > > > > > > > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > YARNSessionFIFOITCase.perJobYarnCluster:184->YarnTestBase.runWithArgs:314 > > > > > > > > > >> >> > >> During the timeout period of 60 seconds the > > expected > > > > > > string > > > > > > > > did > > > > > > > > > >> not > > > > > > > > > >> >> > show up > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > There > > > > > > > > > >> is > > > > > > > > > >> >> at > > > > > > > > > >> >> > >> least one application on the cluster is not > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > There > > > > > > > > > >> is > > > > > > > > > >> >> at > > > > > > > > > >> >> > >> least one application on the cluster is not > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > There > > > > > > > > > >> is > > > > > > > > > >> >> at > > > > > > > > > >> >> > >> least one application on the cluster is not > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > There > > > > > > > > > >> is > > > > > > > > > >> >> at > > > > > > > > > >> >> > >> least one application on the cluster is not > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > There > > > > > > > > > >> is > > > > > > > > > >> >> at > > > > > > > > > >> >> > >> least one application on the cluster is not > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > There > > > > > > > > > >> is > > > > > > > > > >> >> at > > > > > > > > > >> >> > >> least one application on the cluster is not > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146 > > > > > > > > There > > > > > > > > > >> is > > > > > > > > > >> >> at > > > > > > > > > >> >> > >> least one application on the cluster is not > > finished > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> Tests run: 10, Failures: 10, Errors: 0, > Skipped: 0 > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> Anybody else got this problem? > > > > > > > > > >> >> > >> > > > > > > > > > >> >> > >> Cheers, Fabian > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
