I also had errors when running the tests in an Ubuntu virtual machine,
caused by limited memory resources.
Do you think its okay to assume that we have at least 3 GB of main memory
available for running the tests? Thats what we have with Travis. The YARN
tests are very memory intensive, because YARN itself has a bunch of
services (resource manager, 2 node managers) and we allocate some
containers there, which all need memory.
The main problem is that the amount of memory per node manager is
hard-coded to 4GB. I've already opened an issue in the Hadoop project and
will fix the issue there...
But I think I have some ideas how to work around the memory limitations for
now.
On Sat, Jan 24, 2015 at 11:47 PM, Vasiliki Kalavri <
vasilikikala...@gmail.com> wrote:
> Hi,
>
> "mvn clean verify" fails for me on Ubuntu with deleted .m2 repository.
> I'm getting the following:
>
> Results :
>
> Failed tests:
> YARNSessionFIFOITCase.setup:56->YarnTestBase.startYARNWithConfig:249 null
>
>
> YARNSessionCapacitySchedulerITCase.setup:42->YarnTestBase.startYARNWithConfig:249
> null
>
> Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
>
> -V.
>
> On 24 January 2015 at 23:03, Fabian Hueske <fhue...@gmail.com> wrote:
>
> > The build fails also after the .m2 repository was deleted.
> >
> > Does anybody else have this problem?
> >
> > 2015-01-24 21:31 GMT+01:00 Stephan Ewen <se...@apache.org>:
> >
> > > Is this reproducible on a machine when you delete the .m2/repository
> > > directory (local maven cache) ?
> > >
> > > (I currently cannot try that because I am behind a rather low-bandwith
> > > connection and would take very long to re-download all dependency
> > > artifacts)
> > >
> > > On Sat, Jan 24, 2015 at 5:54 AM, Fabian Hueske <fhue...@gmail.com>
> > wrote:
> > >
> > > > I just tried to build ("mvn clean install") on a fresh Ubuntu VM.
> Fails
> > > > with the same exception as natively on MacOS.
> > > > Something strange is going on...
> > > >
> > > > 2015-01-24 11:19 GMT+01:00 Fabian Hueske <fhue...@gmail.com>:
> > > >
> > > > > Thanks Robert! Sounds indeed like an environment problem.
> > > > > Will run the tests again and send you the output.
> > > > >
> > > > > 2015-01-24 11:11 GMT+01:00 Robert Metzger <rmetz...@apache.org>:
> > > > >
> > > > >> Okay, the tests have finished on my local machine, and they
> passed.
> > So
> > > > it
> > > > >> looks like an environment specific issue.
> > > > >> Maybe the log helps me already to figure out whats the issue.
> > > > >> We should make sure that our tests are passing on all platforms ;)
> > > > >>
> > > > >> On Sat, Jan 24, 2015 at 11:06 AM, Robert Metzger <
> > rmetz...@apache.org
> > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi,
> > > > >> >
> > > > >> > the tests are passing on travis. Maybe its a issue with your
> > > > >> environment.
> > > > >> > I'm currently running the tests on my machine as well, just to
> > make
> > > > >> sure.
> > > > >> > I haven't ran the tests on OS X, maybe that's causing the
> issues.
> > > > >> >
> > > > >> > Can you send me (privately) the full output of the tests?
> > > > >> >
> > > > >> > Best,
> > > > >> > Robert
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Sat, Jan 24, 2015 at 11:00 AM, Fabian Hueske <
> > fhue...@gmail.com>
> > > > >> wrote:
> > > > >> >
> > > > >> >> Hi Henry,
> > > > >> >>
> > > > >> >> running "mvn -DskipTests clean install" before "mvn clean
> > install"
> > > > did
> > > > >> not
> > > > >> >> fix the build for me.
> > > > >> >> The failing tests are also integration tests (*ITCase) which
> are
> > > only
> > > > >> >> executed in Maven's verify phase which is not triggered if you
> > run
> > > > "mvn
> > > > >> >> clean test".
> > > > >> >> If I run "mvn test" without "mvn install" it fails for me as
> well
> > > > with
> > > > >> the
> > > > >> >> error you posted.
> > > > >> >>
> > > > >> >> So there seem to be at least two build issues with the current
> > > > master.
> > > > >> >>
> > > > >> >> 2015-01-24 1:47 GMT+01:00 Henry Saputra <
> henry.sapu...@gmail.com
> > >:
> > > > >> >>
> > > > >> >> > Hmm, I think there could be some weird dependencies to get
> the
> > > > Flink
> > > > >> >> > YARN uber jar.
> > > > >> >> >
> > > > >> >> > If you do "mvn clean install -DskipTests" then call "mvn
> test"
> > > all
> > > > >> the
> > > > >> >> > tests passed.
> > > > >> >> >
> > > > >> >> > But if you directly call "mvn clean test" then you see the
> > stack
> > > I
> > > > >> >> > have seen before.
> > > > >> >> >
> > > > >> >> > - Henry
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > On Fri, Jan 23, 2015 at 3:35 PM, Henry Saputra <
> > > > >> henry.sapu...@gmail.com
> > > > >> >> >
> > > > >> >> > wrote:
> > > > >> >> > > Did not see that trace but do see this:
> > > > >> >> > >
> > > > >> >> > > -------------------------------------------------------
> > > > >> >> > >
> > > > >> >> > > T E S T S
> > > > >> >> > >
> > > > >> >> > > -------------------------------------------------------
> > > > >> >> > >
> > > > >> >> > > Running org.apache.flink.yarn.UtilsTest
> > > > >> >> > >
> > > > >> >> > > log4j:WARN No such property [append] in
> > > > >> >> org.apache.log4j.ConsoleAppender.
> > > > >> >> > >
> > > > >> >> > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
> > elapsed:
> > > > >> 0.476
> > > > >> >> > > sec <<< FAILURE! - in org.apache.flink.yarn.UtilsTest
> > > > >> >> > >
> > > > >> >> > > testUberjarLocator(org.apache.flink.yarn.UtilsTest) Time
> > > > elapsed:
> > > > >> >> > > 0.405 sec <<< FAILURE!
> > > > >> >> > >
> > > > >> >> > > java.lang.AssertionError: null
> > > > >> >> > >
> > > > >> >> > > at org.junit.Assert.fail(Assert.java:86)
> > > > >> >> > >
> > > > >> >> > > at org.junit.Assert.assertTrue(Assert.java:41)
> > > > >> >> > >
> > > > >> >> > > at org.junit.Assert.assertNotNull(Assert.java:621)
> > > > >> >> > >
> > > > >> >> > > at org.junit.Assert.assertNotNull(Assert.java:631)
> > > > >> >> > >
> > > > >> >> > > at
> > > > >> >>
> > > org.apache.flink.yarn.UtilsTest.testUberjarLocator(UtilsTest.java:32)
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > Results :
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > Failed tests:
> > > > >> >> > >
> > > > >> >> > > UtilsTest.testUberjarLocator:32 null
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > - Henry
> > > > >> >> > >
> > > > >> >> > > On Fri, Jan 23, 2015 at 2:16 PM, Fabian Hueske <
> > > > fhue...@gmail.com>
> > > > >> >> > wrote:
> > > > >> >> > >> Hi all,
> > > > >> >> > >>
> > > > >> >> > >> I tried to build the current master (mvn clean install)
> and
> > > some
> > > > >> >> tests
> > > > >> >> > in
> > > > >> >> > >> the flink-yarn-tests module fail:
> > > > >> >> > >>
> > > > >> >> > >> Failed tests:
> > > > >> >> > >>
> > > > >> >> > >>
> > > > >> >> > >>
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> YARNSessionCapacitySchedulerITCase.testClientStartup:50->YarnTestBase.runWithArgs:314
> > > > >> >> > >> During the timeout period of 60 seconds the expected
> string
> > > did
> > > > >> not
> > > > >> >> > show up
> > > > >> >> > >>
> > > > >> >> > >>
> > > > >> >>
> > > > YARNSessionCapacitySchedulerITCase>YarnTestBase.checkClusterEmpty:146
> > > > >> >> > >> There is at least one application on the cluster is not
> > > finished
> > > > >> >> > >>
> > > > >> >> > >>
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > YARNSessionFIFOITCase.perJobYarnCluster:184->YarnTestBase.runWithArgs:314
> > > > >> >> > >> During the timeout period of 60 seconds the expected
> string
> > > did
> > > > >> not
> > > > >> >> > show up
> > > > >> >> > >>
> > > > >> >> > >> YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > There
> > > > >> is
> > > > >> >> at
> > > > >> >> > >> least one application on the cluster is not finished
> > > > >> >> > >>
> > > > >> >> > >> YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > There
> > > > >> is
> > > > >> >> at
> > > > >> >> > >> least one application on the cluster is not finished
> > > > >> >> > >>
> > > > >> >> > >> YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > There
> > > > >> is
> > > > >> >> at
> > > > >> >> > >> least one application on the cluster is not finished
> > > > >> >> > >>
> > > > >> >> > >> YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > There
> > > > >> is
> > > > >> >> at
> > > > >> >> > >> least one application on the cluster is not finished
> > > > >> >> > >>
> > > > >> >> > >> YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > There
> > > > >> is
> > > > >> >> at
> > > > >> >> > >> least one application on the cluster is not finished
> > > > >> >> > >>
> > > > >> >> > >> YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > There
> > > > >> is
> > > > >> >> at
> > > > >> >> > >> least one application on the cluster is not finished
> > > > >> >> > >>
> > > > >> >> > >> YARNSessionFIFOITCase>YarnTestBase.checkClusterEmpty:146
> > > There
> > > > >> is
> > > > >> >> at
> > > > >> >> > >> least one application on the cluster is not finished
> > > > >> >> > >>
> > > > >> >> > >>
> > > > >> >> > >> Tests run: 10, Failures: 10, Errors: 0, Skipped: 0
> > > > >> >> > >>
> > > > >> >> > >> Anybody else got this problem?
> > > > >> >> > >>
> > > > >> >> > >> Cheers, Fabian
> > > > >> >> >
> > > > >> >>
> > > > >> >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
