A problem is that all the BlobCache/Server tests don't extend the `TestLogger` class which logs the names of the different tests. Maybe we should that add as well.
On Fri, Jan 15, 2016 at 3:32 PM, Prez Cannady <[email protected]> wrote: > I’ve included the full log of a test run against flink-runtime here: > > https://gist.github.com/revprez/9d38749f6672d1bef9f1 < > https://gist.github.com/revprez/9d38749f6672d1bef9f1> > > Specific issue observed starts at 691, but you’ll see exceptions both > before and after. As per Metzger's observation, I think BlobCache is > simply testing for some previous BlobServer and after its retries run out a > new BlobServer is created to handle requests. All the > FileNotFoundExceptions occur within a cluster of the test run and do not > emerge again. I’m not clear, however, which test is associated with these > printouts. The stack trace doesn’t go any higher than > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103). > > There are other exceptions > > Prez Cannady > p: 617 500 3378 > e: [email protected] <mailto:[email protected]> > GH: https://github.com/opencorrelate <https://github.com/opencorrelate> > LI: https://www.linkedin.com/in/revprez < > https://www.linkedin.com/in/revprez> > > > > > > > > > > > On Jan 15, 2016, at 6:28 AM, Stephan Ewen <[email protected]> wrote: > > > > Hi! > > > > I cannot access the gist. > > > > There are some tests that check proper error reporting in that case. > > Let's make sure there is nothing funky in the sense that this occurs also > > outside where this is tested for and goes unrecognized by the tests. > > > > Can you share more of the log? > > > > Thanks, > > Stephan > > > > > > On Fri, Jan 15, 2016 at 9:52 AM, Robert Metzger <[email protected]> > wrote: > > > >> I think the exceptions are fine. We have many tests which also check > >> certain error conditions. I guess the exceptions are expected by the > tests. > >> > >> On Fri, Jan 15, 2016 at 1:20 AM, Prez Cannady < > [email protected]> > >> wrote: > >> > >>> My bad. Gist with exception is here: > >>> > >>> https://gist.github.com/revprez/5a730a45674f0fc6e52b < > >>> https://gist.github.com/revprez/5a730a45674f0fc6e52b> > >>> > >>> Prez Cannady > >>> p: 617 500 3378 > >>> e: [email protected] <mailto:[email protected]> > >>> GH: https://github.com/opencorrelate <https://github.com/opencorrelate > > > >>> LI: https://www.linkedin.com/in/revprez < > >>> https://www.linkedin.com/in/revprez> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>>> On Jan 14, 2016, at 7:16 PM, Prez Cannady <[email protected]> > >>> wrote: > >>>> > >>>> Steps > >>>> > >>>> Forked, cut bug fix branch off of release–0.10. > >>>> One line change to fix class loader issue. > >>>> Ran mvn clean test in flink-runtime. All 894 tests pass. > >>>> Ran ./tools/change-scala-version 2.11 in the root. > >>>> Ran mvn clean test -Dscala.version=2.11.7 in flink-runtime. > >>>> Expected result > >>>> > >>>> All 894 tests pass. without incident > >>>> > >>>> Actual result > >>>> > >>>> All 894 tests actually did pass, but frequently seeing the exception > >>> below in the log (with context, and also available as a gist: > >>>> > >>>> Discussion: Not sure if this is an issue, but figured I’d ask before > >>> submitting my pull request. > >>>> > >>>> 6 > 19:06:15,250 INFO org.apache.flink.runtime.blob.BlobCache > >>> - Downloading e52e6f94e5afa6502299cf982d31105111e1d9a4 from > >>> localhost/127.0.0.1:61157 (retry 5) > >>>> 6 > 19:06:15,251 DEBUG org.apache.flink.runtime.blob.BlobClient > >>> - GET content addressable BLOB > >>> e52e6f94e5afa6502299cf982d31105111e1d9a4 from /127.0.0.1:61170 > >>>> 6 > 19:06:15,251 ERROR org.apache.flink.runtime.blob.BlobCache > >>> - Failed to fetch BLOB > >>> e52e6f94e5afa6502299cf982d31105111e1d9a4 from localhost/ > 127.0.0.1:61157 > >>> and store it under > >>> > >> > /var/folders/68/82g3zl512nj15b82m5f4nhb40000gn/T/blobStore-dee0aeaa-8199-4024-ba44-4e39154aa8e8/cache/blob_e52e6f94e5afa6502299cf982d31105111e1d9a4 > >>> No retries left. > >>>> java.io.FileNotFoundException: > >>> > >> > /var/folders/68/82g3zl512nj15b82m5f4nhb40000gn/T/blobStore-dee0aeaa-8199-4024-ba44-4e39154aa8e8/cache/blob_e52e6f94e5afa6502299cf982d31105111e1d9a4 > >>> (Permission denied) > >>>> at java.io.FileOutputStream.open0(Native Method) > >>>> at java.io.FileOutputStream.open(FileOutputStream.java:270) > >>>> at java.io.FileOutputStream.<init>(FileOutputStream.java:213) > >>>> at java.io.FileOutputStream.<init>(FileOutputStream.java:162) > >>>> at > >>> org.apache.flink.runtime.blob.BlobCache.getURL(BlobCache.java:126) > >>>> at > >>> > >> > org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager.registerReferenceToBlobKeyAndGetURL(BlobLibraryCacheManager.java:245) > >>>> at > >>> > >> > org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager.registerTask(BlobLibraryCacheManager.java:114) > >>>> at > >>> > >> > org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManagerTest.testRegisterAndDownload(BlobLibraryCacheManagerTest.java:209) > >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>>> at > >>> > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >>>> at > >>> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >>>> at java.lang.reflect.Method.invoke(Method.java:497) > >>>> at > >>> > >> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > >>>> at > >>> > >> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > >>>> at > >>> > >> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > >>>> at > >>> > >> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > >>>> at > >> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > >>>> at > >>> > >> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > >>>> at > >>> > >> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > >>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >>>> at > >>> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > >>>> at > >>> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >>>> at > >>> org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >>>> at > >>> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >>>> at > >>> > >> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283) > >>>> at > >>> > >> > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173) > >>>> at > >>> > >> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) > >>>> at > >>> > >> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128) > >>>> at > >>> > >> > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) > >>>> at > >>> > >> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) > >>>> at > >>> > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) > >>>> > >>>> Prez Cannady > >>>> p: 617 500 3378 > >>>> e: [email protected] <mailto:[email protected]> > >>>> GH: https://github.com/opencorrelate < > https://github.com/opencorrelate > >>> > >>>> LI: https://www.linkedin.com/in/revprez < > >>> https://www.linkedin.com/in/revprez> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>> > >>> > >> > >
