I looked into failing tests, we can put them into 5 categories: - CastTest, invalid test case, fixed in apache/beam#7372 <https://github.com/apache/beam/pull/7372> - UsesMetricsPusher tests running with direct-runner, attempted to fix in apache/beam#7374 <https://github.com/apache/beam/pull/7374/commits> - hanging tests using test stream: BoundedReadFromUnboundedSourceTest#testTimeBound and SplittableDoFnTest#testLateData, needs more investigation - test using Watch transform: WatchTest, FileIOTest, AvroIOTest. Watch transform needs refactoring after apache/beam#6467 <https://github.com/apache/beam/pull/6467> - TFRecordIOTest, needs more investigation
Not sure what would be the best way to proceed while tests are failing. On Fri, Dec 28, 2018 at 5:30 PM Reuven Lax <[email protected]> wrote: > I think they run in postcommit. Seems like this was an oversight maybe. > > On Fri, Dec 28, 2018, 5:21 PM Ismaël Mejía <[email protected] wrote: > >> Big +1 >> >> I am in awe that they are not active anymore, they used to be in the >> good old maven days, when IDE integration worked and most JIRAs were >> not about build or test issues. Sigh. >> >> On Fri, Dec 28, 2018 at 3:24 PM Reuven Lax <[email protected]> wrote: >> > >> > Kenn and I both noticed that some needsRunner tests time out, and we >> were both wondering why our PreCommit was still green. This tests are meant >> to be quick, and IMO should definitely be part of Java PreCommit. >> > >> > On Fri, Dec 28, 2018 at 2:59 PM Gleb Kanterov <[email protected]> wrote: >> >> >> >> After reading Beam Testing I had an impression that NeedsRunner tests >> are executed as a part of Java PreCommit using Direct runner. However, it >> doesn't seem to be the case. I've tried running these tests locally, and >> few of them are failing or timeout. >> >> >> >> My proposal is adding ":beam-runners-direct-java:needsRunnerTests" to >> "Run Java PreCommit" Jenkins job, or running them in parallel as a part of >> a new task, not to increase build time. Running needsRunnerTests on my >> laptop took 3 minutes, so I just lean towards adding them to javaPreCommit. >> Thoughts? >> >> >> >> > Task :beam-runners-direct-java:needsRunnerTests >> >> >> >> org.apache.beam.runners.core.metrics.MetricsPusherTest > test FAILED >> >> java.lang.AssertionError at MetricsPusherTest.java:71 >> >> >> >> org.apache.beam.sdk.io.TFRecordIOTest > testReadInvalidDataMask FAILED >> >> java.lang.AssertionError >> >> >> >> org.apache.beam.sdk.io.TFRecordIOTest > testReadInvalidLengthMask >> FAILED >> >> java.lang.AssertionError >> >> >> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > >> testContinuouslyWriteAndReadMultipleFilepatterns[0: true] FAILED >> >> java.lang.IllegalArgumentException at AvroIOTest.java:557 >> >> >> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > >> testWriteThenReadJavaClass[0: true] FAILED >> >> java.lang.AssertionError at AvroIOTest.java:306 >> >> >> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > >> testContinuouslyWriteAndReadMultipleFilepatterns[1: false] FAILED >> >> java.lang.IllegalArgumentException at AvroIOTest.java:557 >> >> >> >> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests > >> testWriteThenReadJavaClass[1: false] FAILED >> >> java.lang.AssertionError at AvroIOTest.java:306 >> >> >> >> org.apache.beam.sdk.io.FileIOTest > testMatchWatchForNewFiles FAILED >> >> java.lang.IllegalArgumentException at FileIOTest.java:192 >> >> >> >> org.apache.beam.sdk.io.TextIOReadTest$BasicIOTest > >> testReadWatchForNewFiles FAILED >> >> java.lang.IllegalArgumentException at TextIOReadTest.java:846 >> >> >> >> org.apache.beam.sdk.schemas.transforms.CastTest > testTypeNarrowFail >> FAILED >> >> java.lang.AssertionError >> >> >> >> org.apache.beam.sdk.schemas.transforms.CastTest > >> testWeakedNullableFail FAILED >> >> java.lang.AssertionError >> >> >> >> org.apache.beam.sdk.transforms.SplittableDoFnTest > testBoundedness >> FAILED >> >> >> org.apache.beam.sdk.testing.TestPipeline$PipelineRunMissingException >> >> >> >> org.apache.beam.sdk.transforms.SplittableDoFnTest > testLateData FAILED >> >> org.junit.runners.model.TestTimedOutException at >> SplittableDoFnTest.java:683 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testSinglePollMultipleInputsWithSideInput FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:112 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testMultiplePollsWithKeyExtractor FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:203 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testSinglePollMultipleInputs FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:82 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testMultiplePollsWithTerminationDueToTerminationCondition FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:150 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testMultiplePollsWithManyResults FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:308 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testSinglePollWithManyResults FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:258 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testMultiplePollsStopAfterTimeSinceNewOutput FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:232 >> >> >> >> org.apache.beam.sdk.transforms.WatchTest > >> testMultiplePollsWithTerminationBecauseOutputIsFinal FAILED >> >> java.lang.IllegalArgumentException at WatchTest.java:150 >> >> >> >> 647 tests completed, 21 failed, 4 skipped >> > -- Cheers, Gleb
