Is it possible the Python environment the Docker image built from was contaminated to begin with?
On Thu, Mar 26, 2020 at 11:06 AM Udi Meiri <[email protected]> wrote: > The Python HDFS IT uses the jenkins BUILD_TAG to create unique names: > PROJECT_NAME=$(echo hdfs_IT-${BUILD_TAG:-non-jenkins}) > > The BUILD_TAG is unique and easily traced back to the Jenkins job that > made it. > It might need some sanitizing though if it contains any invalid characters. > > On Tue, Mar 24, 2020 at 1:50 PM Hannah Jiang <[email protected]> > wrote: > >> This can be done by 1). passing "-Pdocker-tag=xxx" to the test and 2). >> make sure to specify the custom tag when using docker images. >> For example, *:sdks:python:test-suites:portable:py35:preCommitPy35 >> -Pdocker-tag=20200324 *will create an image with a tag 20200324. >> *--environment_config=path/to/container/image* pipeline option can be >> used for Python pipeline to pass custom docker images. >> >> >> >> On Tue, Mar 24, 2020 at 11:42 AM Brian Hulette <[email protected]> >> wrote: >> >>> Failing run: >>> https://builds.apache.org/job/beam_PostCommit_XVR_Flink_PR/65/ >>> Passing run: >>> https://builds.apache.org/job/beam_PostCommit_XVR_Flink_PR/66/ >>> >>> On Tue, Mar 24, 2020 at 11:33 AM Hannah Jiang <[email protected]> >>> wrote: >>> >>>> Hi Brian >>>> >>>> I think that's possible if we use the default tag for the Jenkins >>>> tests. To prevent this, we can use a customized tag, for example, >>>> timestamp, for each build. >>>> Can you please point me to the failing tests? I will check more details. >>>> >>>> Thanks, >>>> Hannah >>>> >>>> >>>> On Tue, Mar 24, 2020 at 10:11 AM Brian Hulette <[email protected]> >>>> wrote: >>>> >>>>> I ran into a test failure on the XVR tests in [1] which looked like >>>>> the test was executing with a python docker container that did _not_ >>>>> include the python changes in my PR. The test ran successfully after a >>>>> second run. >>>>> >>>>> It seems likely that the initial failure occurred because some other >>>>> job was running concurrently on the same jenkins worker and overwrote the >>>>> ` >>>>> apache/beam_python2.7_sdk:2.21.0.dev` image that my run had >>>>> generated. Is this possible? If so, is there something we should do to >>>>> isolate these images? >>>>> >>>>> [1] https://github.com/apache/beam/pull/10055 >>>>> >>>>
