Thanks
Yes, we changed the API. I think we need to rebuild dataflow worker like we did
there (1) for example
[1] https://issues.apache.org/jira/browse/BEAM-2012
Etienne
Le mercredi 28 mars 2018 à 20:32 +0000, Alan Myrvold a écrit :
> Thomas logged this issue as
> https://issues.apache.org/jira/projects/BEAM/issues/BEAM-3964
>
> On Wed, Mar 28, 2018 at 10:09 AM Alan Myrvold <amyrv...@google.com> wrote:
> > I looked at one failure, and saw this error in the log. Could it be related
> > to https://github.com/apache/beam/pull/4
> > 918 ?
> >
> >
> > 2018-03-28 09:45:26.603 PDTThread 25 died.
> > Expand all | Collapse all
> > {
> > insertId: "7765450222281849523:822129:0:57106"
> > jsonPayload: {
> > exception: "java.lang.NoSuchMethodError:
> > org.apache.beam.sdk.metrics.MetricName.name()Ljava/lang/String;
> > at
> > com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.structuredNameAndMetadata(MetricsToCounterUpdateCon
> > verter.java:99)
> > at
> > com.google.cloud.dataflow.worker.MetricsToCounterUpdateConverter.fromCounter(MetricsToCounterUpdateConverter.java:68
> > )
> > at
> > com.google.cloud.dataflow.worker.BatchModeExecutionContext.lambda$null$1(BatchModeExecutionContext.java:463)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.Iterators$7.transform(Iterators.java:750)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.j
> > ava:47)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.MultitransformedIterator.next(Multitransformed
> > Iterator.java:66)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCo
> > llection.java:388)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableCollection$ArrayBasedBuilder.addAll(I
> > mmutableCollection.java:472)
> > at
> > com.google.cloud.dataflow.worker.repackaged.com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.jav
> > a:669)
> > at
> > com.google.cloud.dataflow.worker.WorkItemStatusClient.populateCounterUpdates(WorkItemStatusClient.java:256)
> > at
> > com.google.cloud.dataflow.worker.WorkItemStatusClient.createStatusUpdate(WorkItemStatusClient.java:240)
> > at
> > com.google.cloud.dataflow.worker.WorkItemStatusClient.reportError(WorkItemStatusClient.java:94)
> > at
> > com.google.cloud.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:358)
> > at
> > com.google.cloud.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:284)
> > at
> > com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:134)
> > at
> > com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:114)
> > at
> > com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:101)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> >
> > On Wed, Mar 28, 2018 at 6:32 AM Etienne Chauchot <echauc...@apache.org>
> > wrote:
> > > Hi all,
> > > Please know that I have submitted a PR to skip IT tests from de PreCommit
> > > java tests (1). They still run in
> > > PostCommit.
> > > [1] https://github.com/apache/beam/pull/4967
> > >
> > > Etienne
> > >
> > > Le jeudi 22 mars 2018 à 11:54 +0100, Etienne Chauchot a écrit :
> > > > Also, WDYT about running these tests as PostCommit instead of preCommit
> > > > as they are integration tests?
> > > >
> > > > Etienne
> > > >
> > > > Le jeudi 22 mars 2018 à 09:49 +0100, Etienne Chauchot a écrit :
> > > > > Hi all,
> > > > > java PreCommit test fails on jenkins on the examples module
> > > > > (woundCountIT). It gives incorrect signal on the
> > > > > build of PRs.
> > > > > It seems to be related to communication issues with dataflow service
> > > > >
> > > > > org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchStaticSharding
> > > > > or
> > > > > org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatchDynamicSharding
> > > > > A work item was attempted 4 times without success. Each time the
> > > > > worker eventually lost contact with the
> > > > > service. The work item was attempted on:
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-qxtj,
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-98n1,
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-47mf,
> > > > > testpipeline-jenkins-0321-03210922-9f05-harness-n1vb
> > > > >
> > > > >
> > > > > org.apache.beam.examples.WordCountIT.testE2EWordCount
> > > > > java.lang.RuntimeException: Workflow failed. Causes: The Dataflow
> > > > > appears to be stuck. You can get help with
> > > > > Cloud Dataflow at https://cloud.google.com/dataflow/support.
> > > > > at
> > > > > org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
> > > > > at
> > > > > org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
> > > > > at
> > > > > org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
> > > > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
> > > > > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
> > > > > at
> > > > > org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
> > > > > at
> > > > > org.apache.beam.examples.WordCountIT.testE2EWordCount(WordCountIT.java:70)
> > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > > at
> > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > > > at
> > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > > > at
> > > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > > > > at
> > > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > > > > at
> > > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> > > > > at
> > > > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> > > > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > > > > at
> > > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> > > > > at
> > > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> > > > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > > > > at
> > > > > org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> > > > > at
> > > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > > > at
> > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > at
> > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > at java.lang.Thread.run(Thread.java:748)
> > > > >
> > > > > Does anyone has a clue?
> > > > >
> > > > > Etienne
