Hi Zack, Not sure if you were able to resolve the issue (please keep the list posted either way), I just wanted to point out that it may be worth using a Maven Bill of Materials (BOM) <https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms> to tie compatible dependency versions together and avoid such issues in the future. This blog post https://beam.apache.org/blog/managing-beam-dependencies-in-java/ provides some details on how to do it in Maven.
+1 on Tomo - if you can't figure it out, please send your dependency graph over (mvn dependency:tree), as it will be helpful to debug this further. Best, Bruno On Mon, Nov 4, 2024 at 7:43 PM Ahmet Altay via user <[email protected]> wrote: > Hi Zack, > > Have you seen Tomo's response (below)? Were you able to resolve your issue? > > Ahmet > > On Tue, Oct 29, 2024 at 5:46 PM Tomo Suzuki via user <[email protected]> > wrote: > >> I think you have incompatible library versions in your dependencies. >> Would you share your dependency graph? >> >> On Tue, Oct 29, 2024 at 8:05 PM Zack Culberson < >> [email protected]> wrote: >> >>> Hi team, >>> >>> I am currently seeing an error on some workers within our QA environment >>> that. The error I get is: >>> org.apache.beam.sdk.util.UserCodeException: >>> java.lang.AbstractMethodError: Method >>> org/apache/beam/fn/harness/control/ExecutionStateSampler$MetricsContainerForTracker.getStringSet(Lorg/apache/beam/sdk/metrics/MetricName;)Lorg/apache/beam/sdk/metrics/StringSet; >>> is abstract >>> >>> at >>> org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:39) >>> >>> at >>> org.apache.beam.sdk.io.kafka.KafkaWriter$DoFnInvoker.invokeProcessElement(Unknown >>> Source) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:803) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo(FnApiDoFnRunner.java:1792) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000(FnApiDoFnRunner.java:143) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output(FnApiDoFnRunner.java:2650) >>> >>> at >>> org.apache.beam.sdk.transforms.MapElements$2.processElement(MapElements.java:151) >>> >>> at >>> org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement(Unknown >>> Source) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:803) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo(FnApiDoFnRunner.java:1792) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000(FnApiDoFnRunner.java:143) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output(FnApiDoFnRunner.java:2650) >>> >>> at >>> org.apache.beam.sdk.transforms.MapElements$2.processElement(MapElements.java:151) >>> >>> at >>> org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement(Unknown >>> Source) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:803) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo(FnApiDoFnRunner.java:1792) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000(FnApiDoFnRunner.java:143) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output(FnApiDoFnRunner.java:2650) >>> >>> at >>> org.albertsons.audi.IBMMQToKafka$TokenizeAndReplaceDoFn_xml.processElement(IBMMQToKafka.java:593) >>> >>> at >>> org.albertsons.audi.IBMMQToKafka$TokenizeAndReplaceDoFn_xml$DoFnInvoker.invokeProcessElement(Unknown >>> Source) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:803) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo(FnApiDoFnRunner.java:1792) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000(FnApiDoFnRunner.java:143) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output(FnApiDoFnRunner.java:2650) >>> >>> at >>> org.apache.beam.sdk.values.ValueWithRecordId$StripIdsDoFn.processElement(ValueWithRecordId.java:139) >>> >>> at >>> org.apache.beam.sdk.values.ValueWithRecordId$StripIdsDoFn$DoFnInvoker.invokeProcessElement(Unknown >>> Source) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:803) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo(FnApiDoFnRunner.java:1792) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.access$3000(FnApiDoFnRunner.java:143) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner$WindowObservingProcessBundleContext.outputWithTimestamp(FnApiDoFnRunner.java:2219) >>> >>> at >>> org.apache.beam.sdk.io.Read$UnboundedSourceAsSDFWrapperFn.processElement(Read.java:580) >>> >>> at >>> org.apache.beam.sdk.io.Read$UnboundedSourceAsSDFWrapperFn$DoFnInvoker.invokeProcessElement(Unknown >>> Source) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForWindowObservingSizedElementAndRestriction(FnApiDoFnRunner.java:1100) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner.access$1500(FnApiDoFnRunner.java:143) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner$4.accept(FnApiDoFnRunner.java:659) >>> >>> at >>> org.apache.beam.fn.harness.FnApiDoFnRunner$4.accept(FnApiDoFnRunner.java:654) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) >>> >>> at >>> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) >>> >>> at >>> org.apache.beam.fn.harness.BeamFnDataReadRunner.forwardElementToConsumer(BeamFnDataReadRunner.java:213) >>> >>> at >>> org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements(BeamFnDataInboundObserver.java:172) >>> >>> at >>> org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle(ProcessBundleHandler.java:537) >>> >>> at >>> org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:150) >>> >>> at >>> org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:115) >>> >>> at >>> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) >>> >>> at >>> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) >>> >>> at >>> org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run(UnboundedScheduledExecutorService.java:163) >>> >>> at >>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >>> >>> at >>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >>> >>> at java.base/java.lang.Thread.run(Thread.java:829) >>> >>> Caused by: java.lang.AbstractMethodError: Method >>> org/apache/beam/fn/harness/control/ExecutionStateSampler$MetricsContainerForTracker.getStringSet(Lorg/apache/beam/sdk/metrics/MetricName;)Lorg/apache/beam/sdk/metrics/StringSet; >>> is abstract >>> >>> at >>> org.apache.beam.fn.harness.control.ExecutionStateSampler$MetricsContainerForTracker.getStringSet(ExecutionStateSampler.java) >>> >>> at >>> org.apache.beam.sdk.metrics.Metrics$DelegatingStringSet.add(Metrics.java:179) >>> >>> at >>> org.apache.beam.sdk.metrics.Lineage.add(Lineage.java:115) >>> >>> at >>> org.apache.beam.sdk.metrics.Lineage.add(Lineage.java:122) >>> >>> at >>> org.apache.beam.sdk.io.kafka.KafkaWriter.processElement(KafkaWriter.java:99) >>> >>> And this is my Kafka writer: >>> >>> tokenizedMessagesXml.get(successTag).apply("WriteToKafka", >>> KafkaIO.<Void, String>write() >>> >>> >>> .withBootstrapServers(consumerConfig.get("bootstrap.servers").toString()) >>> >>> .withTopic(topic) >>> >>> .withValueSerializer(StringSerializer.class) >>> >>> .updateProducerProperties(consumerConfigUpdates) >>> >>> .values() >>> >>> ); >>> >>> >>> >>> The Java version I was using is 21 but I set the pom xml to build with >>> version 11 and we are using apache beam sdk version 2.59.0 >>> >>> <maven.compiler.source>11</maven.compiler.source> >>> >>> <maven.compiler.target>11</maven.compiler.target> >>> >>> >>> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> >>> >>> <main.class>org.albertsons.audi.IBMMQToKafka</main.class> >>> >>> <beam.sdk.version>2.59.0</beam.sdk.version> >>> >>> >>> Any help on why I am getting this error would be greatly appreciated. >>> >>> >>> >>> >>> >>> >>> >>> *Zack Culberson* >>> >>> Data Engineer, Retail >>> >>> [email protected] >>> >>> Albertsons Companies >>> >>> Phone: +1.605.431.1112 <(605)%20431-1112> >>> >>> [image: image005] >>> >>> >>> ------------------------------ >>> Warning: All e-mail sent to this address will be received by the >>> corporate e-mail system, and is subject to archival and review by someone >>> other than the recipient. This e-mail may contain proprietary information >>> and is intended only for the use of the intended recipient(s). If the >>> reader of this message is not the intended recipient(s), you are notified >>> that you have received this message in error and that any review, >>> dissemination, distribution or copying of this message is strictly >>> prohibited. If you have received this message in error, please notify the >>> sender immediately. >>> ------------------------------ >>> >> >> >> -- >> Regards, >> Tomo >> >
