Hi Srikanth, Could you give a bit more information on Hop version and what the pipeline looks like? Have you tried following and running the Beam Dataflow Sample [1]
Kr, Hans [1] https://hop.apache.org//manual/latest/pipeline/beam/beam-samples-dataflow.html On 9 Apr 2025 at 08:37 +0200, Srikanth Komirishetty <srikanthpatel...@gmail.com>, wrote: > Hi Team, > > Hope this email finds you well,I writing this mail to get a help regarding > apache hop. > I am running a cloud dataflow pipeline using Hop with apache beam. > There are some errors .could you please suggest the solutions for > that.Below are the errors. > 2025/04/09 10:36:03 - Hop - Projects enabled > 2025/04/09 11:45:24 - Hop - Pipeline opened. > 2025/04/09 11:45:24 - Hop - Launching pipeline [cus]... > 2025/04/09 11:45:24 - Hop - Started the pipeline execution. > 2025/04/09 11:45:27 - Hop - ERROR: cus: preparing pipeline execution failed > 2025/04/09 11:45:27 - Hop - ERROR: > org.apache.hop.core.exception.HopException: > 2025/04/09 11:45:27 - Hop - Error preparing remote pipeline > 2025/04/09 11:45:27 - Hop - Error converting Hop pipeline to Beam > 2025/04/09 11:45:27 - Hop - > 2025/04/09 11:45:27 - Hop - at > org.apache.hop.beam.engines.BeamPipelineEngine.prepareExecution(BeamPipelineEngine.java:296) > 2025/04/09 11:45:27 - Hop - at > org.apache.hop.ui.hopgui.file.pipeline.HopGuiPipelineGraph.lambda$preparePipeline$15(HopGuiPipelineGraph.java:4532) > 2025/04/09 11:45:27 - Hop - at > java.base/java.lang.Thread.run(Thread.java:842) > 2025/04/09 11:45:27 - Hop - Caused by: java.lang.Exception: Error > converting Hop pipeline to Beam > 2025/04/09 11:45:27 - Hop - at > org.apache.hop.beam.pipeline.HopPipelineMetaToBeamPipelineConverter.createPipeline(HopPipelineMetaToBeamPipelineConverter.java:308) > 2025/04/09 11:45:27 - Hop - at > org.apache.hop.beam.engines.BeamPipelineEngine.prepareExecution(BeamPipelineEngine.java:265) > 2025/04/09 11:45:27 - Hop - ... 2 more > 2025/04/09 11:45:27 - Hop - Caused by: java.lang.RuntimeException: > Encountered checked exception when constructing an instance from factory > method DataflowRunner#fromOptions(interface > org.apache.beam.sdk.options.PipelineOptions) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:158) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.Pipeline.create(Pipeline.java:158) > 2025/04/09 11:45:27 - Hop - at > org.apache.hop.beam.pipeline.HopPipelineMetaToBeamPipelineConverter.createPipeline(HopPipelineMetaToBeamPipelineConverter.java:283) > 2025/04/09 11:45:27 - Hop - ... 3 more > 2025/04/09 11:45:27 - Hop - Caused by: java.lang.NoSuchMethodError: > 'java.lang.Object > com.google.protobuf.DescriptorProtos$ServiceOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)' > 2025/04/09 11:45:27 - Hop - at > com.google.cloud.hadoop.gcsio.GoogleCloudStorageOptions.<clinit>(GoogleCloudStorageOptions.java:58) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.extensions.gcp.util.GcsUtil.<init>(GcsUtil.java:261) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.extensions.gcp.util.GcsUtil$GcsUtilFactory.create(GcsUtil.java:150) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.extensions.gcp.util.GcsUtil$GcsUtilFactory.create(GcsUtil.java:130) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper(ProxyInvocationHandler.java:689) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:630) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:227) > 2025/04/09 11:45:27 - Hop - at > jdk.proxy2/jdk.proxy2.$Proxy82.getGcsUtil(Unknown Source) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.verifyPathIsAccessible(GcsPathValidator.java:83) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.extensions.gcp.storage.GcsPathValidator.validateOutputFilePrefixSupported(GcsPathValidator.java:53) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:385) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:366) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper(ProxyInvocationHandler.java:689) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:630) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:227) > 2025/04/09 11:45:27 - Hop - at > jdk.proxy2/jdk.proxy2.$Proxy55.getGcpTempLocation(Unknown Source) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:290) > 2025/04/09 11:45:27 - Hop - at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > 2025/04/09 11:45:27 - Hop - at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > 2025/04/09 11:45:27 - Hop - at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 2025/04/09 11:45:27 - Hop - at > java.base/java.lang.reflect.Method.invoke(Method.java:568) > 2025/04/09 11:45:27 - Hop - at > org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:217) > 2025/04/09 11:45:27 - Hop - ... 7 more > > Best Regards, > Srikanth komirishetty > 7095450229
