I would recommend reaching out to the Beam project instead, because this
seems like an issue that is entirely on their side.
As for the listed workaround, they are suggesting to modify the
wordcount example to also bundle the listed jackson dependencies.
Maybe you could use a different example instead?
On 08/07/2021 18:10, Joey Tran wrote:
Hi!
I'm trying to just get my bearings with dataproc and flink/beam. When
trying to run the wordcount example
<https://cloud.google.com/dataproc/docs/concepts/components/flink#portable_beam_jobs> with
a long-running YARN sessions
<https://cloud.google.com/dataproc/docs/concepts/components/flink>.
The error looks like:
Traceback (most recent call last):
File "wordcount.py", line 99, in <module>
run()
File "wordcount.py", line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File
"/home/jtran/.local/lib/python3.8/site-packages/apache_beam/pipeline.py",
line 586, in __exit__
self.result.wait_until_finish()
File
"/home/jtran/.local/lib/python3.8/site-packages/apache_beam/runners/portability/portable_runner.py",
line 599, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline
BeamApp-jtran-0708141941-5fcff870_effe0238-2afb-4707-a981-133bc46618fd
failed in state FAILED: java.util.ServiceConfigurationError:
com.fasterxml.jackson.databind.Module: Provider
com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule not a subtype
flink version 1.12, dataproc image 2.0, beam version 2.30
I found some mention of this when trying to run the example with EMR
but no mention with dataproc
(https://issues.apache.org/jira/browse/BEAM-10430
<https://issues.apache.org/jira/browse/BEAM-10430>). The workaround is
unclear to me, is there something wrong I'm doing with setting this
up? Any advice would be greatly appreciated. Thanks in advance!
