[
https://issues.apache.org/jira/browse/HUDI-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-9133:
---------------------------------
Labels: HoodieStreamer pull-request-available (was: HoodieStreamer)
> HoodieStreamer: Encountering "java.lang.IllegalAccessError: tried to access
> field org.apache.avro.Schema.FACTORY from class org.apache.avro.Schemas"
> exception after upgrading to Hudi 1.0.x version
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HUDI-9133
> URL: https://issues.apache.org/jira/browse/HUDI-9133
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: hudi-utilities
> Affects Versions: 1.0.2
> Reporter: Ranga Reddy
> Priority: Blocker
> Labels: HoodieStreamer, pull-request-available
>
> *Problem Statment:*
> After resolving the {{ClassNotFoundException}} for the
> {{ProtobufSchemaProvider}} as documented in
> HUDI-9057(https://issues.apache.org/jira/browse/HUDI-9057), we encountered a
> new issue: a *{{java.lang.IllegalAccessError}} indicating an attempt to
> access the field {{org.apache.avro.Schema.FACTORY}} from the class*
> {{{}*org.apache.avro.Schemas*{}}}.
> Upon further investigation, it was found that the
> *{{kafka-schema-registry-client}}* JAR in use is version {*}5.5.0{*}, which
> internally relies on {*}Avro version 1.9.2{*}. This version of Avro contains
> a known bug related to the {{Schemas}} class, as highlighted in [this GitHub
> issue]([https://github.com/confluentinc/schema-registry/issues/1432]). The
> bug has been addressed in {*}Avro version 1.10.0{*}.
> *Spark Submit Command:*
> {code:java}
> spark-submit \
> --class org.apache.hudi.utilities.streamer.HoodieStreamer
> hudi-utilities-bundle_2.12-1.1.0-SNAPSHOT.jar \
> --props file:///tmp/my_hudi.properties \
> --table-type MERGE_ON_READ \
> --op UPSERT \
> --target-base-path file:\/\/\/tmp/employees_cdc \
> --target-table employees_cdc \
> --source-class
> org.apache.hudi.utilities.sources.debezium.PostgresDebeziumSource \
> --source-ordering-field _event_lsn \
> --payload-class
> org.apache.hudi.common.model.debezium.PostgresDebeziumAvroPayload{code}
> *Exception:*
> {code:java}
> 25/03/07 12:12:14 INFO SparkContext: Successfully stopped SparkContext
> Exception in thread "main" org.apache.hudi.exception.HoodieException: Failed
> to run HoodieStreamer
> at
> org.apache.hudi.utilities.streamer.HoodieStreamer.main(HoodieStreamer.java:652)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1029)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:194)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:217)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1120)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1129)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.lang.IllegalAccessError: tried to access field
> org.apache.avro.Schema.FACTORY from class org.apache.avro.Schemas
> at org.apache.avro.Schemas.toString(Schemas.java:36)
> at org.apache.avro.Schemas.toString(Schemas.java:30)
> at
> io.confluent.kafka.schemaregistry.avro.AvroSchema.canonicalString(AvroSchema.java:136)
> at
> org.apache.hudi.utilities.schema.SchemaRegistryProvider.fetchSchemaFromRegistry(SchemaRegistryProvider.java:180)
> at
> org.apache.hudi.utilities.sources.debezium.DebeziumSource.fetchNextBatch(DebeziumSource.java:127)
> at
> org.apache.hudi.utilities.sources.RowSource.readFromCheckpoint(RowSource.java:61)
> at org.apache.hudi.utilities.sources.Source.fetchNext(Source.java:182)
> at
> org.apache.hudi.utilities.streamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:190)
> at
> org.apache.hudi.utilities.streamer.StreamSync.fetchNextBatchFromSource(StreamSync.java:721)
> at
> org.apache.hudi.utilities.streamer.StreamSync.fetchFromSourceAndPrepareRecords(StreamSync.java:593)
> at
> org.apache.hudi.utilities.streamer.StreamSync.readFromSource(StreamSync.java:565)
> at
> org.apache.hudi.utilities.streamer.StreamSync.syncOnce(StreamSync.java:470)
> at
> org.apache.hudi.utilities.streamer.HoodieStreamer$StreamSyncService.ingestOnce(HoodieStreamer.java:917)
> at
> org.apache.hudi.utilities.ingestion.HoodieIngestionService.startIngestion(HoodieIngestionService.java:72)
> at org.apache.hudi.common.util.Option.ifPresent(Option.java:101)
> at
> org.apache.hudi.utilities.streamer.HoodieStreamer.sync(HoodieStreamer.java:228)
> at
> org.apache.hudi.utilities.streamer.HoodieStreamer.main(HoodieStreamer.java:649)
> ... 12 more {code}
> *Hudi code:*
> https://github.com/apache/hudi/blob/release-1.0.0/hudi-utilities/src/main/java/org/apache/hudi/utilities/schema/SchemaRegistryProvider.java#L175
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
