[
https://issues.apache.org/jira/browse/HUDI-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yann Byron reassigned HUDI-3195:
--------------------------------
Assignee: Yann Byron
> optimize spark3 pom and modify build command
> --------------------------------------------
>
> Key: HUDI-3195
> URL: https://issues.apache.org/jira/browse/HUDI-3195
> Project: Apache Hudi
> Issue Type: Bug
> Components: Spark Integration
> Reporter: Yann Byron
> Assignee: Yann Byron
> Priority: Major
>
> when use `mvn clean install -P "scala-2.12,spark3,spark3.1.x"` to build spark
> and write data by spark, error is as follows:
>
>
> {code:java}
> ERROR Executor: Exception in task 0.0 in stage 26.0 (TID 2005)
> java.lang.RuntimeException: org.apache.hudi.exception.HoodieException:
> org.apache.hudi.exception.HoodieException:
> java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError:
> org/apache/parquet/schema/LogicalTypeAnnotation$LogicalTypeAnnotationVisitor
> at
> org.apache.hudi.client.utils.LazyIterableIterator.next(LazyIterableIterator.java:121)
> at
> scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44)
> at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:484)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:490)
> at
> org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:221)
> {code}
>
>
> That's due to the use of 1.12.1 parquet.
>
> Also, we use this build command in CI, which will mislead users.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
