Zhike Chen created HUDI-4326:
--------------------------------
Summary: Hudi spark datasource error after migrate from 0.8 to 0.11
Key: HUDI-4326
URL: https://issues.apache.org/jira/browse/HUDI-4326
Project: Apache Hudi
Issue Type: Bug
Components: spark
Reporter: Zhike Chen
After updated hudi to 0.11 from 0.8, using {{spark.table(fullTableName)}} to
read a hudi table is not working, the table has been sync to hive metastore and
spark is connected to the metastore. the error is
org.sparkproject.guava.util.concurrent.UncheckedExecutionException:
org.apache.hudi.exception.HoodieException: 'path' or 'Key:
'hoodie.datasource.read.paths' , default: null description: Comma separated
list of file paths to read within a Hudi table. since version: version is not
defined deprecated after: version is not defined)' or both must be specified.
at org.sparkproject.guava.cache.LocalCache$Segment.get(LocalCache.java:2263)
at org.sparkproject.guava.cache.LocalCache.get(LocalCache.java:4000)
at
org.sparkproject.guava.cache.LocalCache$LocalManualCache.get(LocalCache.java:4789)
at org.apache.spark.sql.catalyst.catalog.SessionCatalog.
...
Caused by: org.apache.hudi.exception.HoodieException: 'path' or 'Key:
'hoodie.datasource.read.paths' , default: null description: Comma separated
list of file paths to read within a Hudi table. since version: version is not
defined deprecated after: version is not defined)' or both must be specified.
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:78)
at
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:353)
at
org.apache.spark.sql.execution.datasources.FindDataSourceTable.$anonfun$readDataSourceTable$1(DataSourceStrategy.scala:261)
at
org.sparkproject.guava.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4792)
at
org.sparkproject.guava.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
at
org.sparkproject.guava.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
at
org.sparkproject.guava.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
at
org.sparkproject.guava.cache.LocalCache$Segment.get(LocalCache.java:2257)
After changing the table to the spark data source table, the table SerDeInfo is
missing. I created a pull request.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
