menna224 commented on issue #8018:
URL: https://github.com/apache/hudi/issues/8018#issuecomment-1455154156
thank you @kazdy for your reply. I tried to pass the schema this way to the
config you mentioned but I get an error, I am not totally aware how to pass it
can u please help?
```
schema = "user_id: string, firstname: string, operation: string, timestamp:
double"
'hoodie.write.schema':schema
```
got the following error:
```
Caused by: org.apache.avro.SchemaParseException:
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'user_id':
was expecting (JSON String, Number, Array, Object or token 'null', 'true' or
'false')
at [Source: (String)"user_id: string, firstname: string, operation: string,
timestamp: double"; line: 1, column: 8]
```
seems like I shouldn't pass it as a string, but I couldn't get from the doc
how I should pass it.
Also, regarding your part when U mentioned "missing columns -> add missing
columns to match current table schema"
did u need to add extra logic in your code or by default the missing cols
were added? or it's just by adding
'hoodie.datasource.write.reconcile.schema':"true"?
and if the missing cols were added wo extra logic in your code, were u using
pyspark+glue? or what did u use exactly? thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
