umehrot2 commented on issue #1290: [HUDI-584] Relocate spark-avro dependency by maven-shade-plugin URL: https://github.com/apache/incubator-hudi/pull/1290#issuecomment-582655960 Also another thing to consider here, is `spark-avro` module is only used for converting between spark's struct type and avro schema. While all the code for actual conversion of the data, is maintained inside of Hudi. I recently fixed an issue with this schema conversion https://github.com/apache/incubator-hudi/pull/1223/files#diff-3c046573a91f36ba0f12dad0e3395dc9R346 where since the way spark-avro 2.4.4 created avro namespaces was different than earlier databricks avro. And this resulted in a change in Hudi's code as well. Considering this, it might be good to actually fix the spark-avro version and relocate it, to avoid customer's running into such issues as this would fix the schema conversion logic being used.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
