bhasudha commented on issue #1290: [HUDI-584] Relocate spark-avro dependency by maven-shade-plugin URL: https://github.com/apache/incubator-hudi/pull/1290#issuecomment-582654435 > > If the user wants some changes in spark-avro:3.0-preview2, the right way is modify spark-version to 3.0-preview2 at pom.xml file, then build hudi project source. > > Understood... but we would like to avoid this need for building a version of hudi by themselves. Users should be able to do `--packages <hudi-spark-bundle*.jar>` and be on their way.. This change forces everyone who is not on the spark version used by hudi to do their own builds.. I still feel this is not a good idea. > > Once again, I wish we can have more upfront discussion on the JIRA, rather than on a PR, around issues like this. (I feel bad when someone puts in the work to do the implementation that we cannot take :() +1. Good catch. @lamber-ken I think Vinoth brings up a valid point. Although your PR intends to make it easier for users to not care about scala 2.11 or scala 2.12, we also need to avoid coupling Hudi with specific spark_avro versions be it 2.4.4 or 3.0-preview2.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
