I think option 2 is ideal, but I don't know if there is any hard requirement from ASF/Maven Central side for us to keep backwards compatibility of package names published in maven. If there is a requirement then we cannot change it.
As a mitigation, I stated in https://iceberg.apache.org/multi-engine-support that Spark 2.4 and 3.0 jar names do not follow the naming convention of newer versions for backwards compatibility. Best, Jack Ye On Sun, Feb 20, 2022 at 7:03 PM OpenInx <open...@gmail.com> wrote: > Hi everyone > > The current spark2.4, spark3.0 have the following unaligned runtime > artifact names: > > # Spark 2.4 > iceberg-spark-runtime-0.13.1.jar > # Spark 3.0 > iceberg-spark3-runtime-0.13.1.jar > # Spark 3.1 > iceberg-spark-runtime-3.1_2.12-0.13.1.jar > # Spark 3.2 > iceberg-spark-runtime-3.2_2.12-0.13.1.jar > > From the spark 3.1 and spark 3.2's runtime artifact names, we can easily > recognize: > 1. What's the spark major version that the runtime jar is attached to > 2. What's the spark scala version that the runtime jar is compiled with > > But for spark 3.0 and spark 2.4, it's not easy to understand what's the > above information. I think we kept those legacy names because they were > introduced in older iceberg releases and we wanted to avoid changing the > modules that users depend on and opted not to rename, but they are indeed > causing confusion for the new community users. > > In general, we have two options: > > Option#1: keep the current artifact names, that mean spark 2.4 & spark > 3.0 will always use the iceberg-spark-runtime-<iceberg-version>.jar and > iceberg-spark3-runtime-<iceberg-version>.jar until them get retired in the > apache iceberg official repo. > Option#2: Change the spark2.4 & spark3.0's artifact names to the generic > name format: > iceberg-spark-runtime-<spark-major.minor>_<scala-version>-<iceberg-version>.jar. > It makes sharing all the consistent name format between all the spark > versions. > > Personally, I'd prefer option#2 because that looks more friendly for new > community users (although it will require the old users to change their > pom.xml to the new version). > > What is your preference ? > > Reference: > 1. Created a PR to change the artifact names and we had few discussions > there. https://github.com/apache/iceberg/pull/4158 > 2. https://github.com/apache/iceberg-docs/pull/27#discussion_r800297155 >
