This is an automated email from the ASF dual-hosted git repository. jiayu pushed a commit to branch release-notes in repository https://gitbox.apache.org/repos/asf/sedona.git
commit c57f882ae462568f9bed32abbc87b9f84c7b7454 Author: Jia Yu <[email protected]> AuthorDate: Sun Sep 7 20:52:39 2025 -0700 Update the supported versions --- docs/community/release-manager.md | 14 +++++------ docs/setup/compile.md | 16 ++++++------ docs/setup/databricks.md | 6 ++--- docs/setup/flink/platform.md | 9 ++++++- docs/setup/glue.md | 3 +++ docs/setup/maven-coordinates.md | 52 +++++++++++++++++++-------------------- docs/setup/platform.md | 38 ++++++++++++++-------------- docs/setup/release-notes.md | 15 +++++++++++ 8 files changed, 90 insertions(+), 63 deletions(-) diff --git a/docs/community/release-manager.md b/docs/community/release-manager.md index fede71e4f6..3252154403 100644 --- a/docs/community/release-manager.md +++ b/docs/community/release-manager.md @@ -23,13 +23,13 @@ You only need to perform these steps if this is your first time being a release ## 0. Software requirement -* JDK 8: `brew install openjdk@8` -* Maven 3.X. Your Maven must point to JDK 8 (1.8). Check it by `mvn --version` +* JDK 11/17: `brew install openjdk@11` or `brew install openjdk@17` +* Maven 3.X. Your Maven must point to JDK 11 or 17. Check it by `mvn --version` * Git and SVN -If your Maven (`mvn --version`) points to other JDK versions, you must change it to JDK 8. Steps are as follows: +If your Maven (`mvn --version`) points to other JDK versions, you must change it to JDK 11 or 17. Steps are as follows: -1. Find all Java installed on your machine: `/usr/libexec/java_home -V`. You should see multiple JDK versions including JDK 8. +1. Find all Java installed on your machine: `/usr/libexec/java_home -V`. You should see multiple JDK versions including JDK 11 and 17. 2. Run `whereis mvn` to get the installation location of your Maven. The result is a symlink to the actual location. 3. Open it in the terminal (with `sudo` if needed). It will be like this @@ -38,14 +38,14 @@ If your Maven (`mvn --version`) points to other JDK versions, you must change it JAVA_HOME="${JAVA_HOME:-$(/usr/libexec/java_home)}" exec "/usr/local/Cellar/maven/3.6.3/libexec/bin/mvn" "$@" ``` -4. Change `JAVA_HOME:-$(/usr/libexec/java_home)}` to `JAVA_HOME:-$(/usr/libexec/java_home -v 1.8)}`. The resulting content will be like this: +4. Change `JAVA_HOME:-$(/usr/libexec/java_home)}` to `JAVA_HOME:-$(/usr/libexec/java_home -v 11)}` or `JAVA_HOME:-$(/usr/libexec/java_home -v 17)}`. The resulting content will be like this: ``` #!/bin/bash -JAVA_HOME="${JAVA_HOME:-$(/usr/libexec/java_home -v 1.8)}" exec "/usr/local/Cellar/maven/3.6.3/libexec/bin/mvn" "$@" +JAVA_HOME="${JAVA_HOME:-$(/usr/libexec/java_home -v 11)}" exec "/usr/local/Cellar/maven/3.6.3/libexec/bin/mvn" "$@" ``` -5. Run `mvn --version` again. It should now point to JDK 8. +5. Run `mvn --version` again. It should now point to JDK 11 or 17. ## 1. Obtain Write Access to Sedona GitHub repo diff --git a/docs/setup/compile.md b/docs/setup/compile.md index d9b563bc23..1c66c49244 100644 --- a/docs/setup/compile.md +++ b/docs/setup/compile.md @@ -25,7 +25,7 @@ Sedona Scala/Java code is a project with multiple modules. Each module is a Scala/Java mixed project which is managed by Apache Maven 3. -* Make sure your Linux/Mac machine has Java 1.11, Apache Maven 3.3.1+, and Python3.8+. The compilation of Sedona is not tested on Windows machines. +* Make sure your Linux/Mac machine has Java 11/17, Apache Maven 3.3.1+, and Python3.8+. The compilation of Sedona is not tested on Windows machines. To compile all modules, please make sure you are in the root folder of all modules. Then enter the following command in the terminal: @@ -48,7 +48,7 @@ To compile all modules, please make sure you are in the root folder of all modul Geotools jars will be packaged into the produced fat jars. !!!note - By default, this command will compile Sedona with Spark 3.3 and Scala 2.12 + By default, this command will compile Sedona with Spark 3.4 and Scala 2.12 ### Compile with different targets @@ -57,19 +57,19 @@ User can specify `-Dspark` and `-Dscala` command line options to compile with di * `-Dspark`: `{major}.{minor}`: For example, specify `-Dspark=3.4` to build for Spark 3.4. * `-Dscala`: `2.12` or `2.13` -=== "Spark 3.3+ Scala 2.12" +=== "Spark 3.4+ Scala 2.12" ``` - mvn clean install -DskipTests -Dspark=3.3 -Dscala=2.12 + mvn clean install -DskipTests -Dspark=3.4 -Dscala=2.12 ``` - Please replace `3.3` with Spark major.minor version when building for higher Spark versions. -=== "Spark 3.3+ Scala 2.13" + Please replace `3.4` with Spark major.minor version when building for higher Spark versions. +=== "Spark 3.4+ Scala 2.13" ``` mvn clean install -DskipTests -Dspark=3.4 -Dscala=2.13 ``` - Please replace `3.3` with Spark major.minor version when building for higher Spark versions. + Please replace `3.4` with Spark major.minor version when building for higher Spark versions. !!!tip - To get the Sedona Spark Shaded jar with all GeoTools jars included, simply append `-Dgeotools` option. The command is like this:`mvn clean install -DskipTests -Dscala=2.12 -Dspark=3.0 -Dgeotools` + To get the Sedona Spark Shaded jar with all GeoTools jars included, simply append `-Dgeotools` option. The command is like this:`mvn clean install -DskipTests -Dscala=2.12 -Dspark=3.4 -Dgeotools` ### Download staged jars diff --git a/docs/setup/databricks.md b/docs/setup/databricks.md index af96a2bc64..22f61a71b6 100644 --- a/docs/setup/databricks.md +++ b/docs/setup/databricks.md @@ -48,11 +48,11 @@ curl -o /Workspace/Shared/sedona/{{ sedona.current_version }}/geotools-wrapper-{ curl -o /Workspace/Shared/sedona/{{ sedona.current_version }}/sedona-spark-shaded-3.5_2.12-{{ sedona.current_version }}.jar "https://repo1.maven.org/maven2/org/apache/sedona/sedona-spark-shaded-3.5_2.12/{{ sedona.current_version }}/sedona-spark-shaded-3.5_2.12-{{ sedona.current_version }}.jar" ``` -Here are the software versions used to compile `sedona-spark-shaded-3.5_2.12-1.7.1.jar`: +Here are the software versions used to compile `sedona-spark-shaded-3.5_2.12-{{ sedona.current_version }}.jar`: * Spark 3.5 * Scala 2.12 -* Sedona 1.7.1 +* Sedona {{ sedona.current_version }} Ensure that you use a Databricks Runtime with versions compatible with this jar. @@ -76,7 +76,7 @@ cat > /Workspace/Shared/sedona/sedona-init.sh <<'EOF' # # On cluster startup, this script will copy the Sedona jars to the cluster's default jar directory. -cp /Workspace/Shared/sedona/1.7.1/*.jar /databricks/jars +cp /Workspace/Shared/sedona/{{ sedona.current_version }}/*.jar /databricks/jars EOF ``` diff --git a/docs/setup/flink/platform.md b/docs/setup/flink/platform.md index 082db1e61d..29e9b4de23 100644 --- a/docs/setup/flink/platform.md +++ b/docs/setup/flink/platform.md @@ -17,4 +17,11 @@ under the License. --> -Sedona Flink binary releases are compiled by Java 1.8 and Scala 2.12, and tested with Flink 1.12 - 1.19. +Sedona Flink binary releases are compiled by Java 11/17 and Scala 2.12, and tested with Flink 1.12 - 1.19. + +**Java Requirements:** + +- Flink 1.12 - 1.18: Java 11 +- Flink 1.19+: Java 17 + +**Note:** Java 8 support is dropped since Sedona 1.8.0. diff --git a/docs/setup/glue.md b/docs/setup/glue.md index 1937836764..ec6c6722a8 100644 --- a/docs/setup/glue.md +++ b/docs/setup/glue.md @@ -25,6 +25,9 @@ In the tutorial, we use Sedona {{ sedona.current_version }} and [Glue 4.0](https://docs.aws.amazon.com/glue/latest/dg/release-notes.html) which runs on Spark 3.3.0, Java 8, Scala 2.12, and Python 3.10. We recommend Sedona-1.3.1-incubating and above for Glue. +!!!warning + **Important:** Since Sedona 1.8.0, Java 8 support is dropped and Spark 3.3 support is dropped. For Sedona 1.8.0+, you need to use Glue 5.0+ which supports Java 11 and Spark 3.4+. + ## Gather Maven Links You will need to point your glue job to the Sedona and Geotools jars. We recommend using the jars available from maven. The links below are those intended for Glue 4.0 diff --git a/docs/setup/maven-coordinates.md b/docs/setup/maven-coordinates.md index e54870be7b..0da7eb3973 100644 --- a/docs/setup/maven-coordinates.md +++ b/docs/setup/maven-coordinates.md @@ -37,12 +37,12 @@ The optional GeoTools library is required if you want to use CRS transformation, !!! abstract "Sedona with Apache Spark and Scala 2.12" - === "Spark 3.3 and Scala 2.12" + === "Spark 3.4 and Scala 2.12" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-shaded-3.3_2.12</artifactId> + <artifactId>sedona-spark-shaded-3.4_2.12</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper --> @@ -52,13 +52,12 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - - === "Spark 3.4 and Scala 2.12" + === "Spark 3.5 and Scala 2.12" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-shaded-3.4_2.12</artifactId> + <artifactId>sedona-spark-shaded-3.5_2.12</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper --> @@ -68,12 +67,13 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - === "Spark 3.5 and Scala 2.12" + + === "Spark 4.0 and Scala 2.12" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-shaded-3.5_2.12</artifactId> + <artifactId>sedona-spark-shaded-4.0_2.12</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper --> @@ -86,12 +86,12 @@ The optional GeoTools library is required if you want to use CRS transformation, !!! abstract "Sedona with Apache Spark and Scala 2.13" - === "Spark 3.3 and Scala 2.13" + === "Spark 3.4 and Scala 2.13" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-shaded-3.3_2.13</artifactId> + <artifactId>sedona-spark-shaded-3.4_2.13</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper --> @@ -101,13 +101,12 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - - === "Spark 3.4 and Scala 2.13" + === "Spark 3.5 and Scala 2.13" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-shaded-3.4_2.13</artifactId> + <artifactId>sedona-spark-shaded-3.5_2.13</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper --> @@ -117,12 +116,13 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - === "Spark 3.5 and Scala 2.13" + + === "Spark 4.0 and Scala 2.13" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-shaded-3.5_2.13</artifactId> + <artifactId>sedona-spark-shaded-4.0_2.13</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper --> @@ -184,11 +184,11 @@ The optional GeoTools library is required if you want to use CRS transformation, !!! abstract "Sedona with Apache Spark and Scala 2.12" - === "Spark 3.3 and Scala 2.12" + === "Spark 3.4 and Scala 2.12" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-3.3_2.12</artifactId> + <artifactId>sedona-spark-3.4_2.12</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <dependency> @@ -197,11 +197,11 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - === "Spark 3.4 and Scala 2.12" + === "Spark 3.5 and Scala 2.12" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-3.4_2.12</artifactId> + <artifactId>sedona-spark-3.5_2.12</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <dependency> @@ -210,11 +210,11 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - === "Spark 3.5 and Scala 2.12" + === "Spark 4.0 and Scala 2.12" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-3.5_2.12</artifactId> + <artifactId>sedona-spark-4.0_2.12</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <dependency> @@ -226,11 +226,11 @@ The optional GeoTools library is required if you want to use CRS transformation, !!! abstract "Sedona with Apache Spark and Scala 2.13" - === "Spark 3.3 and Scala 2.13" + === "Spark 3.4 and Scala 2.13" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-3.3_2.13</artifactId> + <artifactId>sedona-spark-3.4_2.13</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <dependency> @@ -239,11 +239,11 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - === "Spark 3.4 and Scala 2.13" + === "Spark 3.5 and Scala 2.13" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-3.4_2.13</artifactId> + <artifactId>sedona-spark-3.5_2.13</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <dependency> @@ -252,11 +252,11 @@ The optional GeoTools library is required if you want to use CRS transformation, <version>{{ sedona.current_geotools }}</version> </dependency> ``` - === "Spark 3.5 and Scala 2.13" + === "Spark 4.0 and Scala 2.13" ```xml <dependency> <groupId>org.apache.sedona</groupId> - <artifactId>sedona-spark-3.5_2.13</artifactId> + <artifactId>sedona-spark-4.0_2.13</artifactId> <version>{{ sedona.current_version }}</version> </dependency> <dependency> diff --git a/docs/setup/platform.md b/docs/setup/platform.md index 1161825583..9ea2abe091 100644 --- a/docs/setup/platform.md +++ b/docs/setup/platform.md @@ -17,31 +17,33 @@ under the License. --> -Sedona binary releases are compiled by Java 1.8 and Scala 2.11/2.12 and tested in the following environments: +Sedona binary releases are compiled by Java 11/17 and Scala 2.12/2.13 and tested in the following environments: -!!!warning - Support of Spark 3.0, 3.1, 3.2 was removed in Sedona 1.7.0+ although some parts of the source code might still be compatible. +**Java Requirements:** + +- Spark 3.4 & 3.5: Java 11 +- Spark 4.0: Java 17 + +**Note:** Java 8 support is dropped since Sedona 1.8.0. Spark 3.3 support is dropped since Sedona 1.8.0. === "Sedona Scala/Java" - | | Spark 3.0 | Spark 3.1 | Spark 3.2| Spark 3.3| Spark 3.4| Spark 3.5 | - |:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:| - | Scala 2.11 | not tested | not tested | not tested |not tested |not tested |not tested | - | Scala 2.12 | not tested | not tested | not tested |✅ |✅ |✅ | - | Scala 2.13 | not tested | not tested | not tested|✅ |✅ |✅ | + | | Spark 3.4| Spark 3.5 | Spark 4.0 | + |:---------:|:---------:|:---------:|:---------:| + | Scala 2.12 |✅ |✅ |✅ | + | Scala 2.13 |✅ |✅ |✅ | === "Sedona Python" - | | Spark 3.0 (Scala 2.12)|Spark 3.1 (Scala 2.12)| Spark 3.2 (Scala 2.12)| Spark 3.3 (Scala 2.12)|Spark 3.4 (Scala 2.12)|Spark 3.5 (Scala 2.12)| - |:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:| - | Python 3.7 | not tested | not tested | not tested | ✅ | ✅ | ✅ | - | Python 3.8 | not tested |not tested |not tested | ✅ | ✅ | ✅ | - | Python 3.9 | not tested |not tested |not tested | ✅ | ✅ | ✅ | - | Python 3.10 | not tested |not tested |not tested | ✅ | ✅ | ✅ | + | | Spark 3.4 (Scala 2.12)|Spark 3.5 (Scala 2.12)| Spark 4.0 (Scala 2.12)| + |:---------:|:---------:|:---------:|:---------:| + | Python 3.7 | ✅ | ✅ | ✅ | + | Python 3.8 | ✅ | ✅ | ✅ | + | Python 3.9 | ✅ | ✅ | ✅ | + | Python 3.10 | ✅ | ✅ | ✅ | === "Sedona R" - | | Spark 3.0 | Spark 3.1 | Spark 3.2 | Spark 3.3 | Spark 3.4 | Spark 3.5 | - |:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:| - | Scala 2.11 | not tested | not tested | not tested | not tested |not tested |not tested | - | Scala 2.12 | not tested | not tested | not tested | ✅ | ✅ | ✅ | + | | Spark 3.4 | Spark 3.5 | Spark 4.0 | + |:---------:|:---------:|:---------:|:---------:| + | Scala 2.12 | ✅ | ✅ | ✅ | diff --git a/docs/setup/release-notes.md b/docs/setup/release-notes.md index 5ca5421113..5470027b92 100644 --- a/docs/setup/release-notes.md +++ b/docs/setup/release-notes.md @@ -17,6 +17,21 @@ under the License. --> +## Sedona 1.8.0 + +Sedona 1.8.0 is compiled against: + +- **Spark**: 3.4, 3.5, 4.0 +- **Flink**: 1.19 +- **Snowflake**: 7+ + +**Java Requirements:** + +- Spark 3.4 & 3.5: Java 11 +- Spark 4.0: Java 17 + +Spark 3.3 support is dropped since the 1.8.0 release. + ## Sedona 1.7.2 Sedona 1.7.2 is compiled against Spark 3.3 / Spark 3.4 / Spark 3.5, Flink 1.19, Snowflake 7+, Java 8.
