This is an automated email from the ASF dual-hosted git repository. jiayu pushed a commit to branch prepare-1.7.2 in repository https://gitbox.apache.org/repos/asf/sedona.git
commit 657325d65013808cea72086a9164a1d0d7162688 Author: Jia Yu <[email protected]> AuthorDate: Tue Mar 25 19:06:57 2025 -0700 [DOCS] Update the Databricks doc to reflect issues on DBR 16 (#1877) --- docs/setup/databricks.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/setup/databricks.md b/docs/setup/databricks.md index d0a8d332d9..ff51a9eb70 100644 --- a/docs/setup/databricks.md +++ b/docs/setup/databricks.md @@ -19,8 +19,10 @@ In Databricks advanced editions, you need to install Sedona via [cluster init-scripts](https://docs.databricks.com/clusters/init-scripts.html) as described below. Sedona is not guaranteed to be 100% compatible with `Databricks photon acceleration`. Sedona requires Spark internal APIs to inject many optimization strategies, which sometimes is not accessible in `Photon`. -!!!note - The following steps use DBR including Apache Spark 3.4.x as an example. Please change the Spark version according to your DBR version. Please pay attention to the Spark version postfix and Scala version postfix on our [Maven Coordinate page](maven-coordinates.md). Databricks Spark and Apache Spark's compatibility can be found [here](https://docs.databricks.com/en/release-notes/runtime/index.html). +The following steps use DBR including Apache Spark 3.4.x as an example. Please change the Spark version according to your DBR version. Please pay attention to the Spark version postfix and Scala version postfix on our [Maven Coordinate page](maven-coordinates.md). Databricks Spark and Apache Spark's compatibility can be found [here](https://docs.databricks.com/en/release-notes/runtime/index.html). + +!!! bug + Databricks Runtime 16.2 (non-LTS) introduces a change in the json4s dependency, which may lead to compatibility issues with Apache Sedona. We recommend using a currently supported LTS version, such as Databricks Runtime 15.4 LTS or 14.3 LTS, to ensure stability. A patch will be provided once an official Databricks Runtime 16 LTS version is released. ### Download Sedona jars @@ -75,6 +77,7 @@ From your cluster configuration (`Cluster` -> `Edit` -> `Configuration` -> `Adva spark.sql.extensions org.apache.sedona.viz.sql.SedonaVizExtensions,org.apache.sedona.sql.SedonaSqlExtensions spark.serializer org.apache.spark.serializer.KryoSerializer spark.kryo.registrator org.apache.sedona.core.serde.SedonaKryoRegistrator +spark.sedona.enableParserExtension false ``` From your cluster configuration (`Cluster` -> `Edit` -> `Configuration` -> `Advanced options` -> `Init Scripts`) add the newly created `Workspace` init script
