This is an automated email from the ASF dual-hosted git repository. jiayu pushed a commit to branch dbr-1602 in repository https://gitbox.apache.org/repos/asf/sedona.git
commit ddea0f717a71c3823d044365aa4473b53a1aae20 Author: Jia Yu <[email protected]> AuthorDate: Tue Mar 25 19:01:01 2025 -0700 Update the Databricks doc --- docs/setup/databricks.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/setup/databricks.md b/docs/setup/databricks.md index d0a8d332d9..ff51a9eb70 100644 --- a/docs/setup/databricks.md +++ b/docs/setup/databricks.md @@ -19,8 +19,10 @@ In Databricks advanced editions, you need to install Sedona via [cluster init-scripts](https://docs.databricks.com/clusters/init-scripts.html) as described below. Sedona is not guaranteed to be 100% compatible with `Databricks photon acceleration`. Sedona requires Spark internal APIs to inject many optimization strategies, which sometimes is not accessible in `Photon`. -!!!note - The following steps use DBR including Apache Spark 3.4.x as an example. Please change the Spark version according to your DBR version. Please pay attention to the Spark version postfix and Scala version postfix on our [Maven Coordinate page](maven-coordinates.md). Databricks Spark and Apache Spark's compatibility can be found [here](https://docs.databricks.com/en/release-notes/runtime/index.html). +The following steps use DBR including Apache Spark 3.4.x as an example. Please change the Spark version according to your DBR version. Please pay attention to the Spark version postfix and Scala version postfix on our [Maven Coordinate page](maven-coordinates.md). Databricks Spark and Apache Spark's compatibility can be found [here](https://docs.databricks.com/en/release-notes/runtime/index.html). + +!!! bug + Databricks Runtime 16.2 (non-LTS) introduces a change in the json4s dependency, which may lead to compatibility issues with Apache Sedona. We recommend using a currently supported LTS version, such as Databricks Runtime 15.4 LTS or 14.3 LTS, to ensure stability. A patch will be provided once an official Databricks Runtime 16 LTS version is released. ### Download Sedona jars @@ -75,6 +77,7 @@ From your cluster configuration (`Cluster` -> `Edit` -> `Configuration` -> `Adva spark.sql.extensions org.apache.sedona.viz.sql.SedonaVizExtensions,org.apache.sedona.sql.SedonaSqlExtensions spark.serializer org.apache.spark.serializer.KryoSerializer spark.kryo.registrator org.apache.sedona.core.serde.SedonaKryoRegistrator +spark.sedona.enableParserExtension false ``` From your cluster configuration (`Cluster` -> `Edit` -> `Configuration` -> `Advanced options` -> `Init Scripts`) add the newly created `Workspace` init script
