Niranjan Jayakar created SPARK-51817:
----------------------------------------
Summary: [CONNECT][SCALA] Removing ansiConfig parameter in
error-class causes client to be incompatible
Key: SPARK-51817
URL: https://issues.apache.org/jira/browse/SPARK-51817
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 4.0.0
Reporter: Niranjan Jayakar
In Spark Connect, we guarantee that older clients are compatible with newer
versions of the Spark Connect service.
A previous change[1] broke this compatibility by removing the "ansiConfig"
field in the message parameters for two error codes - "CAST_OVERFLOW" and
"CAST_INVALID_INPUT".
The Spark Connect client includes GrpcExceptionConverter.scala[2] to convert
error codes from the server to produce SQL compliant error codes on the client.
This uses the error-conditions.json file. Older clients do not include the
change[1] to this file and still include the `ansiConfig` parameter. Later
versions of the Spark Connect service don't return this parameter resulting in
an internal error[3] that the correct error condition could not be formulated.
[1]:
[https://github.com/apache/spark/commit/e28c33bcf784b8a5b7cf29c2527ce78e4a3ada0f|https://github.com/apache/spark/commit/e28c33bcf784b8a5b7cf29c2527ce78e4a3ada0f#diff-34e280062b2b58a677fe8fae16f67bfff596e80f59f61aeea6743dd9903eb7f0]
[2]:
[https://github.com/apache/spark/blob/2ba156096e83adf7b0b2f5c38453d6fd37d95ded/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/GrpcExceptionConverter.scala#L184]
[3]:
https://github.com/apache/spark/blob/2ba156096e83adf7b0b2f5c38453d6fd37d95ded/common/utils/src/main/scala/org/apache/spark/ErrorClassesJSONReader.scala#L58
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
