[
https://issues.apache.org/jira/browse/SPARK-58036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Javier Neira Sanchez updated SPARK-58036:
-----------------------------------------
Description:
Hi, as we can check in the source code of
[AggregatedDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala],
it tries to combine the dialects being aggregated but not for all mehods. For
those methods (f.e. compileValue) it relies in the default implementation of
[JdbcDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala],
dropping the implementation of the dialectes being aggregated.
We hit this because we registered a custom dialect for oracle and then the
compileValue method implementation of buitin oracle dialect was dropped,
causing error in sqls containing timestamp literals.
Ideally all methods of JdbcDialect interface should be overriden by the
Aggregate one, chosing the implementation of the first registered dialect if
the method returns a raw (not Option) value.
The workaround would be un-register the builtin jdbc dialect and make the
custom one replicate all its logic (including the methods returning Option)
Thanks in advance!
was:
Hi, as we can check in the source code of
[AggregatedDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala],
it tries to combine the dialects being aggregated but not for all mehods. For
those methods (f.e. compileValue) it relies in the default implementation of
[JdbcDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala],
dropping the implementation of the dialectes being aggregated.
We hit this because we registered a custom dialect for oracle and then the
compileValue method implementation of buitin oracle dialect was dropped,
causing error in sqls containing timestamp literals.
Ideally all methods of JdbcDialect interface should be overriden by the
Aggregate one, chosing the implementation if the method returns a raw (not
Option) value.
Thanks in advance!
> Aggregate jdbc dialect does not honour some methods (i.e compileValue)
> ----------------------------------------------------------------------
>
> Key: SPARK-58036
> URL: https://issues.apache.org/jira/browse/SPARK-58036
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.5.8, 4.2.0, 4.1.2, 4.0.3
> Reporter: Javier Neira Sanchez
> Priority: Minor
>
> Hi, as we can check in the source code of
> [AggregatedDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala],
> it tries to combine the dialects being aggregated but not for all mehods.
> For those methods (f.e. compileValue) it relies in the default implementation
> of
> [JdbcDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala],
> dropping the implementation of the dialectes being aggregated.
>
> We hit this because we registered a custom dialect for oracle and then the
> compileValue method implementation of buitin oracle dialect was dropped,
> causing error in sqls containing timestamp literals.
> Ideally all methods of JdbcDialect interface should be overriden by the
> Aggregate one, chosing the implementation of the first registered dialect if
> the method returns a raw (not Option) value.
>
> The workaround would be un-register the builtin jdbc dialect and make the
> custom one replicate all its logic (including the methods returning Option)
>
> Thanks in advance!
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
