[
https://issues.apache.org/jira/browse/SPARK-58036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Javier Neira Sanchez updated SPARK-58036:
-----------------------------------------
Description:
Hi, as we can check in the source code of
[AggregatedDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala],
it tries to combine the dialects being aggregated but not for all mehods. For
those methods (f.e. compileValue) it relies in the default implementation of
[JdbcDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala],
dropping the implementation of the dialectes being aggregated.
We hit this because we registered a custom dialect for oracle and then the
compileValue method implementation of buitin oracle dialect was dropped,
causing error in sqls containing timestamp literals.
Ideally all methods of JdbcDialect interface should be overriden by the
Aggregate one, chosing the implementation if the method returns a raw (not
Option) value.
Thanks in advance!
was:
Hi, as we can check in the source code of
[AggregatedDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala],
it tries to combine the dialects being aggregated but not for all mehods. For
those methods it relies in the default implementation of
[JdbcDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala],
sropping the implementation of the dialectes being aggregated.
We hit this because we registered a custom dialect for oracle and then the
compileValue method implementation of buitin oracle dialect was dropped,
causing error in sqls containing timestamp literals.
Ideally all methods of JdbcDialect interface should be overriden by the
Aggregate one, chosing the implementation if the method returns a raw (not
Option) value.
Thanks in advance!
> Aggregate jdbc dialect does not honour some methods (i.e compileValue)
> ----------------------------------------------------------------------
>
> Key: SPARK-58036
> URL: https://issues.apache.org/jira/browse/SPARK-58036
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.5.8, 4.2.0, 4.1.2, 4.0.3
> Reporter: Javier Neira Sanchez
> Priority: Minor
>
> Hi, as we can check in the source code of
> [AggregatedDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala],
> it tries to combine the dialects being aggregated but not for all mehods.
> For those methods (f.e. compileValue) it relies in the default implementation
> of
> [JdbcDialect|https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala],
> dropping the implementation of the dialectes being aggregated.
>
> We hit this because we registered a custom dialect for oracle and then the
> compileValue method implementation of buitin oracle dialect was dropped,
> causing error in sqls containing timestamp literals.
> Ideally all methods of JdbcDialect interface should be overriden by the
> Aggregate one, chosing the implementation if the method returns a raw (not
> Option) value.
>
> Thanks in advance!
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
