[
https://issues.apache.org/jira/browse/SPARK-51498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shreyas updated SPARK-51498:
----------------------------
Description:
The JDBC API has *getUpdateCount* and *getLargeUpdateCount* functions that
support clients to obtain the rows affected by the SQL operation.
Spark operations already capture the # of rows affected by a SQL operation,
which are available in the DAGs. It would be nice if Spark Thrift Server could
pass on this information and make it available to the getUpdateCount and
getLargeUpdateCount APIs, so that JDBC clients can make use of the same.
For e.g. Trino provides this information, which can be utilized by the clients
([SQLRunner |https://github.com/profphreak/sqlrunner/tree/master]below is a
command line JDBC client):
{code:java}
$ echo "insert into tbl select 1 union all select 2" | sqlrunner db=trino log -
--SQL START: Thu Mar 13 08:17:18 EDT 2025
-----------------------------------
insert into tbl select 1 union all select 2
;
-----------------------------------
-- driverjar is not defined.
-- driver class: io.trino.jdbc.TrinoDriver
-- connecting to: jdbc:trino://xxx
--SQL END: Thu Mar 13 08:17:21 EDT 2025; 0.04 min
-- UpdateCount: 2
-----------------------------------
-- File - completed in: 0.04 minutes.
-- Total SQLRunner execution time: 0.04 minutes.
--closing connection to: io.trino.jdbc.TrinoConnection@797cf65c
{code}
was:
The JDBC API has *getUpdateCount* and *getLargeUpdateCount* functions that
support clients to obtain the rows affected by the SQL operation.
Spark operations already capture the # of rows affected by a SQL operation,
which are available in the DAGs. It would be nice if Spark Thrift Server could
pass on this information and make it available to the getUpdateCount and
getLargeUpdateCount APIs, so that JDBC clients can make use of the same.
For e.g. Trino provides this information, which can be utilized by the clients
([SQLRunner |https://github.com/profphreak/sqlrunner/tree/master]below is a
command line JDBC client):
{code:java}
$ echo "insert into mrp_adhoc.t select 1 union all select 2" | sqlrunner
db=trino log -
--SQL START: Thu Mar 13 08:17:18 EDT 2025
-----------------------------------
insert into tbl select 1 union all select 2
;
-----------------------------------
-- driverjar is not defined.
-- driver class: io.trino.jdbc.TrinoDriver
-- connecting to: jdbc:trino://xxx
--SQL END: Thu Mar 13 08:17:21 EDT 2025; 0.04 min
-- UpdateCount: 2
-----------------------------------
-- File - completed in: 0.04 minutes.
-- Total SQLRunner execution time: 0.04 minutes.
--closing connection to: io.trino.jdbc.TrinoConnection@797cf65c
{code}
> Implement getUpdateCount & getLargeUpdateCount in Thrift Server
> ---------------------------------------------------------------
>
> Key: SPARK-51498
> URL: https://issues.apache.org/jira/browse/SPARK-51498
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.5.5
> Reporter: Shreyas
> Priority: Major
>
> The JDBC API has *getUpdateCount* and *getLargeUpdateCount* functions that
> support clients to obtain the rows affected by the SQL operation.
> Spark operations already capture the # of rows affected by a SQL operation,
> which are available in the DAGs. It would be nice if Spark Thrift Server
> could pass on this information and make it available to the getUpdateCount
> and getLargeUpdateCount APIs, so that JDBC clients can make use of the same.
> For e.g. Trino provides this information, which can be utilized by the
> clients ([SQLRunner
> |https://github.com/profphreak/sqlrunner/tree/master]below is a command line
> JDBC client):
> {code:java}
> $ echo "insert into tbl select 1 union all select 2" | sqlrunner db=trino log
> -
> --SQL START: Thu Mar 13 08:17:18 EDT 2025
> -----------------------------------
> insert into tbl select 1 union all select 2
> ;
> -----------------------------------
> -- driverjar is not defined.
> -- driver class: io.trino.jdbc.TrinoDriver
> -- connecting to: jdbc:trino://xxx
> --SQL END: Thu Mar 13 08:17:21 EDT 2025; 0.04 min
> -- UpdateCount: 2
> -----------------------------------
> -- File - completed in: 0.04 minutes.
> -- Total SQLRunner execution time: 0.04 minutes.
> --closing connection to: io.trino.jdbc.TrinoConnection@797cf65c
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
