[jira] [Created] (SPARK-36800) The create table as select statement verifies the valid column name

2021-09-19 Thread melin (Jira)
melin created SPARK-36800: - Summary: The create table as select statement verifies the valid column name Key: SPARK-36800 URL: https://issues.apache.org/jira/browse/SPARK-36800 Project: Spark Issue

[jira] [Commented] (SPARK-24907) Migrate JDBC data source to DataSource API v2

2021-11-08 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17440893#comment-17440893 ] melin commented on SPARK-24907: --- add presql and postsql  presql: Run the presql before th

[jira] [Commented] (SPARK-24907) Migrate JDBC data source to DataSource API v2

2021-11-08 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17440894#comment-17440894 ] melin commented on SPARK-24907: --- SaveMode > Migrate JDBC data source to DataSource API v2

[jira] (SPARK-24907) Migrate JDBC data source to DataSource API v2

2021-11-08 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24907 ] melin deleted comment on SPARK-24907: --- was (Author: melin): SaveMode > Migrate JDBC data source to DataSource API v2 > - > > Key: SPARK-

[jira] [Commented] (SPARK-24907) Migrate JDBC data source to DataSource API v2

2021-11-08 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17440895#comment-17440895 ] melin commented on SPARK-24907: --- SaveMode.Overwrite,  Support for the mysql replace into t

[jira] [Commented] (SPARK-24907) Migrate JDBC data source to DataSource API v2

2021-11-16 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17444916#comment-17444916 ] melin commented on SPARK-24907: --- The hive table data is written to a relational database,

[jira] (SPARK-24907) Migrate JDBC data source to DataSource API v2

2021-11-16 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24907 ] melin deleted comment on SPARK-24907: --- was (Author: melin): The hive table data is written to a relational database, generally an online production database. If the writing speed has no traffic co

[jira] [Created] (SPARK-37351) Supports write data flow control

2021-11-16 Thread melin (Jira)
melin created SPARK-37351: - Summary: Supports write data flow control Key: SPARK-37351 URL: https://issues.apache.org/jira/browse/SPARK-37351 Project: Spark Issue Type: Sub-task Components:

[jira] [Created] (SPARK-37607) Optimized antlr4 syntax file

2021-12-09 Thread melin (Jira)
melin created SPARK-37607: - Summary: Optimized antlr4 syntax file Key: SPARK-37607 URL: https://issues.apache.org/jira/browse/SPARK-37607 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-37607) Refactor antlr4 syntax file

2021-12-13 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-37607: -- Description: Refer to the ShardingSphere project, split into multiple files by type, very clear [https://gith

[jira] [Updated] (SPARK-37607) Refactor antlr4 syntax file

2021-12-13 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-37607: -- Description: Refer to the ShardingSphere project, split into multiple files by type, very clear https://gi

[jira] [Created] (SPARK-39303) timestamp type support precision, example: timestamp(3)

2022-05-26 Thread melin (Jira)
melin created SPARK-39303: - Summary: timestamp type support precision, example: timestamp(3) Key: SPARK-39303 URL: https://issues.apache.org/jira/browse/SPARK-39303 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-39303) timestamp type support precision, example: timestamp(3)

2022-05-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17542448#comment-17542448 ] melin commented on SPARK-39303: --- [~Qin Yao]  > timestamp type support precision, example:

[jira] [Updated] (SPARK-39303) timestamp type support precision, example: timestamp(3)

2022-05-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-39303: -- Description: Dremio supports syntax: select CAST(-123456789 AS TIMESTAMP(3)) as times; Dremio uses calcite,Th

[jira] [Updated] (SPARK-39303) timestamp type support precision, example: timestamp(3)

2022-05-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-39303: -- Description: Dremio supports syntax: select CAST(-123456789 AS TIMESTAMP(3)) as times,spark sql cannot specif

[jira] [Created] (SPARK-39378) Customize metastore and catalog, use ParqurtTable, Cannot directly use CTAS and Insert SQL

2022-06-03 Thread melin (Jira)
melin created SPARK-39378: - Summary: Customize metastore and catalog, use ParqurtTable, Cannot directly use CTAS and Insert SQL Key: SPARK-39378 URL: https://issues.apache.org/jira/browse/SPARK-39378 Project

[jira] [Commented] (SPARK-39378) Customize metastore and catalog, use ParqurtTable, Cannot directly use CTAS and Insert SQL

2022-06-04 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547918#comment-17547918 ] melin commented on SPARK-39378: --- The following two errors: {code:java} String sql = "CREAT

[jira] [Created] (SPARK-39462) [Support]add contains function

2022-06-13 Thread melin (Jira)
melin created SPARK-39462: - Summary: [Support]add contains function Key: SPARK-39462 URL: https://issues.apache.org/jira/browse/SPARK-39462 Project: Spark Issue Type: Improvement Components

[jira] [Created] (SPARK-39478) Support timestampdiff function

2022-06-14 Thread melin (Jira)
melin created SPARK-39478: - Summary: Support timestampdiff function Key: SPARK-39478 URL: https://issues.apache.org/jira/browse/SPARK-39478 Project: Spark Issue Type: Improvement Components

[jira] [Created] (SPARK-39498) Support function: percentile_approx、percentile_count、percentile_disc

2022-06-16 Thread melin (Jira)
melin created SPARK-39498: - Summary: Support function: percentile_approx、percentile_count、percentile_disc Key: SPARK-39498 URL: https://issues.apache.org/jira/browse/SPARK-39498 Project: Spark Issue

[jira] [Created] (SPARK-39499) Substring function support pattern expression

2022-06-16 Thread melin (Jira)
melin created SPARK-39499: - Summary: Substring function support pattern expression Key: SPARK-39499 URL: https://issues.apache.org/jira/browse/SPARK-39499 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-39499) Substring function support pattern expression

2022-06-16 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin resolved SPARK-39499. --- Resolution: Duplicate > Substring function support pattern expression >

[jira] [Commented] (SPARK-39498) Support function: percentile_approx、percentile_count、percentile_disc

2022-06-17 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555806#comment-17555806 ] melin commented on SPARK-39498: --- redshit, dremio has these two functions > Support functi

[jira] [Commented] (SPARK-39478) Support timestampdiff function

2022-06-17 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555807#comment-17555807 ] melin commented on SPARK-39478: --- mysql, dremio has these two functions > Support timestam

[jira] [Created] (SPARK-37840) Dynamically update the loaded Spark UDF JAR

2022-01-07 Thread melin (Jira)
melin created SPARK-37840: - Summary: Dynamically update the loaded Spark UDF JAR Key: SPARK-37840 URL: https://issues.apache.org/jira/browse/SPARK-37840 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37840) Dynamically update the loaded Hive UDF JAR

2022-01-07 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-37840: -- Summary: Dynamically update the loaded Hive UDF JAR (was: Dynamically update the loaded Spark UDF JAR) > Dyn

[jira] [Commented] (SPARK-37840) Dynamically update the loaded Spark UDF JAR

2022-01-07 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17470446#comment-17470446 ] melin commented on SPARK-37840: --- [~cloud_fan]  > Dynamically update the loaded Spark UDF

[jira] [Created] (SPARK-37848) Supports counting the number of records written to each partition when inserting partitioned tables

2022-01-09 Thread melin (Jira)
melin created SPARK-37848: - Summary: Supports counting the number of records written to each partition when inserting partitioned tables Key: SPARK-37848 URL: https://issues.apache.org/jira/browse/SPARK-37848

[jira] [Created] (SPARK-37849) Supports limiting the maximum amount of column data

2022-01-09 Thread melin (Jira)
melin created SPARK-37849: - Summary: Supports limiting the maximum amount of column data Key: SPARK-37849 URL: https://issues.apache.org/jira/browse/SPARK-37849 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-37848) Supports counting the number of records written to each partition when inserting partitioned tables

2022-01-09 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-37848: -- Attachment: image-2022-01-10-09-10-47-483.png > Supports counting the number of records written to each partit

[jira] [Updated] (SPARK-37848) Supports counting the number of records written to each partition when inserting partitioned tables

2022-01-09 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-37848: -- Description: InputMetrics and OutputMetrics have no table name information. Write information can be counted

[jira] [Commented] (SPARK-37849) Supports limiting the maximum amount of column data

2022-01-09 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17471633#comment-17471633 ] melin commented on SPARK-37849: --- Restrictions can be added in HiveOutputWriter {code:java}

[jira] [Commented] (SPARK-37840) Dynamically update the loaded Hive UDF JAR

2022-01-11 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472833#comment-17472833 ] melin commented on SPARK-37840: --- If the JAR file update time changes, create a new classlo

[jira] [Created] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-14 Thread melin (Jira)
melin created SPARK-38200: - Summary: [SQL] Spark JDBC Savemode Supports replace Key: SPARK-38200 URL: https://issues.apache.org/jira/browse/SPARK-38200 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-14 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Description: When writing data into a relational database, data duplication needs to be considered. Both mysq

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-14 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17491853#comment-17491853 ] melin commented on SPARK-38200: --- [~beliefer]  > [SQL] Spark JDBC Savemode Supports replac

[jira] [Created] (SPARK-38209) Selectively include EXTERNAL TABLE source files via REGEX

2022-02-14 Thread melin (Jira)
melin created SPARK-38209: - Summary: Selectively include EXTERNAL TABLE source files via REGEX Key: SPARK-38209 URL: https://issues.apache.org/jira/browse/SPARK-38209 Project: Spark Issue Type: Impro

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-16 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Description: When writing data into a relational database, data duplication needs to be considered. Both mysq

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-16 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493642#comment-17493642 ] melin commented on SPARK-38200: --- [~beliefer]  Upsert SQL is not ANSI standard, Mysql and P

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-20 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17495322#comment-17495322 ] melin commented on SPARK-38200: --- [~beliefer]   > [SQL] Spark JDBC Savemode Supports repla

[jira] [Comment Edited] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-20 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17495322#comment-17495322 ] melin edited comment on SPARK-38200 at 2/21/22, 5:17 AM: - [~beli

[jira] [Comment Edited] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2022-02-20 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17495322#comment-17495322 ] melin edited comment on SPARK-38200 at 2/21/22, 5:17 AM: - [~beli

[jira] [Created] (SPARK-38359) Support Create/Drop/Alter Catalog

2022-02-28 Thread melin (Jira)
melin created SPARK-38359: - Summary: Support Create/Drop/Alter Catalog Key: SPARK-38359 URL: https://issues.apache.org/jira/browse/SPARK-38359 Project: Spark Issue Type: Improvement Compone

[jira] [Updated] (SPARK-38359) Support Create/Drop/Alter Catalog SQL

2022-02-28 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38359: -- Summary: Support Create/Drop/Alter Catalog SQL (was: Support Create/Drop/Alter Catalog) > Support Create/Dro

[jira] [Created] (SPARK-47389) spark jdbc one insert with multiple values

2024-03-14 Thread melin (Jira)
melin created SPARK-47389: - Summary: spark jdbc one insert with multiple values Key: SPARK-47389 URL: https://issues.apache.org/jira/browse/SPARK-47389 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-48404) Driver and Executor support merge and run in a single jvm

2024-05-23 Thread melin (Jira)
melin created SPARK-48404: - Summary: Driver and Executor support merge and run in a single jvm Key: SPARK-48404 URL: https://issues.apache.org/jira/browse/SPARK-48404 Project: Spark Issue Type: Impro

[jira] [Updated] (SPARK-48450) Support Jdbc datasource custom data partitioning

2024-05-28 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-48450: -- Description: "partitionColumn, lowerBound, upperBound" is not an efficient table partitioning scheme for some

[jira] [Updated] (SPARK-48450) Support Jdbc datasource custom data partitioning

2024-05-28 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-48450: -- Description: "partitionColumn, lowerBound, upperBound" is not an efficient table partitioning scheme for some

[jira] [Commented] (SPARK-48286) Analyze 'exists' default expression instead of 'current' default expression in structField to v2 column conversion

2024-06-06 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852984#comment-17852984 ] melin commented on SPARK-48286: --- defaultValueNotConstantError method not exists > Analyze

[jira] [Comment Edited] (SPARK-48286) Analyze 'exists' default expression instead of 'current' default expression in structField to v2 column conversion

2024-06-11 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852984#comment-17852984 ] melin edited comment on SPARK-48286 at 6/11/24 2:51 PM: defaultV

[jira] [Comment Edited] (SPARK-48286) Analyze 'exists' default expression instead of 'current' default expression in structField to v2 column conversion

2024-06-11 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17852984#comment-17852984 ] melin edited comment on SPARK-48286 at 6/11/24 2:51 PM: defaultV

[jira] [Updated] (SPARK-45140) Support ddl output json format

2024-07-04 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-45140: -- Description: hive supports ddl output json format. set hive.ddl.output.format=json; hive table metadata is co

[jira] [Commented] (SPARK-45140) Support ddl output json format

2024-07-04 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17863125#comment-17863125 ] melin commented on SPARK-45140: --- cc [~gurwls223] ,  > Support ddl output json format > --

[jira] [Created] (SPARK-48867) Upgrade the kubernetes-client dependency okhttp version 4.12

2024-07-10 Thread melin (Jira)
melin created SPARK-48867: - Summary: Upgrade the kubernetes-client dependency okhttp version 4.12 Key: SPARK-48867 URL: https://issues.apache.org/jira/browse/SPARK-48867 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-48867) Upgrade the kubernetes-client dependency okhttp version 4.12

2024-07-10 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864931#comment-17864931 ] melin commented on SPARK-48867: --- cc [~panbingkun]  > Upgrade the kubernetes-client depend

[jira] [Updated] (SPARK-48867) Upgrade the kubernetes-client dependency okhttp version 4.12

2024-07-10 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-48867: -- Description: Right now Fabric8 Kubernetes Client doesn't work with IPv6 based Kubernetes Clusters due to an i

[jira] [Commented] (SPARK-48867) Upgrade the kubernetes-client dependency okhttp version 4.12

2024-07-11 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864955#comment-17864955 ] melin commented on SPARK-48867: --- ok, let me try, esdk-obs-java 3.21.8 version has been upg

[jira] [Comment Edited] (SPARK-48867) Upgrade the kubernetes-client dependency okhttp version 4.12

2024-07-11 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864955#comment-17864955 ] melin edited comment on SPARK-48867 at 7/11/24 8:09 AM: ok, let

[jira] [Commented] (SPARK-48867) Upgrade the kubernetes-client dependency okhttp version 4.12

2024-07-11 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864994#comment-17864994 ] melin commented on SPARK-48867: --- I first initiated the hadoop-huaweicloud project to upgra

[jira] [Commented] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

2023-07-17 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17743701#comment-17743701 ] melin commented on SPARK-43338: --- [~yao] Would consider setting a custom catalog name?  ex

[jira] [Created] (SPARK-45140) Support ddl output json format

2023-09-12 Thread melin (Jira)
melin created SPARK-45140: - Summary: Support ddl output json format Key: SPARK-45140 URL: https://issues.apache.org/jira/browse/SPARK-45140 Project: Spark Issue Type: New Feature Components

[jira] [Updated] (SPARK-45140) Support ddl output json format

2023-09-12 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-45140: -- Environment:         was: hive supports ddl output json format. set hive.ddl.output.format=json; hive tab

[jira] [Updated] (SPARK-45140) Support ddl output json format

2023-09-12 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-45140: -- Description: hive supports ddl output json format. set hive.ddl.output.format=json; hive table metadata is co

[jira] [Updated] (SPARK-45140) Support ddl output json format

2023-09-12 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-45140: -- Description: hive supports ddl output json format. set hive.ddl.output.format=json; hive table metadata is co

[jira] [Commented] (SPARK-45140) Support ddl output json format

2023-09-12 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764488#comment-17764488 ] melin commented on SPARK-45140: --- We have a product that uses jdbc to collect metadata from

[jira] [Created] (SPARK-45818) Support columnar/vectorized evaluation engine JDK17/21

2023-11-07 Thread melin (Jira)
melin created SPARK-45818: - Summary: Support columnar/vectorized evaluation engine JDK17/21 Key: SPARK-45818 URL: https://issues.apache.org/jira/browse/SPARK-45818 Project: Spark Issue Type: New Feat

[jira] [Created] (SPARK-46195) Supports parse multiple sql statements

2023-11-30 Thread melin (Jira)
melin created SPARK-46195: - Summary: Supports parse multiple sql statements Key: SPARK-46195 URL: https://issues.apache.org/jira/browse/SPARK-46195 Project: Spark Issue Type: New Feature Co

[jira] [Commented] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

2023-12-14 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17796975#comment-17796975 ] melin commented on SPARK-43338: --- [~yao]   databricks support change:    spark.databricks.s

[jira] [Created] (SPARK-46511) Optimize spark jdbc write speed with Multi-Row Inserts

2023-12-25 Thread melin (Jira)
melin created SPARK-46511: - Summary: Optimize spark jdbc write speed with Multi-Row Inserts Key: SPARK-46511 URL: https://issues.apache.org/jira/browse/SPARK-46511 Project: Spark Issue Type: New Feat

[jira] [Updated] (SPARK-46511) Optimize spark jdbc write speed with Multi-Row Inserts

2023-12-25 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-46511: -- Description: [https://brian.pontarelli.com/2011/06/21/jdbc-batch-vs-multi-row-inserts/] mysql, pg、Oracle 23c,

[jira] [Updated] (SPARK-46511) Optimize spark jdbc write speed with Multi-Row Inserts

2023-12-25 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-46511: -- Description: [https://brian.pontarelli.com/2011/06/21/jdbc-batch-vs-multi-row-inserts/] mysql, pg、Oracle 23c,

[jira] [Resolved] (SPARK-46511) Optimize spark jdbc write speed with Multi-Row Inserts

2023-12-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin resolved SPARK-46511. --- Resolution: Fixed > Optimize spark jdbc write speed with Multi-Row Inserts > ---

[jira] [Created] (SPARK-46518) Support for copy from write compatible postgresql databases (pg, redshift, snowflake, gauss)

2023-12-26 Thread melin (Jira)
melin created SPARK-46518: - Summary: Support for copy from write compatible postgresql databases (pg, redshift, snowflake, gauss) Key: SPARK-46518 URL: https://issues.apache.org/jira/browse/SPARK-46518 Projec

[jira] [Updated] (SPARK-46518) Support for copy from write compatible postgresql databases (pg, redshift, snowflake, gauss)

2023-12-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-46518: -- Attachment: image-2023-12-27-09-44-19-292.png > Support for copy from write compatible postgresql databases (p

[jira] [Updated] (SPARK-46518) Support for copy from write compatible postgresql databases (pg, redshift, snowflake, gauss)

2023-12-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-46518: -- Description: Now many databases are compatible with pg syntax and support copy from syntax. The copy form imp

[jira] [Updated] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

2023-12-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-43338: -- Attachment: image-2023-12-27-09-55-55-693.png > Support modify the SESSION_CATALOG_NAME value > -

[jira] [Updated] (SPARK-43338) Support modify the SESSION_CATALOG_NAME value

2023-12-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-43338: -- Description: {code:java} private[sql] object CatalogManager { val SESSION_CATALOG_NAME: String = "spark_catalo

[jira] [Created] (SPARK-46572) [SQL][Enhancement] hint enhancement

2024-01-02 Thread melin (Jira)
melin created SPARK-46572: - Summary: [SQL][Enhancement] hint enhancement Key: SPARK-46572 URL: https://issues.apache.org/jira/browse/SPARK-46572 Project: Spark Issue Type: New Feature Compo

[jira] [Created] (SPARK-47114) In the spark driver pod. Failed to access the krb5 file

2024-02-20 Thread melin (Jira)
melin created SPARK-47114: - Summary: In the spark driver pod. Failed to access the krb5 file Key: SPARK-47114 URL: https://issues.apache.org/jira/browse/SPARK-47114 Project: Spark Issue Type: New Fea

[jira] [Updated] (SPARK-47114) In the spark driver pod. Failed to access the krb5 file

2024-02-20 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-47114: -- Description: spark runs in kubernetes and accesses an external hdfs cluster (kerberos),pod error logs {code:j

[jira] [Updated] (SPARK-47114) In the spark driver pod. Failed to access the krb5 file

2024-02-20 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-47114: -- Description: spark runs in kubernetes and accesses an external hdfs cluster (kerberos),pod error logs {code:j

[jira] [Updated] (SPARK-47114) In the spark driver pod. Failed to access the krb5 file

2024-02-20 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-47114: -- Description: spark runs in kubernetes and accesses an external hdfs cluster (kerberos),pod error logs {code:j

[jira] [Updated] (SPARK-47114) In the spark driver pod. Failed to access the krb5 file

2024-02-21 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-47114: -- Description: spark runs in kubernetes and accesses an external hdfs cluster (kerberos),pod error logs {code:j

[jira] [Commented] (SPARK-47114) In the spark driver pod. Failed to access the krb5 file

2024-02-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17821469#comment-17821469 ] melin commented on SPARK-47114: --- 默认jre17 不支持kerberos,换成jdk  > In the spark driver pod. Fa

[jira] [Resolved] (SPARK-47114) In the spark driver pod. Failed to access the krb5 file

2024-02-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin resolved SPARK-47114. --- Resolution: Resolved > In the spark driver pod. Failed to access the krb5 file > ---

[jira] [Created] (SPARK-47198) Is it possible to dynamically add backend service to ingress with Kubernetes?

2024-02-27 Thread melin (Jira)
melin created SPARK-47198: - Summary: Is it possible to dynamically add backend service to ingress with Kubernetes? Key: SPARK-47198 URL: https://issues.apache.org/jira/browse/SPARK-47198 Project: Spark

[jira] [Updated] (SPARK-47198) Is it possible to dynamically add backend service to ingress with Kubernetes?

2024-02-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-47198: -- Description: spark on k8s runs multiple spark apps at the same time. proxy/[sparkappid] path, forwarding to d

[jira] [Updated] (SPARK-47198) Is it possible to dynamically add backend service to ingress with Kubernetes?

2024-02-29 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-47198: -- Description: spark on k8s runs multiple spark apps at the same time. proxy/[sparkappid] path, forwarding to d

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2023-03-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705765#comment-17705765 ] melin commented on SPARK-38200: --- upsert sql for different databases, Most databases suppor

[jira] [Comment Edited] (SPARK-38200) [SQL] Spark JDBC Savemode Supports replace

2023-03-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705765#comment-17705765 ] melin edited comment on SPARK-38200 at 3/28/23 3:00 AM: upsert s

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-03-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Summary: [SQL] Spark JDBC Savemode Supports Upsert (was: [SQL] Spark JDBC Savemode Supports replace) > [SQL]

[jira] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-03-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200 ] melin deleted comment on SPARK-38200: --- was (Author: melin): upsert sql for different databases, Most databases support merge sql: sqlserver merge into sql : [https://github.com/apache/incubator-s

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-03-27 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Description: upsert sql for different databases, Most databases support merge sql: sqlserver merge into sql :

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-04-01 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Description: upsert sql for different databases, Most databases support merge sql: sqlserver merge into sql :

[jira] [Commented] (SPARK-42627) Spark: Getting SQLException: Unsupported type -102 reading from Oracle

2023-04-01 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707619#comment-17707619 ] melin commented on SPARK-42627: --- not support type: TIMESTAMP (3) WITH LOCAL TIME ZONE NOT

[jira] [Comment Edited] (SPARK-42627) Spark: Getting SQLException: Unsupported type -102 reading from Oracle

2023-04-01 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707619#comment-17707619 ] melin edited comment on SPARK-42627 at 4/2/23 6:50 AM: --- not suppor

[jira] [Updated] (SPARK-42627) Spark: Getting SQLException: Unsupported type -102 reading from Oracle

2023-04-01 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-42627: -- Description:   {code:java} Exception in thread "main" org.apache.spark.SparkSQLException: Unrecognized SQL ty

[jira] [Updated] (SPARK-42627) Spark: Getting SQLException: Unsupported type -102 reading from Oracle

2023-04-06 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-42627: -- Description:   {code:java} Exception in thread "main" org.apache.spark.SparkSQLException: Unrecognized SQL ty

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-04-06 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Description: upsert sql for different databases, Most databases support merge sql: sqlserver merge into sql :

[jira] [Updated] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

2023-04-06 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated SPARK-38200: -- Description: upsert sql for different databases, Most databases support merge sql: sqlserver merge into sql :

  1   2   >