[
https://issues.apache.org/jira/browse/HUDI-3781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-3781:
---------------------------------
Labels: pull-request-available (was: )
> spark delete sql can't delete record
> ------------------------------------
>
> Key: HUDI-3781
> URL: https://issues.apache.org/jira/browse/HUDI-3781
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark, spark-sql
> Reporter: KnightChess
> Priority: Critical
> Labels: pull-request-available
>
> create a table and set *hoodie.datasource.write.operation* upsert
> when I use _*sql*_ to delete, the delete *operation key* will be overwrite by
> *hoodie.datasource.write.operation* from table or env
>
> {code:java}
> withSparkConf(sparkSession, hoodieCatalogTable.catalogProperties) {
> Map(
> "path" -> path,
> RECORDKEY_FIELD.key -> hoodieCatalogTable.primaryKeys.mkString(","),
> TBL_NAME.key -> tableConfig.getTableName,
> HIVE_STYLE_PARTITIONING.key -> tableConfig.getHiveStylePartitioningEnable,
> URL_ENCODE_PARTITIONING.key -> tableConfig.getUrlEncodePartitioning,
> KEYGENERATOR_CLASS_NAME.key -> classOf[SqlKeyGenerator].getCanonicalName,
> SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME ->
> tableConfig.getKeyGeneratorClassName,
> OPERATION.key -> DataSourceWriteOptions.DELETE_OPERATION_OPT_VAL,
> PARTITIONPATH_FIELD.key -> tableConfig.getPartitionFieldProp,
> HiveSyncConfig.HIVE_SYNC_MODE.key -> HiveSyncMode.HMS.name(),
> HiveSyncConfig.HIVE_SUPPORT_TIMESTAMP_TYPE.key -> "true",
> HoodieWriteConfig.DELETE_PARALLELISM_VALUE.key -> "200",
> SqlKeyGenerator.PARTITION_SCHEMA -> partitionSchema.toDDL
> )
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
