[jira] [Created] (SPARK-52824) Classify errors thrown from CheckpointFileManager.create

2025-07-16 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-52824: - Summary: Classify errors thrown from CheckpointFileManager.create Key: SPARK-52824 URL: https://issues.apache.org/jira/browse/SPARK-52824 Project: Spark Issue Type

[jira] [Created] (SPARK-52740) Fix NPE in HDFSBackedStateStoreProvider accessing StateStoreConf.sqlConf when checkpoint format version is >=2

2025-07-09 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-52740: - Summary: Fix NPE in HDFSBackedStateStoreProvider accessing StateStoreConf.sqlConf when checkpoint format version is >=2 Key: SPARK-52740 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Created] (SPARK-52637) Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load

2025-07-01 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-52637: - Summary: Fix VERSIONID_MISMATCH error when RocksDB compaction leads to incorrect file mapping during load Key: SPARK-52637 URL: https://issues.apache.org/jira/browse/SPARK-52637

[jira] [Created] (SPARK-52198) Fix keySchema in KeyWithIndexToValueStore for join state format v3 with column families

2025-05-16 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-52198: - Summary: Fix keySchema in KeyWithIndexToValueStore for join state format v3 with column families Key: SPARK-52198 URL: https://issues.apache.org/jira/browse/SPARK-52198 Pro

[jira] [Created] (SPARK-52171) Add StateDataSource implementation for state format v3 for join

2025-05-15 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-52171: - Summary: Add StateDataSource implementation for state format v3 for join Key: SPARK-52171 URL: https://issues.apache.org/jira/browse/SPARK-52171 Project: Spark Is

[jira] [Created] (SPARK-52008) Add TaskCompletionListener for state stores that abort updating store and throw error

2025-05-05 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-52008: - Summary: Add TaskCompletionListener for state stores that abort updating store and throw error Key: SPARK-52008 URL: https://issues.apache.org/jira/browse/SPARK-52008 Proje

[jira] [Updated] (SPARK-51291) Reclassify Validation Errors from CANNOT_LOAD_STATE_STORE.UNCATEGORIZED

2025-04-30 Thread Livia Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Livia Zhu updated SPARK-51291: -- Description: If HDFS state store checkpoint is corrupted, STATE_STORE_KEY_ROW_FORMAT_VALIDATION_FAILU

[jira] [Created] (SPARK-51922) Fix UTFDataFormatException thrown from StateStoreChangelogReaderFactory for v1

2025-04-25 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-51922: - Summary: Fix UTFDataFormatException thrown from StateStoreChangelogReaderFactory for v1 Key: SPARK-51922 URL: https://issues.apache.org/jira/browse/SPARK-51922 Project: Spa

[jira] [Updated] (SPARK-51596) Fix concurrent StateStoreProvider maintenance and closing

2025-04-04 Thread Livia Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Livia Zhu updated SPARK-51596: -- Description: Currently, both the task thread and maintenance thread can call unload() on a provider.

[jira] [Updated] (SPARK-51596) Fix concurrent StateStoreProvider maintenance and closing

2025-03-24 Thread Livia Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Livia Zhu updated SPARK-51596: -- Description: Currently, both the task thread and maintenance thread can call unload() on a provider.

[jira] [Updated] (SPARK-51596) Fix concurrent StateStoreProvider maintenance and closing

2025-03-24 Thread Livia Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Livia Zhu updated SPARK-51596: -- Description: Currently, both the task thread and maintenance thread can call unload() on a provider.

[jira] [Created] (SPARK-51596) Fix concurrent StateStoreProvider maintenance and closing

2025-03-24 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-51596: - Summary: Fix concurrent StateStoreProvider maintenance and closing Key: SPARK-51596 URL: https://issues.apache.org/jira/browse/SPARK-51596 Project: Spark Issue Typ

[jira] [Created] (SPARK-51573) Fix Streaming State Checkpoint v2 checkpointInfo race condition

2025-03-20 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-51573: - Summary: Fix Streaming State Checkpoint v2 checkpointInfo race condition Key: SPARK-51573 URL: https://issues.apache.org/jira/browse/SPARK-51573 Project: Spark Is

[jira] [Created] (SPARK-51291) Reclassify Validation Errors from CANNOT_LOAD_STATE_STORE.UNCATEGORIZED

2025-02-21 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-51291: - Summary: Reclassify Validation Errors from CANNOT_LOAD_STATE_STORE.UNCATEGORIZED Key: SPARK-51291 URL: https://issues.apache.org/jira/browse/SPARK-51291 Project: Spark

[jira] [Updated] (SPARK-51291) Reclassify Validation Errors from CANNOT_LOAD_STATE_STORE.UNCATEGORIZED

2025-02-21 Thread Livia Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Livia Zhu updated SPARK-51291: -- Description: If HDFS state store checkpoint is corrupted, STATE_STORE_KEY_ROW_FORMAT_VALIDATION_FAILU

[jira] [Created] (SPARK-50791) Fix NullPointerException during error handling for streaming state store

2025-01-10 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-50791: - Summary: Fix NullPointerException during error handling for streaming state store Key: SPARK-50791 URL: https://issues.apache.org/jira/browse/SPARK-50791 Project: Spark

[jira] [Updated] (SPARK-50492) Fix java.util.NoSuchElementException when event time column is dropped after dropDuplicatesWithinWatermark

2024-12-05 Thread Livia Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Livia Zhu updated SPARK-50492: -- Summary: Fix java.util.NoSuchElementException when event time column is dropped after dropDuplicatesWi

[jira] [Created] (SPARK-50492) Fix java.util.NoSuchElementException when watermark column is dropped after dropDuplicatesWithinWatermark

2024-12-04 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-50492: - Summary: Fix java.util.NoSuchElementException when watermark column is dropped after dropDuplicatesWithinWatermark Key: SPARK-50492 URL: https://issues.apache.org/jira/browse/SPARK-5049

[jira] [Created] (SPARK-50163) Fix Streaming RocksDB AcquireLock double release

2024-10-29 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-50163: - Summary: Fix Streaming RocksDB AcquireLock double release Key: SPARK-50163 URL: https://issues.apache.org/jira/browse/SPARK-50163 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-49772) Simplify and clarify dbOptions for RocksDB

2024-09-24 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-49772: - Summary: Simplify and clarify dbOptions for RocksDB Key: SPARK-49772 URL: https://issues.apache.org/jira/browse/SPARK-49772 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-49474) Classify error class for flatMapGroupsWithState user function error

2024-08-30 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-49474: - Summary: Classify error class for flatMapGroupsWithState user function error Key: SPARK-49474 URL: https://issues.apache.org/jira/browse/SPARK-49474 Project: Spark