[jira] [Created] (SPARK-47362) Enhance the console sink to provide watermark and state information

2024-03-12 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-47362: -- Summary: Enhance the console sink to provide watermark and state information Key: SPARK-47362 URL: https://issues.apache.org/jira/browse/SPARK-47362 Project: Spar

[jira] [Created] (SPARK-47369) Fix performance regression in JDK 17 caused from RocksDB logging

2024-03-12 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-47369: -- Summary: Fix performance regression in JDK 17 caused from RocksDB logging Key: SPARK-47369 URL: https://issues.apache.org/jira/browse/SPARK-47369 Project: Spark

[jira] [Commented] (SPARK-47369) Fix performance regression in JDK 17 caused from RocksDB logging

2024-03-12 Thread Neil Ramaswamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825828#comment-17825828 ] Neil Ramaswamy commented on SPARK-47369: cc: [~dongjoon] and [~LuciferYang], who

[jira] [Commented] (SPARK-47369) Fix performance regression in JDK 17 caused from RocksDB logging

2024-03-13 Thread Neil Ramaswamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17826810#comment-17826810 ] Neil Ramaswamy commented on SPARK-47369: [~dongjoon], I will create a minimal Sp

[jira] [Created] (SPARK-47744) Add support for negative byte types in RocksDB range scan encoder

2024-04-05 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-47744: -- Summary: Add support for negative byte types in RocksDB range scan encoder Key: SPARK-47744 URL: https://issues.apache.org/jira/browse/SPARK-47744 Project: Spark

[jira] [Created] (SPARK-47746) Use column ordinals instead of prefix ordering columns in the range scan encoder

2024-04-05 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-47746: -- Summary: Use column ordinals instead of prefix ordering columns in the range scan encoder Key: SPARK-47746 URL: https://issues.apache.org/jira/browse/SPARK-47746

[jira] [Created] (SPARK-48707) RocksDB setLevelCompactionDynamicLevelBytes should be set to false

2024-06-24 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-48707: -- Summary: RocksDB setLevelCompactionDynamicLevelBytes should be set to false Key: SPARK-48707 URL: https://issues.apache.org/jira/browse/SPARK-48707 Project: Spark

[jira] [Created] (SPARK-48997) Maintenance thread pool error should not cause the entire executor to crash

2024-07-24 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-48997: -- Summary: Maintenance thread pool error should not cause the entire executor to crash Key: SPARK-48997 URL: https://issues.apache.org/jira/browse/SPARK-48997 Proje

[jira] [Created] (SPARK-45363) Avro connector should use new error class framework

2023-09-27 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-45363: -- Summary: Avro connector should use new error class framework Key: SPARK-45363 URL: https://issues.apache.org/jira/browse/SPARK-45363 Project: Spark Issue

[jira] [Created] (SPARK-49131) TransformWithState doesn't set implicit grouping key when lazy iterators are used

2024-08-06 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-49131: -- Summary: TransformWithState doesn't set implicit grouping key when lazy iterators are used Key: SPARK-49131 URL: https://issues.apache.org/jira/browse/SPARK-49131

[jira] [Created] (SPARK-49378) Break apart the Structured Streaming Programming Guide

2024-08-25 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-49378: -- Summary: Break apart the Structured Streaming Programming Guide Key: SPARK-49378 URL: https://issues.apache.org/jira/browse/SPARK-49378 Project: Spark Is

[jira] [Created] (SPARK-49456) Spark website doesn't properly scroll to hash links

2024-08-28 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-49456: -- Summary: Spark website doesn't properly scroll to hash links Key: SPARK-49456 URL: https://issues.apache.org/jira/browse/SPARK-49456 Project: Spark Issu

[jira] [Created] (SPARK-49592) OperatorStateMetadataV2 can return incorrect metadata for the given batchId

2024-09-10 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-49592: -- Summary: OperatorStateMetadataV2 can return incorrect metadata for the given batchId Key: SPARK-49592 URL: https://issues.apache.org/jira/browse/SPARK-49592 Proje

[jira] [Resolved] (SPARK-49592) OperatorStateMetadataV2 can return incorrect metadata for the given batchId

2024-09-12 Thread Neil Ramaswamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Ramaswamy resolved SPARK-49592. Resolution: Not A Bug > OperatorStateMetadataV2 can return incorrect metadata for the give

[jira] [Commented] (SPARK-49592) OperatorStateMetadataV2 can return incorrect metadata for the given batchId

2024-09-12 Thread Neil Ramaswamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881188#comment-17881188 ] Neil Ramaswamy commented on SPARK-49592: After further consideration (see discus

[jira] [Updated] (SPARK-49592) OperatorStateMetadataV2 can return incorrect metadata for the given batchId

2024-09-12 Thread Neil Ramaswamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Ramaswamy updated SPARK-49592: --- Issue Type: Bug (was: Improvement) > OperatorStateMetadataV2 can return incorrect metadata

[jira] [Created] (SPARK-49944) Fix Javascript and image imports in the documentation site

2024-10-12 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-49944: -- Summary: Fix Javascript and image imports in the documentation site Key: SPARK-49944 URL: https://issues.apache.org/jira/browse/SPARK-49944 Project: Spark

[jira] [Created] (SPARK-50301) TransformWithState metrics incorrectly counts the time for state removal

2024-11-13 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-50301: -- Summary: TransformWithState metrics incorrectly counts the time for state removal Key: SPARK-50301 URL: https://issues.apache.org/jira/browse/SPARK-50301 Project:

[jira] [Created] (SPARK-50302) TransformWithState secondary index sizes should be proportional to primary indexes

2024-11-13 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-50302: -- Summary: TransformWithState secondary index sizes should be proportional to primary indexes Key: SPARK-50302 URL: https://issues.apache.org/jira/browse/SPARK-50302

[jira] [Created] (SPARK-50346) TransformWithState timers should compare expiration with less than or equal to

2024-11-18 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-50346: -- Summary: TransformWithState timers should compare expiration with less than or equal to Key: SPARK-50346 URL: https://issues.apache.org/jira/browse/SPARK-50346 Pr

[jira] [Created] (SPARK-51083) JavaUtils swallow interrupted exceptions and can cause executor loss

2025-02-04 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-51083: -- Summary: JavaUtils swallow interrupted exceptions and can cause executor loss Key: SPARK-51083 URL: https://issues.apache.org/jira/browse/SPARK-51083 Project: Spa

[jira] [Created] (SPARK-50584) Modify state eviction metrics in TransformWithState to represent physical deletions

2024-12-15 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-50584: -- Summary: Modify state eviction metrics in TransformWithState to represent physical deletions Key: SPARK-50584 URL: https://issues.apache.org/jira/browse/SPARK-50584

[jira] [Created] (SPARK-51011) Add logging for whether a task is going to be interrupted when killing tasks

2025-01-27 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-51011: -- Summary: Add logging for whether a task is going to be interrupted when killing tasks Key: SPARK-51011 URL: https://issues.apache.org/jira/browse/SPARK-51011 Proj

[jira] [Updated] (SPARK-51376) TransformWithState should prescribe RocksDB in its error message when HDFS is in use

2025-03-03 Thread Neil Ramaswamy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neil Ramaswamy updated SPARK-51376: --- Summary: TransformWithState should prescribe RocksDB in its error message when HDFS is in us

[jira] [Created] (SPARK-51376) TransformWithState should throw a nice error message when HDFS is in use

2025-03-03 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-51376: -- Summary: TransformWithState should throw a nice error message when HDFS is in use Key: SPARK-51376 URL: https://issues.apache.org/jira/browse/SPARK-51376 Project: