[jira] [Created] (SPARK-50853) Temp shuffle files not closed

2025-01-16 Thread Michael Chen (Jira)
Michael Chen created SPARK-50853: Summary: Temp shuffle files not closed Key: SPARK-50853 URL: https://issues.apache.org/jira/browse/SPARK-50853 Project: Spark Issue Type: Bug Compo

[jira] [Created] (SPARK-44897) Local Property Propagation to Subquery Broadcast Exec

2023-08-21 Thread Michael Chen (Jira)
Michael Chen created SPARK-44897: Summary: Local Property Propagation to Subquery Broadcast Exec Key: SPARK-44897 URL: https://issues.apache.org/jira/browse/SPARK-44897 Project: Spark Issue T

[jira] [Created] (SPARK-37442) In AQE, wrong InMemoryRelation size estimation causes "Cannot broadcast the table that is larger than 8GB: 8 GB" failure

2021-11-22 Thread Michael Chen (Jira)
Michael Chen created SPARK-37442: Summary: In AQE, wrong InMemoryRelation size estimation causes "Cannot broadcast the table that is larger than 8GB: 8 GB" failure Key: SPARK-37442 URL: https://issues.apache.org/j

[jira] [Created] (SPARK-36911) Add AQE Planning Times to SQL Metrics

2021-10-01 Thread Michael Chen (Jira)
Michael Chen created SPARK-36911: Summary: Add AQE Planning Times to SQL Metrics Key: SPARK-36911 URL: https://issues.apache.org/jira/browse/SPARK-36911 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-36795) Explain Formatted has Duplicated Node IDs with InMemoryRelation Present

2021-09-17 Thread Michael Chen (Jira)
Michael Chen created SPARK-36795: Summary: Explain Formatted has Duplicated Node IDs with InMemoryRelation Present Key: SPARK-36795 URL: https://issues.apache.org/jira/browse/SPARK-36795 Project: Spar

[jira] [Commented] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-04-14 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321270#comment-17321270 ] Michael Chen commented on SPARK-34780: -- Hi [~csun]. Also sorry for late reply. My u

[jira] [Commented] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-29 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311008#comment-17311008 ] Michael Chen commented on SPARK-34780: -- Hey [~csun]. I did find a conf that can cau

[jira] [Commented] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-26 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17309514#comment-17309514 ] Michael Chen commented on SPARK-34780: -- Yes, I think as long as there aren't any co

[jira] [Commented] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-25 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17308811#comment-17308811 ] Michael Chen commented on SPARK-34780: -- Np [~csun]. If the cache isn't materialized

[jira] [Comment Edited] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-19 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305162#comment-17305162 ] Michael Chen edited comment on SPARK-34780 at 3/19/21, 8:00 PM: --

[jira] [Comment Edited] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-19 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305162#comment-17305162 ] Michael Chen edited comment on SPARK-34780 at 3/19/21, 8:00 PM: --

[jira] [Commented] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-19 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17305162#comment-17305162 ] Michael Chen commented on SPARK-34780: -- I used computeStats because it was a simple

[jira] [Updated] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-18 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Chen updated SPARK-34780: - Description: When a dataframe is cached, the logical plan can contain copies of the spark sessi

[jira] [Updated] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-18 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Chen updated SPARK-34780: - Affects Version/s: 3.1.1 > Cached Table (parquet) with old Configs Used > --

[jira] [Updated] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-17 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Chen updated SPARK-34780: - Description: When a dataframe is cached, the logical plan can contain copies of the spark sessi

[jira] [Created] (SPARK-34780) Cached Table (parquet) with old Configs Used

2021-03-17 Thread Michael Chen (Jira)
Michael Chen created SPARK-34780: Summary: Cached Table (parquet) with old Configs Used Key: SPARK-34780 URL: https://issues.apache.org/jira/browse/SPARK-34780 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-29 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962361#comment-16962361 ] Michael Chen commented on SPARK-29561: -- If I increase the memory, it will run into

[jira] [Comment Edited] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-29 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962361#comment-16962361 ] Michael Chen edited comment on SPARK-29561 at 10/29/19 7:59 PM: --

[jira] [Comment Edited] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-29 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962361#comment-16962361 ] Michael Chen edited comment on SPARK-29561 at 10/29/19 7:59 PM: --

[jira] [Updated] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-22 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Chen updated SPARK-29561: - Description: Spark Configuration spark.driver.memory = 1g spark.master = "local" spark.deploy

[jira] [Updated] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-22 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Chen updated SPARK-29561: - Attachment: apacheSparkCase.sql > Large Case Statement Code Generation OOM > ---

[jira] [Updated] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-22 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Chen updated SPARK-29561: - Description: Spark Configuration spark.driver.memory = 1g spark.master = "local" spark.deploy

[jira] [Created] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-22 Thread Michael Chen (Jira)
Michael Chen created SPARK-29561: Summary: Large Case Statement Code Generation OOM Key: SPARK-29561 URL: https://issues.apache.org/jira/browse/SPARK-29561 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-29561) Large Case Statement Code Generation OOM

2019-10-22 Thread Michael Chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Chen updated SPARK-29561: - Description: Spark Configuration spark.driver.memory = 1g spark.master = "local" spark.deploy