[jira] [Created] (SPARK-47327) Fix thread safety issue in ICU Collator

2024-03-08 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47327: - Summary: Fix thread safety issue in ICU Collator Key: SPARK-47327 URL: https://issues.apache.org/jira/browse/SPARK-47327 Project: Spark Issue Type: Improve

[jira] [Created] (SPARK-47328) Change utf8 collation names to UTF8_BINARY

2024-03-08 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47328: - Summary: Change utf8 collation names to UTF8_BINARY Key: SPARK-47328 URL: https://issues.apache.org/jira/browse/SPARK-47328 Project: Spark Issue Type: Impr

[jira] [Created] (SPARK-47395) Add collate and collation to non-sql APIs

2024-03-14 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47395: - Summary: Add collate and collation to non-sql APIs Key: SPARK-47395 URL: https://issues.apache.org/jira/browse/SPARK-47395 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-47399) Disable generated columns on expressions with collations

2024-03-14 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47399: - Summary: Disable generated columns on expressions with collations Key: SPARK-47399 URL: https://issues.apache.org/jira/browse/SPARK-47399 Project: Spark Is

[jira] [Created] (SPARK-47484) Allow trailing comma in column definition list

2024-03-20 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47484: - Summary: Allow trailing comma in column definition list Key: SPARK-47484 URL: https://issues.apache.org/jira/browse/SPARK-47484 Project: Spark Issue Type:

[jira] [Created] (SPARK-47485) Create column with collations in dataframe API

2024-03-20 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47485: - Summary: Create column with collations in dataframe API Key: SPARK-47485 URL: https://issues.apache.org/jira/browse/SPARK-47485 Project: Spark Issue Type:

[jira] [Created] (SPARK-47506) Support all file source formats for collated data types

2024-03-21 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47506: - Summary: Support all file source formats for collated data types Key: SPARK-47506 URL: https://issues.apache.org/jira/browse/SPARK-47506 Project: Spark Iss

[jira] [Created] (SPARK-47657) Implement filter pushdown support for collation per file source

2024-03-29 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47657: - Summary: Implement filter pushdown support for collation per file source Key: SPARK-47657 URL: https://issues.apache.org/jira/browse/SPARK-47657 Project: Spark

[jira] [Created] (SPARK-47873) Write collated strings to hive as regular strings

2024-04-16 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47873: - Summary: Write collated strings to hive as regular strings Key: SPARK-47873 URL: https://issues.apache.org/jira/browse/SPARK-47873 Project: Spark Issue Typ

[jira] [Created] (SPARK-48175) Store collation information in metadata and not in type for SER/DE

2024-05-07 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-48175: - Summary: Store collation information in metadata and not in type for SER/DE Key: SPARK-48175 URL: https://issues.apache.org/jira/browse/SPARK-48175 Project: Spark

[jira] [Updated] (SPARK-48263) Collate expression not working when default collation set

2024-05-14 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-48263: -- Epic Link: SPARK-46830 > Collate expression not working when default collation set > -

[jira] [Created] (SPARK-48339) Revert converting collated strings to regular strings when writing to hive metastore

2024-05-20 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-48339: - Summary: Revert converting collated strings to regular strings when writing to hive metastore Key: SPARK-48339 URL: https://issues.apache.org/jira/browse/SPARK-48339

[jira] [Created] (SPARK-48379) Cancel build during a PR when a new commit is pushed

2024-05-21 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-48379: - Summary: Cancel build during a PR when a new commit is pushed Key: SPARK-48379 URL: https://issues.apache.org/jira/browse/SPARK-48379 Project: Spark Issue

[jira] [Created] (SPARK-48697) Fix consumption of filters that have widened parts of the predicate to be AlwaysTrue

2024-06-24 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-48697: - Summary: Fix consumption of filters that have widened parts of the predicate to be AlwaysTrue Key: SPARK-48697 URL: https://issues.apache.org/jira/browse/SPARK-48697

[jira] [Updated] (SPARK-48697) Properly translate filters on collated columns

2024-06-27 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-48697: -- Summary: Properly translate filters on collated columns (was: Fix consumption of filters that

[jira] [Updated] (SPARK-48697) Add collation aware string filters

2024-06-27 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-48697: -- Summary: Add collation aware string filters (was: Properly translate filters on collated colu

[jira] [Created] (SPARK-48739) Disable writing collated data to file formats that don't support them in non managed tables

2024-06-27 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-48739: - Summary: Disable writing collated data to file formats that don't support them in non managed tables Key: SPARK-48739 URL: https://issues.apache.org/jira/browse/SPARK-48739

[jira] [Updated] (SPARK-48981) Fix pyspark simpleString method for collations

2024-07-23 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-48981: -- Summary: Fix pyspark simpleString method for collations (was: Fix pyspark simpleString method

[jira] [Created] (SPARK-49013) Change key in collationsMap for Map and Array types in scala

2024-07-26 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-49013: - Summary: Change key in collationsMap for Map and Array types in scala Key: SPARK-49013 URL: https://issues.apache.org/jira/browse/SPARK-49013 Project: Spark

[jira] [Created] (SPARK-46488) Skip trimAll call when doing string to timestamp

2023-12-22 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-46488: - Summary: Skip trimAll call when doing string to timestamp Key: SPARK-46488 URL: https://issues.apache.org/jira/browse/SPARK-46488 Project: Spark Issue Type

[jira] [Created] (SPARK-46539) SELECT * EXCEPT(all fields from a struct) results in an assertion failure

2023-12-28 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-46539: - Summary: SELECT * EXCEPT(all fields from a struct) results in an assertion failure Key: SPARK-46539 URL: https://issues.apache.org/jira/browse/SPARK-46539 Project:

[jira] [Updated] (SPARK-46725) Add DAYNAME function

2024-01-15 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-46725: -- Description: Add function DAYNAME which returns three-letter abbreviation of a given date/tim

[jira] [Created] (SPARK-47007) Add SortMap function

2024-02-08 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47007: - Summary: Add SortMap function Key: SPARK-47007 URL: https://issues.apache.org/jira/browse/SPARK-47007 Project: Spark Issue Type: Improvement Comp

[jira] [Created] (SPARK-47009) Create table with collation

2024-02-08 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47009: - Summary: Create table with collation Key: SPARK-47009 URL: https://issues.apache.org/jira/browse/SPARK-47009 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-47015) Disable partitioning on collated columns

2024-02-09 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47015: - Summary: Disable partitioning on collated columns Key: SPARK-47015 URL: https://issues.apache.org/jira/browse/SPARK-47015 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-47015) Disable partitioning on collated columns

2024-02-20 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-47015: -- Epic Link: SPARK-46830 > Disable partitioning on collated columns > --

[jira] [Updated] (SPARK-47102) Add COLLATION_ENABLED config flag

2024-02-20 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-47102: -- Epic Link: SPARK-46830 > Add COLLATION_ENABLED config flag > -

[jira] [Updated] (SPARK-47009) Create table with collation

2024-02-20 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-47009: -- Epic Link: SPARK-46830 > Create table with collation > --- > >

[jira] [Created] (SPARK-47168) Disable filter pushdown for non default collated strings

2024-02-26 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47168: - Summary: Disable filter pushdown for non default collated strings Key: SPARK-47168 URL: https://issues.apache.org/jira/browse/SPARK-47168 Project: Spark Is

[jira] [Updated] (SPARK-47168) Disable parquet filter pushdown for non default collated strings

2024-02-26 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-47168: -- Summary: Disable parquet filter pushdown for non default collated strings (was: Disable filte

[jira] [Created] (SPARK-49895) Add better error when encountering trailing comma in SELECT clause

2024-10-07 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-49895: - Summary: Add better error when encountering trailing comma in SELECT clause Key: SPARK-49895 URL: https://issues.apache.org/jira/browse/SPARK-49895 Project: Spark

[jira] [Created] (SPARK-49916) Throw appropriate Exception for type mismatch between ColumnType and data type in some rows

2024-10-09 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-49916: - Summary: Throw appropriate Exception for type mismatch between ColumnType and data type in some rows Key: SPARK-49916 URL: https://issues.apache.org/jira/browse/SPARK-49916

[jira] [Created] (SPARK-49934) Fix implicit cast for accessing collated map

2024-10-11 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-49934: - Summary: Fix implicit cast for accessing collated map Key: SPARK-49934 URL: https://issues.apache.org/jira/browse/SPARK-49934 Project: Spark Issue Type: Im

[jira] [Updated] (SPARK-49934) Add implicit cast for accessing collated map with literal

2024-10-11 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-49934: -- Summary: Add implicit cast for accessing collated map with literal (was: Fix implicit cast fo

[jira] [Created] (SPARK-49992) Session level collation should not impact DDL queries

2024-10-16 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-49992: - Summary: Session level collation should not impact DDL queries Key: SPARK-49992 URL: https://issues.apache.org/jira/browse/SPARK-49992 Project: Spark Issue

[jira] [Created] (SPARK-49962) Simplify AbstractStringTypes class hierarchy

2024-10-14 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-49962: - Summary: Simplify AbstractStringTypes class hierarchy Key: SPARK-49962 URL: https://issues.apache.org/jira/browse/SPARK-49962 Project: Spark Issue Type: Im

[jira] [Created] (SPARK-50175) Change collation precedence

2024-10-30 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50175: - Summary: Change collation precedence Key: SPARK-50175 URL: https://issues.apache.org/jira/browse/SPARK-50175 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-50214) From json/xml should not change collations in the given schema

2024-11-04 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50214: - Summary: From json/xml should not change collations in the given schema Key: SPARK-50214 URL: https://issues.apache.org/jira/browse/SPARK-50214 Project: Spark

[jira] [Created] (SPARK-49843) Fix change comment for char/varchar columns

2024-10-01 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-49843: - Summary: Fix change comment for char/varchar columns Key: SPARK-49843 URL: https://issues.apache.org/jira/browse/SPARK-49843 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-49843) Fix change comment for char/varchar columns

2024-10-03 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-49843: -- Affects Version/s: 3.5.0 3.4.0 > Fix change comment for char/varchar co

[jira] [Updated] (SPARK-49843) Fix change comment for char/varchar columns

2024-10-03 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-49843: -- Fix Version/s: 3.4.4 3.5.4 > Fix change comment for char/varchar columns >

[jira] [Created] (SPARK-50093) Collations that use ICU should have the version of the used ICU library

2024-10-24 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50093: - Summary: Collations that use ICU should have the version of the used ICU library Key: SPARK-50093 URL: https://issues.apache.org/jira/browse/SPARK-50093 Project: Sp

[jira] [Created] (SPARK-50413) Add a flag to insert commands to know if they are part of CTAS

2024-11-25 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50413: - Summary: Add a flag to insert commands to know if they are part of CTAS Key: SPARK-50413 URL: https://issues.apache.org/jira/browse/SPARK-50413 Project: Spark

[jira] [Created] (SPARK-50522) Support for indeterminate collation

2024-12-09 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50522: - Summary: Support for indeterminate collation Key: SPARK-50522 URL: https://issues.apache.org/jira/browse/SPARK-50522 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-50530) Fix bad implicit string type context calculation #49114

2024-12-10 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50530: - Summary: Fix bad implicit string type context calculation #49114 Key: SPARK-50530 URL: https://issues.apache.org/jira/browse/SPARK-50530 Project: Spark Iss

[jira] [Created] (SPARK-50904) Fix collation expression walker query execution

2025-01-21 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50904: - Summary: Fix collation expression walker query execution Key: SPARK-50904 URL: https://issues.apache.org/jira/browse/SPARK-50904 Project: Spark Issue Type:

[jira] [Created] (SPARK-50895) Create common interface for expressions which produce default string type

2025-01-20 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50895: - Summary: Create common interface for expressions which produce default string type Key: SPARK-50895 URL: https://issues.apache.org/jira/browse/SPARK-50895 Project:

[jira] [Created] (SPARK-50649) Fix inconsistencies with casting between different collations

2024-12-23 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50649: - Summary: Fix inconsistencies with casting between different collations Key: SPARK-50649 URL: https://issues.apache.org/jira/browse/SPARK-50649 Project: Spark

[jira] [Updated] (SPARK-50774) Centralize collation names in one place

2025-01-10 Thread Stefan Kandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Kandic updated SPARK-50774: -- Summary: Centralize collation names in one place (was: Make collation names public in Collati

[jira] [Created] (SPARK-50774) Make collation names public in CollationFactory

2025-01-09 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50774: - Summary: Make collation names public in CollationFactory Key: SPARK-50774 URL: https://issues.apache.org/jira/browse/SPARK-50774 Project: Spark Issue Type:

[jira] [Created] (SPARK-50829) Add flag to disable session collation by default

2025-01-15 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-50829: - Summary: Add flag to disable session collation by default Key: SPARK-50829 URL: https://issues.apache.org/jira/browse/SPARK-50829 Project: Spark Issue Type

[jira] [Created] (SPARK-51468) Revert "From json/xml should not change collations in the given schema"

2025-03-11 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-51468: - Summary: Revert "From json/xml should not change collations in the given schema" Key: SPARK-51468 URL: https://issues.apache.org/jira/browse/SPARK-51468 Project: Sp

[jira] [Created] (SPARK-51501) Disable ObjectHashAggregate for group by on collated columns

2025-03-13 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-51501: - Summary: Disable ObjectHashAggregate for group by on collated columns Key: SPARK-51501 URL: https://issues.apache.org/jira/browse/SPARK-51501 Project: Spark

[jira] [Created] (SPARK-51502) Move collations test to collations package

2025-03-13 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-51502: - Summary: Move collations test to collations package Key: SPARK-51502 URL: https://issues.apache.org/jira/browse/SPARK-51502 Project: Spark Issue Type: Impr

[jira] [Created] (SPARK-51310) Resolve the type default string producing expressions

2025-02-25 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-51310: - Summary: Resolve the type default string producing expressions Key: SPARK-51310 URL: https://issues.apache.org/jira/browse/SPARK-51310 Project: Spark Issue

[jira] [Created] (SPARK-52753) Make parseDataType binary compatible with previous versions

2025-07-10 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-52753: - Summary: Make parseDataType binary compatible with previous versions Key: SPARK-52753 URL: https://issues.apache.org/jira/browse/SPARK-52753 Project: Spark

[jira] [Created] (SPARK-53130) Fix toJson method for collated strings

2025-08-05 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-53130: - Summary: Fix toJson method for collated strings Key: SPARK-53130 URL: https://issues.apache.org/jira/browse/SPARK-53130 Project: Spark Issue Type: Improvem