[ANNOUNCE] Apache Spark 3.5.5 released

2025-02-27 Thread Dongjoon Hyun
We are happy to announce the availability of Apache Spark 3.5.5! Spark 3.5.5 is the fifth maintenance release based on the branch-3.5 maintenance branch of Spark. It contains many fixes including security and correctness domains. We strongly recommend all 3.5 users to upgrade to this stable releas

Optimizing file size of an iceberg table

2025-02-27 Thread Pathum Wijethunge
Hi Team, I have a use-case of writing data into an iceberg table using spark. The table has 3 partition columns (file_date, city, creation_date) and a bucket by another column (user_id,4). "partition-specs" : [ { "spec-id" : 0, "fields" : [ { "name" : "file_date", "transfo