RE: Inquiry: Best Practices for Replacing Snappy with LZ4/LZF Compression Across Spark Codebase (including test cases)

discussions on this topic would be immensely helpful. Thanks, Balaji From: Balaji Sudharsanam V Sent: 02 June 2025 21:02 To: dev@spark.apache.org Cc: Dongjoon Hyun ; Steven Jones ; NICHOLAS MARION ; Vishal Kolki ; ANTO JOHN Subject: Inquiry: Best Practices for Replacing Snappy with LZ4/LZF

Inquiry: Best Practices for Replacing Snappy with LZ4/LZF Compression Across Spark Codebase (including test cases)

Dear Spark Developer Community, I hope this email finds you well. My name is Balaji, and I am a Software Engineer working with Apache Spark in IBM Z Systems (z/OS). We are exploring a scenario where we would like to move away from using the Snappy compression library within our Spark applicatio

RE: Spark 4.0 vulnerable with hive-metastore-2.3.x.jar versions

advised by many security teams. Thanks, Balaji From: Sean Owen Sent: 28 January 2025 18:53 To: Balaji Sudharsanam V Cc: dev Subject: [EXTERNAL] Re: Spark 4.0 vulnerable with hive-metastore-2.3.x.jar versions If you use vulnerable code in your application, sure, you might be exposed to its

RE: Spark 4.0 vulnerable with hive-metastore-2.3.x.jar versions

From: Sean Owen Sent: 28 January 2025 10:31 To: Balaji Sudharsanam V Cc: Mich Talebzadeh ; dev Subject: [EXTERNAL] Re: Spark 4.0 vulnerable with hive-metastore-2.3.x.jar versions Can you connect the CVE to Spark? Spark does not run a Hive metastore itself nor use Hive for executing queries

RE: Spark 4.0 vulnerable with hive-metastore-2.3.x.jar versions

can be a different discussion. As long as Spark contains a vulnerable jar in its packaging, it is good to address this. Thanks, Balaji From: Mich Talebzadeh Sent: 27 January 2025 20:41 To: Sean Owen Cc: Balaji Sudharsanam V ; dev@spark.apache.org Subject: [EXTERNAL] Re: Spark 4.0 vulnerabl

Spark 4.0 vulnerable with hive-metastore-2.3.x.jar versions

Hi All, There is a vulnerability with 'High' severity found in the Apache Spark 3.x and 4.0.0 preview (2) releases, with the hive-metastore-2.3.x.jar. This is defined here, Apache Hive security bypass CVE-2021-34538 Vulnerability Report

RE: [Connect] Install additional python packages after session creation

Hi Hyukjin Kwon, Sorry for bringing in off the topic discussion, Is there a Java Client that is similar to PySpark to work with Spark connect? Thanks, Balaji From: Hyukjin Kwon Sent: 25 January 2025 04:46 To: Deependra Patel Cc: dev@spark.apache.org Subject: [EXTERNAL] Re: [Connect] Install a

RE: FYI: A Hallucination about Spark Connect Stability in Spark 4

Do we have a Java Client for Spark Connect which is something like PySpark? From: Mich Talebzadeh Sent: 22 January 2025 15:05 To: Hyukjin Kwon Cc: Martin Grund ; Holden Karau ; Dongjoon Hyun ; dev Subject: [EXTERNAL] Re: FYI: A Hallucination about Spark Connect Stability in Spark 4 CI broke

Java Client for Spark Connect

Hello Team, Do we already have a Java Client for Spark Connect similar to PySpark ?