user  

Messages by Thread

• [Spark Connect ] Date Data type formatting issue Ilango
• [ANNOUNCE] Apache Spark 3.5.2 released Kent Yao
  • Re: [ANNOUNCE] Apache Spark 3.5.2 released Xiao Li
• Is it possible to configure batch size immutability between two `mapInPandas`? Azamat G.
• Re: Spark 3.5.0 bug - Writing a small paraquet dataframe to storage using spark 3.5.0 taking too long Bijoy Deb
• [spark connect] unable to utilize stand alone cluster Ilango
  • Re: [spark connect] unable to utilize stand alone cluster Prabodh Agarwal
  • Re: [spark connect] unable to utilize stand alone cluster Ilango
  • Re: [spark connect] unable to utilize stand alone cluster Prabodh Agarwal
  • Re: [spark connect] unable to utilize stand alone cluster Ilango
  • Re: [spark connect] unable to utilize stand alone cluster Prabodh Agarwal
• Spark History CORS header ‘Access-Control-Allow-Origin’ missing Thomas Mauran
• dynamically infer json data not working as expected Perez
  • Re: dynamically infer json data not working as expected Perez
  • Re: dynamically infer json data not working as expected Mich Talebzadeh
  • Re: dynamically infer json data not working as expected Perez
  • Re: dynamically infer json data not working as expected Perez
• Feature Engineering for Data Engineers: Building Blocks for ML Success Mich Talebzadeh
• Error While Running Merge Statement With Iceberg PRASHANT L
• A code change for spark ui in Sql tab Donvi
• Question about installing Apache Spark [PySpark] computer requirements mike Jadoo
  • Re: Question about installing Apache Spark [PySpark] computer requirements Sadha Chilukoori
  • Re: Question about installing Apache Spark [PySpark] computer requirements mike Jadoo
  • Re: Question about installing Apache Spark [PySpark] computer requirements Sadha Chilukoori
  • Re: Question about installing Apache Spark [PySpark] computer requirements Meena Rajani
• [ANNOUNCE] Apache Celeborn 0.5.1 available Ethan Feng
• [ANNOUNCE] Apache Celeborn 0.4.2 available Fu Chen
• [Spark Connect] connection issue Ilango
  • Re: [Spark Connect] connection issue Prabodh Agarwal
  • Re: [Spark Connect] connection issue Ilango
  • Re: [Spark Connect] connection issue Prabodh Agarwal
  • Re: [Spark Connect] connection issue Ilango
  • Re: [Spark Connect] connection issue Prabodh Agarwal
• Issue with comparing structs (possible bug) Dhruv Singla
  • Re: Issue with comparing structs (possible bug) Dhruv Singla
• [ANNOUNCE] Apache Kyuubi v1.9.2 is available Fu Chen
• [Spark SQL]: Why the OptimizeSkewedJoin rule does not optimize FullOuterJoin? 王仲轩(万章)
• Re: issue forwarding SPARK_CONF_DIR to start workers Patrice Duroux
  • Re: issue forwarding SPARK_CONF_DIR to start workers Holden Karau
• [spark connect] issue in testing spark connect Ilango
• Error on Spark history UI api when used with Apache Knox thomas.mau...@etu.umontpellier.fr.INVALID
• problem using spark 3.4 with spots wafa gabouj
• binary stream Jeff Pang
• heap fragmentation in G1 aka.fe2s
• Help wanted on securing spark with Apache Knox / JWT Thomas Mauran
  • Re: Help wanted on securing spark with Apache Knox / JWT Adam Binford
• Sometimes TaskContext configuration is almost empty Asaf Mesika
• [Issue] Spark SQL - broadcast failure Sudharshan V
  • Re: [Issue] Spark SQL - broadcast failure Mich Talebzadeh
  • Re: [Issue] Spark SQL - broadcast failure Meena Rajani
  • Re: [Issue] Spark SQL - broadcast failure Sudharshan V
  • Re: [Issue] Spark SQL - broadcast failure Sudharshan V
  • Re: [Issue] Spark SQL - broadcast failure Sudharshan V
• running snowflake query using spark connect on a standalone cluster Prabodh Agarwal
• Need help to confirm vulnerable issue Will.Qin
• Assistance needed with PySpark Streaming for retrieving past messages. Mai Trang
• How to use spark.connect.Plan from dependency in a custom Spark Connect RelationPlugin? Sem
• AWS Glue and Python Perez
• Does Spark 4.0 add Sparkstreaming SQL ????
• log4j2.properties file load fails when upgrading from 3.3.0 to 3.5.1 Edgar H
• Help Needed: Distributed Logging in Spark Application Zsuzsanna D
• AttributeError: 'MulticlassMetrics' object has no attribute '_sc' Azhuvath, RajeevX
  • Re: AttributeError: 'MulticlassMetrics' object has no attribute '_sc' Saurabh Kumar
• Pyspark DataFrame.drop wrong type hints Oliver Beagley
  • Pyspark DataFrame.drop wrong type hints Oliver Beagley
• Help in understanding Exchange in Spark UI Dhruv Singla
  • Re: Help in understanding Exchange in Spark UI Mich Talebzadeh
• Spark Decommission Rajesh Mahindra
  • Re: Spark Decommission Khaldi, Ahmed
  • Re: Spark Decommission Rajesh Mahindra
• [K8S] Divergense in dockerfiles between official repositories. Andrei L
• Update mode in spark structured streaming Om Prakash
  • Re: Update mode in spark structured streaming Mich Talebzadeh
• Unable to load MongoDB atlas data via PySpark because of BsonString error Perez
  • Re: Unable to load MongoDB atlas data via PySpark because of BsonString error Perez
• OOM issue in Spark Driver Karthick Nk
  • Re: OOM issue in Spark Driver Andrzej Zera
  • Re: Re: OOM issue in Spark Driver Mich Talebzadeh
• 7368396 - Apache Spark 3.5.1 (Support) SANTOS SOUZA, ALEX
  • Re: 7368396 - Apache Spark 3.5.1 (Support) Sadha Chilukoori
• Kubernetes cluster: change log4j configuration using uploaded `--files` Jennifer Wirth
  • Re: Kubernetes cluster: change log4j configuration using uploaded `--files` Mich Talebzadeh
• [SPARK-48423] Unable to save ML Pipeline to azure blob storage Chhavi Bansal
  • Re: [SPARK-48423] Unable to save ML Pipeline to azure blob storage Chhavi Bansal
• [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Chhavi Bansal
  • Re: [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Someshwar Kale
  • Re: [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Chhavi Bansal
  • Re: [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Someshwar Kale
• Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Mich Talebzadeh
  • Re: Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Mich Talebzadeh
  • Re: Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Gourav Sengupta
• Inquiry Regarding Security Compliance of Apache Spark Docker Image Tonmoy Sagar
• Classification request VARGA, Sara
  • Re: Classification request Artemis User
  • Re: Classification request Dirk-Willem van Gulik
• [ANNOUNCE] Announcing Apache Spark 4.0.0-preview1 Wenchen Fan
• [ANNOUNCE] Apache Kyuubi released 1.9.1 Cheng Pan
• Terabytes data processing via Glue Perez
  • Re: Terabytes data processing via Glue Perez
  • Re: Terabytes data processing via Glue Russell Jurney
  • Re: Terabytes data processing via Glue Perez
• [apache-spark][spark-dataframe] DataFrameWriter.partitionBy does not guarantee previous sort result leeyc0
• [Spark on k8s] A issue of k8s resource creation order Tao Yang
• Tox and Pyspark Perez
• Spark Protobuf Deserialization Satyam Raj
  • Re: Spark Protobuf Deserialization Sandish Kumar HN
• [Spark SQL]: Does Spark support processing records with timestamp NULL in stateful streaming? Juan Casse
  • Re: [Spark SQL]: Does Spark support processing records with timestamp NULL in stateful streaming? Mich Talebzadeh
• OOM concern Perez
  • Re: OOM concern Meena Rajani
  • Re: OOM concern Russell Jurney
  • Re: OOM concern Perez
  • Re: OOM concern Mich Talebzadeh
  • Re: OOM concern Perez
  • Re: OOM concern Mich Talebzadeh
  • Re: OOM concern Russell Jurney
  • Re: OOM concern Perez
• Subject: [Spark SQL] [Debug] Spark Memory Issue with DataFrame Processing Gaurav Madan
  • Re: Subject: [Spark SQL] [Debug] Spark Memory Issue with DataFrame Processing Mich Talebzadeh
  • Re: Subject: [Spark SQL] [Debug] Spark Memory Issue with DataFrame Processing Shay Elbaz
• Can Spark Catalog Perform Multimodal Database Query Analysis ????
  • Re: Can Spark Catalog Perform Multimodal Database Query Analysis Mich Talebzadeh
• BUG :: UI Spark Prem Sahoo
  • Re: BUG :: UI Spark Prem Sahoo
  • Re: BUG :: UI Spark Prem Sahoo
  • Re: BUG :: UI Spark Sathi Chowdhury
  • Re: BUG :: UI Spark Mich Talebzadeh
  • Re: BUG :: UI Spark Mich Talebzadeh
  • Re: BUG :: UI Spark Mich Talebzadeh
• [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
• Remote File change detection in S3 when spark queries are running and parquet files in S3 changes Raghvendra Yadav
• [ANNOUNCE] Apache Celeborn 0.4.1 available Nicholas Jiang
• Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming ashok34...@yahoo.com.INVALID
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Mich Talebzadeh
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Tathagata Das
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Tathagata Das
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Mich Talebzadeh
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Mich Talebzadeh
• Re: EXT: Dual Write to HDFS and MinIO in faster way Prem Sahoo
  • Re: Re: EXT: Dual Write to HDFS and MinIO in faster way eab...@163.com
  • Re: Re: EXT: Dual Write to HDFS and MinIO in faster way Prem Sahoo
  • Re: Re: EXT: Dual Write to HDFS and MinIO in faster way Gera Shegalov
  • Re: Re: EXT: Dual Write to HDFS and MinIO in faster way Subhasis Mukherjee
• A handy tool called spark-column-analyser Mich Talebzadeh
  • Re: A handy tool called spark-column-analyser Mich Talebzadeh
  • Re: A handy tool called spark-column-analyser ashok34...@yahoo.com.INVALID
• Request for Assistance: Adding User Authentication to Apache Spark Application NIKHIL RAJ SHRIVASTAVA
• How to provide a Zstd "training mode" dictionary object Saha, Daniel
• Query Regarding UDF Support in Spark Connect with Kubernetes as Cluster Manager Nagatomi Yasukazu
• Display a warning in EMR welcome screen Abhishek Basu
• Spark 3.5.x on Java 21? Stephen Coy
• Spark not creating staging dir for insertInto partitioned table Sanskar Modi
• [Spark Streaming]: Save the records that are dropped by watermarking in spark structured streaming Nandha Kumar
  • Re: [Spark Streaming]: Save the records that are dropped by watermarking in spark structured streaming Mich Talebzadeh
• Spark Materialized Views: Improve Query Performance and Data Management Mich Talebzadeh
• Help needed optimize spark history server performance Vikas Tharyani
• Issue with Materialized Views in Spark SQL Mich Talebzadeh
  • Re: Issue with Materialized Views in Spark SQL Walaa Eldin Moustafa
  • Re: Issue with Materialized Views in Spark SQL Jungtaek Lim
  • Re: Issue with Materialized Views in Spark SQL Mich Talebzadeh
  • Re: Issue with Materialized Views in Spark SQL Mich Talebzadeh
• ********Spark streaming issue to Elastic data********** Karthick Nk
  • Re: ********Spark streaming issue to Elastic data********** Mich Talebzadeh
  • Re: ********Spark streaming issue to Elastic data********** Karthick Nk
  • Re: ********Spark streaming issue to Elastic data********** Mich Talebzadeh
• Traceback is missing content in pyspark when invoked with UDF Indivar Mishra
• spark.sql.shuffle.partitions=auto second_co...@yahoo.com.INVALID
  • Re: spark.sql.shuffle.partitions=auto Mich Talebzadeh
• Re: Python for the kids and now PySpark Farshid Ashouri
  • Re: Python for the kids and now PySpark Meena Rajani
• [Release Question]: Estimate on 3.5.2 release? Paul Gerver
• [SparkListener] Accessing classes loaded via the '--packages' option Damien Hawes
• DataFrameReader: timestampFormat default value keen
• [spark-graphframes]: Generating incorrect edges Nijland, J.G.W. (Jelle, Student M-CS)
  • Re: [spark-graphframes]: Generating incorrect edges Mich Talebzadeh
  • Re: [spark-graphframes]: Generating incorrect edges Nijland, J.G.W. (Jelle, Student M-CS)
  • Re: [spark-graphframes]: Generating incorrect edges Mich Talebzadeh
  • Re: [spark-graphframes]: Generating incorrect edges Nijland, J.G.W. (Jelle, Student M-CS)
  • Re: [spark-graphframes]: Generating incorrect edges Stephen Coy
  • Re: [spark-graphframes]: Generating incorrect edges Mich Talebzadeh
  • Re: [spark-graphframes]: Generating incorrect edges Nijland, J.G.W. (Jelle, Student M-CS)
• How to add MaxDOP option in spark mssql JDBC Elite
  • RE: How to add MaxDOP option in spark mssql JDBC Appel, Kevin

• Earlier messages
• Later messages