user  

Messages by Thread

• Issues with CharType() and VarcharType() datatype Souvik Saha
• Compatibility Issue: Spark 3.5.2 Schema Recognition vs. Spark 3.4.0 with Hive Metastore (Case Sensitivity) Mich Talebzadeh
• Spark SQL readSideCharPadding issue while reading ENUM column from mysql Suyash Ajmera
• Re: is it possible to run spark2 on EMR 7.2.0 Prem Sahoo
• ERROR: GROUP BY position 0 is not in select list , when using catalyst parser Rommel Holmes
  • Re: ERROR: GROUP BY position 0 is not in select list , when using catalyst parser Sudhanshu
  • Re: ERROR: GROUP BY position 0 is not in select list , when using catalyst parser joshita mishra
• [Spark Core]: Spark 3.5.0 incompatible with Embedded Kafka Adesh Dsilva
• Spark 3.2.1 vs Spark 3.5.2 Stephen Coy
• [CONNECT] Why Can't We Specify Cluster Deploy Mode for Spark Connect? Nagatomi Yasukazu
  • Re: [CONNECT] Why Can't We Specify Cluster Deploy Mode for Spark Connect? Nagatomi Yasukazu
  • Re: [CONNECT] Why Can't We Specify Cluster Deploy Mode for Spark Connect? Prabodh Agarwal
  • Re: [CONNECT] Why Can't We Specify Cluster Deploy Mode for Spark Connect? Nagatomi Yasukazu
• Spark Thrift Server - Not Scaling Down Executors 3.4.2+ Jayabindu Singh
  • Re: Spark Thrift Server - Not Scaling Down Executors 3.4.2+ Cheng Pan
• Setting forarch microbatch processing data count in structured streaming Karthick Nk
• Question about Releases and EOL Miles Fryhover (V)
  • Re: Question about Releases and EOL Mich Talebzadeh
• Fwd: [ANNOUNCE] Apache Sedona 1.6.1 released Jia Yu
• unable to deploy Pyspark application on GKE, Spark installed using bitnami helm chart karan alang
  • Re: unable to deploy Pyspark application on GKE, Spark installed using bitnami helm chart Mat Schaffer
• Job Opportunities in India or UK with Tier 2 Sponsorship - Spark Expert sri hari kali charan Tummala
• Re: Batch to Kafka Rommel Holmes
• Need support for vulnerability CVE-2023-39410 Ravichandran, Abhirami
• Expected Release date for Spark 4? Vikash Lavaniya
• Hitting SPARK-45858 on Kubernetes - Unavoidable bug or misconfiguration? Aaron Grubb
  • Re: Hitting SPARK-45858 on Kubernetes - Unavoidable bug or misconfiguration? Aaron Grubb
  • Re: Hitting SPARK-45858 on Kubernetes - Unavoidable bug or misconfiguration? Cheng Pan
  • Re: Hitting SPARK-45858 on Kubernetes - Unavoidable bug or misconfiguration? Cheng Pan
• Spark Job Fails while writing data to a S3 location in Parquet Nipuna Shantha
• Handling load distribution and addressing data skew. Karthick
  • Re: Handling load distribution and addressing data skew. Raghavendra Ganesh
• Issue with pyspark : Add custom shutdown hook aarushi agarwal
• Spark Reads from MapR and Write to MinIO fails for few batches Prem Sahoo
  • Re: Spark Reads from MapR and Write to MinIO fails for few batches Prem Sahoo
  • Re: Spark Reads from MapR and Write to MinIO fails for few batches Prem Sahoo
  • Re: Spark Reads from MapR and Write to MinIO fails for few batches Prem Sahoo
• Redundant(?) shuffle after join Shay Elbaz
  • Re: Redundant(?) shuffle after join Mich Talebzadeh
  • Re: Redundant(?) shuffle after join Shay Elbaz
  • Re: Redundant(?) shuffle after join Mich Talebzadeh
  • Re: [External] Re: Redundant(?) shuffle after join Ofir Manor
• Need help understanding tuning docs Sreyan Chakravarty
  • Re: Need help understanding tuning docs Subhasis Mukherjee
• [Spark Connect ] Date Data type formatting issue Ilango
• [ANNOUNCE] Apache Spark 3.5.2 released Kent Yao
  • Re: [ANNOUNCE] Apache Spark 3.5.2 released Xiao Li
• Is it possible to configure batch size immutability between two `mapInPandas`? Azamat G.
• Re: Spark 3.5.0 bug - Writing a small paraquet dataframe to storage using spark 3.5.0 taking too long Bijoy Deb
• [spark connect] unable to utilize stand alone cluster Ilango
  • Re: [spark connect] unable to utilize stand alone cluster Prabodh Agarwal
  • Re: [spark connect] unable to utilize stand alone cluster Ilango
  • Re: [spark connect] unable to utilize stand alone cluster Prabodh Agarwal
  • Re: [spark connect] unable to utilize stand alone cluster Ilango
  • Re: [spark connect] unable to utilize stand alone cluster Prabodh Agarwal
• Spark History CORS header ‘Access-Control-Allow-Origin’ missing Thomas Mauran
• dynamically infer json data not working as expected Perez
  • Re: dynamically infer json data not working as expected Perez
  • Re: dynamically infer json data not working as expected Mich Talebzadeh
  • Re: dynamically infer json data not working as expected Perez
  • Re: dynamically infer json data not working as expected Perez
• Feature Engineering for Data Engineers: Building Blocks for ML Success Mich Talebzadeh
• Error While Running Merge Statement With Iceberg PRASHANT L
• A code change for spark ui in Sql tab Donvi
• Question about installing Apache Spark [PySpark] computer requirements mike Jadoo
  • Re: Question about installing Apache Spark [PySpark] computer requirements Sadha Chilukoori
  • Re: Question about installing Apache Spark [PySpark] computer requirements mike Jadoo
  • Re: Question about installing Apache Spark [PySpark] computer requirements Sadha Chilukoori
  • Re: Question about installing Apache Spark [PySpark] computer requirements Meena Rajani
• [ANNOUNCE] Apache Celeborn 0.5.1 available Ethan Feng
• [ANNOUNCE] Apache Celeborn 0.4.2 available Fu Chen
• [Spark Connect] connection issue Ilango
  • Re: [Spark Connect] connection issue Prabodh Agarwal
  • Re: [Spark Connect] connection issue Ilango
  • Re: [Spark Connect] connection issue Prabodh Agarwal
  • Re: [Spark Connect] connection issue Ilango
  • Re: [Spark Connect] connection issue Prabodh Agarwal
• Issue with comparing structs (possible bug) Dhruv Singla
  • Re: Issue with comparing structs (possible bug) Dhruv Singla
• [ANNOUNCE] Apache Kyuubi v1.9.2 is available Fu Chen
• [Spark SQL]: Why the OptimizeSkewedJoin rule does not optimize FullOuterJoin? 王仲轩(万章)
• Re: issue forwarding SPARK_CONF_DIR to start workers Patrice Duroux
  • Re: issue forwarding SPARK_CONF_DIR to start workers Holden Karau
• [spark connect] issue in testing spark connect Ilango
• Error on Spark history UI api when used with Apache Knox thomas.mau...@etu.umontpellier.fr.INVALID
• problem using spark 3.4 with spots wafa gabouj
• binary stream Jeff Pang
• heap fragmentation in G1 aka.fe2s
• Help wanted on securing spark with Apache Knox / JWT Thomas Mauran
  • Re: Help wanted on securing spark with Apache Knox / JWT Adam Binford
• Sometimes TaskContext configuration is almost empty Asaf Mesika
• [Issue] Spark SQL - broadcast failure Sudharshan V
  • Re: [Issue] Spark SQL - broadcast failure Mich Talebzadeh
  • Re: [Issue] Spark SQL - broadcast failure Meena Rajani
  • Re: [Issue] Spark SQL - broadcast failure Sudharshan V
  • Re: [Issue] Spark SQL - broadcast failure Sudharshan V
  • Re: [Issue] Spark SQL - broadcast failure Sudharshan V
• running snowflake query using spark connect on a standalone cluster Prabodh Agarwal
• Need help to confirm vulnerable issue Will.Qin
• Assistance needed with PySpark Streaming for retrieving past messages. Mai Trang
• How to use spark.connect.Plan from dependency in a custom Spark Connect RelationPlugin? Sem
• AWS Glue and Python Perez
• Does Spark 4.0 add Sparkstreaming SQL ????
• log4j2.properties file load fails when upgrading from 3.3.0 to 3.5.1 Edgar H
• Help Needed: Distributed Logging in Spark Application Zsuzsanna D
• AttributeError: 'MulticlassMetrics' object has no attribute '_sc' Azhuvath, RajeevX
  • Re: AttributeError: 'MulticlassMetrics' object has no attribute '_sc' Saurabh Kumar
• Pyspark DataFrame.drop wrong type hints Oliver Beagley
  • Pyspark DataFrame.drop wrong type hints Oliver Beagley
• Help in understanding Exchange in Spark UI Dhruv Singla
  • Re: Help in understanding Exchange in Spark UI Mich Talebzadeh
• Spark Decommission Rajesh Mahindra
  • Re: Spark Decommission Khaldi, Ahmed
  • Re: Spark Decommission Rajesh Mahindra
• [K8S] Divergense in dockerfiles between official repositories. Andrei L
• Update mode in spark structured streaming Om Prakash
  • Re: Update mode in spark structured streaming Mich Talebzadeh
• Unable to load MongoDB atlas data via PySpark because of BsonString error Perez
  • Re: Unable to load MongoDB atlas data via PySpark because of BsonString error Perez
• OOM issue in Spark Driver Karthick Nk
  • Re: OOM issue in Spark Driver Andrzej Zera
  • Re: Re: OOM issue in Spark Driver Mich Talebzadeh
• 7368396 - Apache Spark 3.5.1 (Support) SANTOS SOUZA, ALEX
  • Re: 7368396 - Apache Spark 3.5.1 (Support) Sadha Chilukoori
• Kubernetes cluster: change log4j configuration using uploaded `--files` Jennifer Wirth
  • Re: Kubernetes cluster: change log4j configuration using uploaded `--files` Mich Talebzadeh
• [SPARK-48423] Unable to save ML Pipeline to azure blob storage Chhavi Bansal
  • Re: [SPARK-48423] Unable to save ML Pipeline to azure blob storage Chhavi Bansal
• [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Chhavi Bansal
  • Re: [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Someshwar Kale
  • Re: [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Chhavi Bansal
  • Re: [SPARK-48463] Mllib Feature transformer failing with nested dataset (Dot notation) Someshwar Kale
• Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Mich Talebzadeh
  • Re: Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Mich Talebzadeh
  • Re: Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Perez
  • Re: Do we need partitioning while loading data from JDBC sources? Gourav Sengupta
• Inquiry Regarding Security Compliance of Apache Spark Docker Image Tonmoy Sagar
• Classification request VARGA, Sara
  • Re: Classification request Artemis User
  • Re: Classification request Dirk-Willem van Gulik
• [ANNOUNCE] Announcing Apache Spark 4.0.0-preview1 Wenchen Fan
• [ANNOUNCE] Apache Kyuubi released 1.9.1 Cheng Pan
• Terabytes data processing via Glue Perez
  • Re: Terabytes data processing via Glue Perez
  • Re: Terabytes data processing via Glue Russell Jurney
  • Re: Terabytes data processing via Glue Perez
• [apache-spark][spark-dataframe] DataFrameWriter.partitionBy does not guarantee previous sort result leeyc0
• [Spark on k8s] A issue of k8s resource creation order Tao Yang
• Tox and Pyspark Perez
• Spark Protobuf Deserialization Satyam Raj
  • Re: Spark Protobuf Deserialization Sandish Kumar HN
• [Spark SQL]: Does Spark support processing records with timestamp NULL in stateful streaming? Juan Casse
  • Re: [Spark SQL]: Does Spark support processing records with timestamp NULL in stateful streaming? Mich Talebzadeh
• OOM concern Perez
  • Re: OOM concern Meena Rajani
  • Re: OOM concern Russell Jurney
  • Re: OOM concern Perez
  • Re: OOM concern Mich Talebzadeh
  • Re: OOM concern Perez
  • Re: OOM concern Mich Talebzadeh
  • Re: OOM concern Russell Jurney
  • Re: OOM concern Perez
• Subject: [Spark SQL] [Debug] Spark Memory Issue with DataFrame Processing Gaurav Madan
  • Re: Subject: [Spark SQL] [Debug] Spark Memory Issue with DataFrame Processing Mich Talebzadeh
  • Re: Subject: [Spark SQL] [Debug] Spark Memory Issue with DataFrame Processing Shay Elbaz
• Can Spark Catalog Perform Multimodal Database Query Analysis ????
  • Re: Can Spark Catalog Perform Multimodal Database Query Analysis Mich Talebzadeh
• BUG :: UI Spark Prem Sahoo
  • Re: BUG :: UI Spark Prem Sahoo
  • Re: BUG :: UI Spark Prem Sahoo
  • Re: BUG :: UI Spark Sathi Chowdhury
  • Re: BUG :: UI Spark Mich Talebzadeh
  • Re: BUG :: UI Spark Mich Talebzadeh
  • Re: BUG :: UI Spark Mich Talebzadeh
• [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
  • Re: [s3a] Spark is not reading s3 object content Mich Talebzadeh
  • Re: [s3a] Spark is not reading s3 object content Amin Mosayyebzadeh
• Remote File change detection in S3 when spark queries are running and parquet files in S3 changes Raghvendra Yadav
• [ANNOUNCE] Apache Celeborn 0.4.1 available Nicholas Jiang
• Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming ashok34...@yahoo.com.INVALID
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Mich Talebzadeh
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Tathagata Das
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Tathagata Das
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Anil Dasari
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Mich Talebzadeh
  • Re: Dstream HasOffsetRanges equivalent in Structured streaming Mich Talebzadeh
• Re: EXT: Dual Write to HDFS and MinIO in faster way Prem Sahoo

• Earlier messages
• Later messages