user

Messages by Date

2024/02/19 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Chao Sun
2024/02/19 [Spark on Kubernetes]: Seeking Guidance on Handling Persistent Executor Failures Sri Potluri
2024/02/19 Re: Regarding Spark on Kubernetes(EKS) Jagannath Majhi
2024/02/19 Re: Regarding Spark on Kubernetes(EKS) Jagannath Majhi
2024/02/19 Re: Regarding Spark on Kubernetes(EKS) Mich Talebzadeh
2024/02/19 Re: Regarding Spark on Kubernetes(EKS) Mich Talebzadeh
2024/02/19 Re: Regarding Spark on Kubernetes(EKS) Mich Talebzadeh
2024/02/19 Re: Regarding Spark on Kubernetes(EKS) Richard Smith
2024/02/19 Regarding Spark on Kubernetes(EKS) Jagannath Majhi
2024/02/19 Re: Re-create SparkContext of SparkSession inside long-lived Spark app Mich Talebzadeh
2024/02/19 Re: Re-create SparkContext of SparkSession inside long-lived Spark app Saha, Daniel
2024/02/18 Re: Re-create SparkContext of SparkSession inside long-lived Spark app Mich Talebzadeh
2024/02/17 Re: Re-create SparkContext of SparkSession inside long-lived Spark app Jörn Franke
2024/02/17 Re: Re-create SparkContext of SparkSession inside long-lived Spark app Adam Binford
2024/02/16 Re: job uuid not unique Mich Talebzadeh
2024/02/16 Effectively append the dataset to avro directory Rushikesh Kavar
2024/02/16 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Mich Talebzadeh
2024/02/15 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Mich Talebzadeh
2024/02/14 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Chao Sun
2024/02/14 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow praveen sinha
2024/02/14 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Chao Sun
2024/02/13 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow John Zhuge
2024/02/13 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Yufei Gu
2024/02/13 Re: Facing Error org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for s3ablock-0001- Mich Talebzadeh
2024/02/13 Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Holden Karau
2024/02/13 Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow Chao Sun
2024/02/13 Re: Facing Error org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for s3ablock-0001- Bjørn Jørgensen
2024/02/13 Re: Facing Error org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for s3ablock-0001- Abhishek Singla
2024/02/12 Re: Null pointer exception while replying WAL Mich Talebzadeh
2024/02/12 Re: Null pointer exception while replying WAL nayan sharma
2024/02/11 Re: Null pointer exception while replying WAL Mich Talebzadeh
2024/02/09 Null pointer exception while replying WAL nayan sharma
2024/02/09 Re: Building an Event-Driven Real-Time Data Processor with Spark Structured Streaming and API Integration Mich Talebzadeh
2024/02/09 Building an Event-Driven Real-Time Data Processor with Spark Structured Streaming and API Integration Mich Talebzadeh
2024/02/08 performance of union vs insert into Manish Mehra
2024/02/06 [ANNOUNCE] Apache Celeborn(incubating) 0.4.0 available Fu Chen
2024/02/03 Community over Code EU 2024 Travel Assistance Applications now open! Gavin McDonald
2024/02/03 [no subject] Gavin McDonald
2024/01/31 Re: Issue in Creating Temp_view in databricks and using spark.sql(). Mich Talebzadeh
2024/01/31 Re: Issue in Creating Temp_view in databricks and using spark.sql(). Mich Talebzadeh
2024/01/31 deploy spark as cluster ali sharifi
2024/01/31 Create Custom Logs PRASHANT L
2024/01/31 Re: Issue in Creating Temp_view in databricks and using spark.sql(). Jungtaek Lim
2024/01/31 randomsplit has issue? second_co...@yahoo.com.INVALID
2024/01/30 Issue in Creating Temp_view in databricks and using spark.sql(). Karthick Nk
2024/01/29 [Spark SQL]: Crash when attempting to select PostgreSQL bpchar without length specifier in Spark 3.5.0 Lily Hahn
2024/01/29 Re: startTimestamp doesn't work when using rate-micro-batch format Mich Talebzadeh
2024/01/29 Re: startTimestamp doesn't work when using rate-micro-batch format Perfect Stranger
2024/01/28 Re: startTimestamp doesn't work when using rate-micro-batch format Mich Talebzadeh
2024/01/28 startTimestamp doesn't work when using rate-micro-batch format Perfect Stranger
2024/01/26 subscribe Sahib Aulakh
2024/01/26 subscribe Sahib Aulakh
2024/01/24 Re: [Structured Streaming] Avoid one microbatch delay with multiple stateful operations Andrzej Zera
2024/01/23 Some optimization questions about our beloved engine Spark Aissam Chia
2024/01/17 Facing Error org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for s3ablock-0001- Abhishek Singla
2024/01/17 Re: unsubscribe Крюков Виталий Семенович
2024/01/16 unsubscribe Leandro Martelli
2024/01/13 Unsubscribe Andrew Redd
2024/01/12 Re: [spark.local.dir] comma separated list does not work Andrew Petersen
2024/01/12 Re: [spark.local.dir] comma separated list does not work Andrew Petersen
2024/01/12 Re: [spark.local.dir] comma separated list does not work Koert Kuipers
2024/01/12 [spark.local.dir] comma separated list does not work Andrew Petersen
2024/01/12 [GraphFrames Spark Package]: Why is there not a distribution for Spark 3.3? Boileau, Brad
2024/01/11 Re: [Structured Streaming] Keeping checkpointing cost under control Mich Talebzadeh
2024/01/11 Re: Structured Streaming Process Each Records Individually Mich Talebzadeh
2024/01/11 Best option to process single kafka stream in parallel: PySpark Vs Dask lab22
2024/01/11 Re: [Structured Streaming] Keeping checkpointing cost under control Jungtaek Lim
2024/01/11 Re: [Structured Streaming] Avoid one microbatch delay with multiple stateful operations Jungtaek Lim
2024/01/11 Re: Okio Vulnerability in Spark 3.4.1 Bjørn Jørgensen
2024/01/10 Re: [Structured Streaming] Avoid one microbatch delay with multiple stateful operations Ant Kutschera
2024/01/10 Re: Structured Streaming Process Each Records Individually Ant Kutschera
2024/01/10 Re: Structured Streaming Process Each Records Individually Mich Talebzadeh
2024/01/10 Re: Structured Streaming Process Each Records Individually Khalid Mammadov
2024/01/10 Structured Streaming Process Each Records Individually PRASHANT L
2024/01/10 Re: [Structured Streaming] Keeping checkpointing cost under control Andrzej Zera
2024/01/10 Re: [Structured Streaming] Keeping checkpointing cost under control Mich Talebzadeh
2024/01/10 [Structured Streaming] Avoid one microbatch delay with multiple stateful operations Andrzej Zera
2024/01/10 Re: [Structured Streaming] Keeping checkpointing cost under control Andrzej Zera
2024/01/10 Re: [Structured Streaming] Keeping checkpointing cost under control Mich Talebzadeh
2024/01/10 Re: [Structured Streaming] Keeping checkpointing cost under control Andrzej Zera
2024/01/10 unsubscribe Daniel Maangi
2024/01/10 [apache-spark] documentation on File Metadata _metadata struct Jason Horner
2024/01/09 Unsubscribe qi bryce
2024/01/09 Re: Spark Structured Streaming and Flask REST API for Real-Time Data Ingestion and Analytics. Mich Talebzadeh
2024/01/09 Re: Spark Structured Streaming and Flask REST API for Real-Time Data Ingestion and Analytics. ashok34...@yahoo.com.INVALID
2024/01/09 Unsubscribe mahzad kalantari
2024/01/09 Unsubscribe Kalhara Gurugamage
2024/01/08 Re: Spark Structured Streaming and Flask REST API for Real-Time Data Ingestion and Analytics. Mich Talebzadeh
2024/01/08 Spark Structured Streaming and Flask REST API for Real-Time Data Ingestion and Analytics. Mich Talebzadeh
2024/01/08 Re: Pyspark UDF as a data source for streaming Mich Talebzadeh
2024/01/07 [ANNOUNCE] Apache Celeborn(incubating) 0.3.2 available Nicholas Jiang
2024/01/07 Re: [Structured Streaming] Keeping checkpointing cost under control Mich Talebzadeh
2024/01/07 Re: [Structured Streaming] Keeping checkpointing cost under control Andrzej Zera
2024/01/06 Re: [Structured Streaming] Keeping checkpointing cost under control Mich Talebzadeh
2024/01/05 [Structured Streaming] Keeping checkpointing cost under control Andrzej Zera
2024/01/05 Re: Issue with Spark Session Initialization in Kubernetes Deployment Mich Talebzadeh
2024/01/04 Issue with Spark Session Initialization in Kubernetes Deployment Atul Patil
2024/01/02 Unsubscribe Atlas - Samir Souidi
2023/12/30 Re: Select Columns from Dataframe in Java Grisha Weintraub
2023/12/30 Re: Select Columns from Dataframe in Java PRASHANT L
2023/12/30 Re: Select Columns from Dataframe in Java Grisha Weintraub
2023/12/29 Unsubscribe Vinti Maheshwari
2023/12/29 Re: Pyspark UDF as a data source for streaming Mich Talebzadeh
2023/12/29 Re: the life cycle shuffle Dependency murat migdisoglu
2023/12/29 Select Columns from Dataframe in Java PRASHANT L
2023/12/28 Re: Pyspark UDF as a data source for streaming Mich Talebzadeh
2023/12/28 RE: Pyspark UDF as a data source for streaming Поротиков Станислав Вячеславович
2023/12/28 Re: Pyspark UDF as a data source for streaming Mich Talebzadeh
2023/12/28 Re: Pyspark UDF as a data source for streaming Hyukjin Kwon
2023/12/27 RE: Pyspark UDF as a data source for streaming Поротиков Станислав Вячеславович
2023/12/27 Fwd: the life cycle shuffle Dependency yang chen
2023/12/27 RE: Pyspark UDF as a data source for streaming Поротиков Станислав Вячеславович
2023/12/27 RE: Pyspark UDF as a data source for streaming Поротиков Станислав Вячеславович
2023/12/27 Re: Pyspark UDF as a data source for streaming Mich Talebzadeh
2023/12/27 Pyspark UDF as a data source for streaming Поротиков Станислав Вячеславович
2023/12/26 Re: Validate spark sql Gourav Sengupta
2023/12/26 Re: Validate spark sql Mich Talebzadeh
2023/12/25 Re: Validate spark sql Bjørn Jørgensen
2023/12/25 Re: Validate spark sql Bjørn Jørgensen
2023/12/25 回复：Validate spark sql tianlangstudio
2023/12/24 Re: Validate spark sql ram manickam
2023/12/24 Re: Validate spark sql Mich Talebzadeh
2023/12/24 Re: Validate spark sql Nicholas Chammas
2023/12/21 Unsubscribe yxj1141
2023/12/21 India Scala & Big Data Job Referral sri hari kali charan Tummala
2023/12/20 About shuffle partition size Nebi Aydin
2023/12/16 [ANNOUNCE] Apache Spark 3.3.4 released Dongjoon Hyun
2023/12/16 Unsubscribe Andrew Milkowski
2023/12/15 Re: Does Spark support role-based authentication and access to Amazon S3? (Kubernetes cluster deployment) Mich Talebzadeh
2023/12/15 Re: Does Spark support role-based authentication and access to Amazon S3? (Kubernetes cluster deployment) Mich Talebzadeh
2023/12/14 Re: Architecture of Spark Connect Hyukjin Kwon
2023/12/14 Re: Architecture of Spark Connect Kezhi Xiong
2023/12/14 Re: Architecture of Spark Connect Nikhil Goyal
2023/12/14 Architecture of Spark Connect Nikhil Goyal
2023/12/13 Re: Does Spark support role-based authentication and access to Amazon S3? (Kubernetes cluster deployment) Koert Kuipers
2023/12/13 Unsubscribe kritika jain
2023/12/13 Does Spark support role-based authentication and access to Amazon S3? (Kubernetes cluster deployment) Atul Patil
2023/12/13 Does Spark support role-based authentication and access to Amazon S3? (Kubernetes cluster deployment) Patil, Atul
2023/12/12 Unsubscribe Daniel Maangi
2023/12/12 Unsubscribe Klaus Schaefers
2023/12/12 Unsubscribe Sergey Boytsov
2023/12/12 Re: Cluster-mode job compute-time/cost metrics murat migdisoglu
2023/12/12 Re: Cluster-mode job compute-time/cost metrics Jörn Franke
2023/12/11 Cluster-mode job compute-time/cost metrics Jack Wells
2023/12/11 Unsubscribe 18706753459
2023/12/11 Unsubscribe Dusty Williams
2023/12/11 unsubscribe Stevens, Clay
2023/12/11 Spark 3.1.3 with Hive dynamic partitions fails while driver moves the staged files Shay Elbaz
2023/12/11 unsubscribe Sergey Boytsov
2023/12/11 unsubscribe Klaus Schaefers
2023/12/11 Re: [EXTERNAL] Re: [EXTERNAL] Re: Spark-submit without access to HDFS Eugene Miretsky
2023/12/11 Re: [PySpark][Spark Dataframe][Observation] Why empty dataframe join doesn't let you get metrics from observation? Михаил Кулаков
2023/12/11 Re: [EXTERNAL] Re: Spark-submit without access to HDFS Mich Talebzadeh
2023/12/10 Re: [EXTERNAL] Re: Spark-submit without access to HDFS Eugene Miretsky
2023/12/10 Re: [EXTERNAL] Re: Spark-submit without access to HDFS Eugene Miretsky
2023/12/10 unsubscribe Rajanikant V
2023/12/09 Re: Spark on Java 17 Jörn Franke
2023/12/09 Re: Spark on Java 17 Faiz Halde
2023/12/09 Re: Spark on Java 17 Jörn Franke
2023/12/09 RE: Spark on Java 17 Luca Canali
2023/12/07 Spark on Java 17 Faiz Halde
2023/12/06 Re: SSH Tunneling issue with Apache Spark Venkatesan Muniappan
2023/12/06 Re: SSH Tunneling issue with Apache Spark Nicholas Chammas
2023/12/06 Re: SSH Tunneling issue with Apache Spark Venkatesan Muniappan
2023/12/05 SSH Tunneling issue with Apache Spark Venkatesan Muniappan
2023/12/05 Re: ordering of rows in dataframe Enrico Minack
2023/12/05 ordering of rows in dataframe Som Lima
2023/12/05 Re: [PySpark][Spark Dataframe][Observation] Why empty dataframe join doesn't let you get metrics from observation? Enrico Minack
2023/12/04 Re: Spark-Connect: Param `--packages` does not take effect for executors. Holden Karau
2023/12/04 ML advice Zahid Rahman
2023/12/04 Re: Do we have any mechanism to control requests per second for a Kafka connect sink? Yeikel Santana
2023/12/04 Do we have any mechanism to control requests per second for a Kafka connect sink? Yeikel Santana
2023/12/04 Re: Spark-Connect: Param `--packages` does not take effect for executors. Aironman DirtDiver
2023/12/04 Spark-Connect: Param `--packages` does not take effect for executors. Xiaolong Wang
2023/12/04 Re: [PySpark][Spark Dataframe][Observation] Why empty dataframe join doesn't let you get metrics from observation? Enrico Minack
2023/12/02 [PySpark][Spark Dataframe][Observation] Why empty dataframe join doesn't let you get metrics from observation? Михаил Кулаков
2023/12/02 Re: [FYI] SPARK-45981: Improve Python language test coverage Hyukjin Kwon
2023/12/01 ML using Spark Connect Faiz Halde
2023/12/01 [FYI] SPARK-45981: Improve Python language test coverage Dongjoon Hyun
2023/12/01 Re: [Streaming (DStream) ] : Does Spark Streaming supports pause/resume consumption of message from Kafka? Mich Talebzadeh
2023/12/01 [Streaming (DStream) ] : Does Spark Streaming supports pause/resume consumption of message from Kafka? Saurabh Agrawal (180813)
2023/11/30 Re:[ANNOUNCE] Apache Spark 3.4.2 released beliefer
2023/11/30 [ANNOUNCE] Apache Spark 3.4.2 released Dongjoon Hyun
2023/11/30 unsubscribe Dharmin Siddesh J
2023/11/29 [sql] how to connect query stage to Spark job/stages? Chenghao Lyu
2023/11/29 Re: Tuning Best Practices Bryant Wright
2023/11/29 RE: Re: Spark Compatibility with Spring Boot 3.x Guru Panda
2023/11/28 Re: Tuning Best Practices Jack Goodson
2023/11/28 Re: Classpath isolation per SparkSession without Spark Connect Pasha Finkelshtein
2023/11/28 Tuning Best Practices Bryant Wright
2023/11/28 Re: Classpath isolation per SparkSession without Spark Connect Faiz Halde
2023/11/28 Re: Classpath isolation per SparkSession without Spark Connect Pasha Finkelshtein
2023/11/27 Re: Classpath isolation per SparkSession without Spark Connect Faiz Halde
2023/11/27 Re: Classpath isolation per SparkSession without Spark Connect Holden Karau
2023/11/27 Classpath isolation per SparkSession without Spark Connect Faiz Halde
2023/11/24 Re: Spark structured streaming tab is missing from spark web UI Jungtaek Lim
2023/11/24 [Spark-sql 3.2.4] Wrong Statistic INFO From 'ANALYZE TABLE' Command Nick Luo
2023/11/22 Query fails on CASE statement depending on order of summed columns Evgenii Ignatev
2023/11/21 Re: How exactly does dropDuplicatesWithinWatermark work? Jungtaek Lim
2023/11/19 How exactly does dropDuplicatesWithinWatermark work? Perfect Stranger