user
Thread
Date
Earlier messages
Later messages
Messages by Thread
Re: Viewing UI for spark jobs running on K8s
Qian Sun
ChatGPT and prediction of Spark future
Mich Talebzadeh
Re: ChatGPT and prediction of Spark future
Winston Lai
Re: ChatGPT and prediction of Spark future
Mich Talebzadeh
Structured streaming append mode picture question
Hill Liu
JDK version support information
Poorna Murali
Re: JDK version support information
Aironman DirtDiver
Re: JDK version support information
Poorna Murali
Re: JDK version support information
Sean Owen
Re: maven with Spark 3.4.0 fails compilation
Bjørn Jørgensen
Re: maven with Spark 3.4.0 fails compilation
Mich Talebzadeh
Re: Re: maven with Spark 3.4.0 fails compilation
Lingzhe Sun
Re: Re: maven with Spark 3.4.0 fails compilation
Bjørn Jørgensen
Re: Re: maven with Spark 3.4.0 fails compilation
Mich Talebzadeh
Re: Re: maven with Spark 3.4.0 fails compilation
Bjørn Jørgensen
[Spark Structured Streaming]: Dynamic Scaling of Executors
Aishwarya Panicker
Re: [Spark Structured Streaming]: Dynamic Scaling of Executors
Mich Talebzadeh
Re: [Spark Structured Streaming]: Dynamic Scaling of Executors
Aishwarya Panicker
Re: [MLlib] how-to find implementation of Decision Tree Regressor fit function
Sean Owen
Dynamic value as the offset of lag() function
Nipuna Shantha
Incremental Value dependents on another column of Data frame Spark
Nipuna Shantha
Re: Incremental Value dependents on another column of Data frame Spark
Raghavendra Ganesh
Re: Incremental Value dependents on another column of Data frame Spark
Enrico Minack
cannot load model using pyspark
second_co...@yahoo.com.INVALID
Data Stream Processing applications testing
Alexandre Strapacao Guedes Vianna
Understanding Spark S3 Read Performance
Shashank Rao
RE: Understanding Spark S3 Read Performance
info
Spark shuffle and inevitability of writing to Disk
Mich Talebzadeh
Re: Spark shuffle and inevitability of writing to Disk
Mich Talebzadeh
[spark-core] Can executors recover/reuse shuffle files upon failure?
Faiz Halde
Re: [spark-core] Can executors recover/reuse shuffle files upon failure?
Mich Talebzadeh
Re: [spark-core] Can executors recover/reuse shuffle files upon failure?
vaquar khan
RE: Re: [spark-core] Can executors recover/reuse shuffle files upon failure?
Maksym M
Re: Re: [spark-core] Can executors recover/reuse shuffle files upon failure?
Mich Talebzadeh
Re: Re: [spark-core] Can executors recover/reuse shuffle files upon failure?
Mich Talebzadeh
Pyspark cluster mode on standalone deployment
خالد القحطاني
Shuffle with Window().partitionBy(<column>)
ashok34...@yahoo.com.INVALID
Re: Shuffle with Window().partitionBy(<column>)
Rauf Khan
Re: Shuffle with Window().partitionBy(<column>)
ashok34...@yahoo.com.INVALID
Error while merge in delta table
Karthick Nk
Re: Error while merge in delta table
Jacek Laskowski
Re: Error while merge in delta table
Farhan Misarwala
Re: Error while merge in delta table
Karthick Nk
Re: Error while merge in delta table
Farhan Misarwala
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Winston Lai
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Nitin Siwach
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Mich Talebzadeh
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Nitin Siwach
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Mich Talebzadeh
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Nitin Siwach
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Mich Talebzadeh
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Nitin Siwach
Re: Does spark read the same file twice, if two stages are using the same DataFrame?
Mich Talebzadeh
Does spark read the same file twice, if two stages are using the same DataFrame?
Vijay B
Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count?
Yong Zhang
Re: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count?
Mich Talebzadeh
Re: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count?
Yong Zhang
Re: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count?
Yong Zhang
RE: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count?
Vijay B
Write DataFrame with Partition and choose Filename in PySpark
Marco Costantini
Re: Write DataFrame with Partition and choose Filename in PySpark
Mich Talebzadeh
Re: Write DataFrame with Partition and choose Filename in PySpark
Marco Costantini
Re: Write DataFrame with Partition and choose Filename in PySpark
Mich Talebzadeh
Re: Write DataFrame with Partition and choose Filename in PySpark
Marco Costantini
Re: Write DataFrame with Partition and choose Filename in PySpark
Mich Talebzadeh
Write custom JSON from DataFrame in PySpark
Marco Costantini
Re: Write custom JSON from DataFrame in PySpark
Enrico Minack
Re: Write custom JSON from DataFrame in PySpark
Marco Costantini
CVE-2023-32007: Apache Spark: Shell command injection via Spark UI
Arnout Engelen
Change column values using several when conditions
marc nicole
Re: Change column values using several when conditions
Bjørn Jørgensen
How to change column values using several when conditions ?
marc nicole
Any experience with K8s Remote Shuffling Service at scale?
Andrey Gourine
How to read text files with GBK encoding in the spark core
lianyou1...@126.com
Tensorflow on Spark CPU
second_co...@yahoo.com.INVALID
Re: Tensorflow on Spark CPU
Sean Owen
Re: Tensorflow on Spark CPU
second_co...@yahoo.com.INVALID
Re: Tensorflow on Spark CPU
Sean Owen
driver and executors shared same Kubernetes PVC
second_co...@yahoo.com.INVALID
***pyspark.sql.functions.monotonically_increasing_id()***
Karthick Nk
Re: ***pyspark.sql.functions.monotonically_increasing_id()***
Winston Lai
config: minOffsetsPerTrigger not working
Abhishek Singla
Re: config: minOffsetsPerTrigger not working
Mich Talebzadeh
Re: config: minOffsetsPerTrigger not working
Abhishek Singla
What is the best way to organize a join within a foreach?
Marco Costantini
What is the best way to organize a join within a foreach?
Marco Costantini
Re: What is the best way to organize a join within a foreach?
Mich Talebzadeh
Re: What is the best way to organize a join within a foreach?
Marco Costantini
Re: What is the best way to organize a join within a foreach?
Mich Talebzadeh
Re: What is the best way to organize a join within a foreach?
Marco Costantini
Re: What is the best way to organize a join within a foreach?
Mich Talebzadeh
Re: What is the best way to organize a join within a foreach?
Marco Costantini
Re: What is the best way to organize a join within a foreach?
Mich Talebzadeh
Re: What is the best way to organize a join within a foreach?
ayan guha
Re: What is the best way to organize a join within a foreach?
Mich Talebzadeh
Re: What is the best way to organize a join within a foreach?
Marco Costantini
Re: What is the best way to organize a join within a foreach?
Mich Talebzadeh
Re: What is the best way to organize a join within a foreach?
Amit Joshi
Reg: create spark using virtual machine through chef
sunkara akhil sai teja
Unsubcribing
phiroc
Use Spark Aggregator in PySpark
Thomas Wang
Re: Use Spark Aggregator in PySpark
Enrico Minack
Spark Aggregator with ARRAY<BOOLEAN> input and ARRAY<LONG> output
Thomas Wang
Re: Spark Aggregator with ARRAY<BOOLEAN> input and ARRAY<LONG> output
Raghavendra Ganesh
Re: Spark Aggregator with ARRAY<BOOLEAN> input and ARRAY<LONG> output
Thomas Wang
State of GraphX and GraphFrames
g
Dependency injection for spark executors
Deepak Patankar
Partition by on dataframe causing a Sort
Nikhil Goyal
Re: Partition by on dataframe causing a Sort
Nikhil Goyal
[Spark on SBT] Executor just keeps running
Dhruv Singla
Re: [Spark on SBT] Executor just keeps running
Dhruv Singla
Spark Log Shipper to Cloud Bucket
Jayabindu Singh
Re: Spark Multiple Hive Metastore Catalog Support
Ankit Gupta
Re: Spark Multiple Hive Metastore Catalog Support
Elliot West
Re: Spark Multiple Hive Metastore Catalog Support
Cheng Pan
Re: Spark Multiple Hive Metastore Catalog Support
Ankit Gupta
CVE-2023-22946: Apache Spark proxy-user privilege escalation from malicious configuration class
Sean R. Owen
Scala commands syntax shortcuts(alias)
Ankit Singla
Spark Kubernetes Operator
Yuval Itzchakov
Re: Spark Kubernetes Operator
Mich Talebzadeh
Re: Spark Kubernetes Operator
Yuval Itzchakov
RE: Spark Kubernetes Operator
Aldo Culquicondor
How to create spark udf use functioncatalog?
??????
Re: How to create spark udf use functioncatalog?
Jacek Laskowski
How to create spark udf use functioncatalog?
tzxxh
[ANNOUNCE] Apache Spark 3.2.4 released
Dongjoon Hyun
_spark_metadata path issue with S3 lifecycle policy
Yuval Itzchakov
Re: _spark_metadata path issue with S3 lifecycle policy
Yuri Oleynikov (יורי אולייניקוב)
Re: _spark_metadata path issue with S3 lifecycle policy
Yuval Itzchakov
Re: _spark_metadata path issue with S3 lifecycle policy
Yuri Oleynikov (יורי אולייניקוב)
Re: _spark_metadata path issue with S3 lifecycle policy
Yuval Itzchakov
Accessing python runner file in AWS EKS kubernetes cluster as in local://
Mich Talebzadeh
Re: Accessing python runner file in AWS EKS kubernetes cluster as in local://
Bjørn Jørgensen
Re: Accessing python runner file in AWS EKS kubernetes cluster as in local://
Mich Talebzadeh
Re: Accessing python runner file in AWS EKS kubernetes cluster as in local://
Mich Talebzadeh
Re: How to determine the function of tasks on each stage in an Apache Spark application?
Jacek Laskowski
Re: How to determine the function of tasks on each stage in an Apache Spark application?
Maytas Monsereenusorn
Re: How to determine the function of tasks on each stage in an Apache Spark application?
Trường Trần Phan An
Re: How to determine the function of tasks on each stage in an Apache Spark application?
Jacek Laskowski
Re: How to determine the function of tasks on each stage in an Apache Spark application?
Trường Trần Phan An
PySpark tests are failed with the java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.sources.FakeSourceOne not found
Ranga Reddy
Non string type partitions
Charles vinodh
Re: Non string type partitions
Chitral Verma
Re: Non string type partitions
Charles vinodh
Re: Non string type partitions
Charles vinodh
Re: Non string type partitions
Bjørn Jørgensen
[ANNOUNCE] Apache Uniffle(Incubating) 0.7.0 available
Junfan Zhang
[SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI
Chenghao Lyu
Re: [SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI
Chitral Verma
Re: [SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI
Jacek Laskowski
spark streaming and kinesis integration
Rajesh Katkar
Re: spark streaming and kinesis integration
Mich Talebzadeh
Re: spark streaming and kinesis integration
Rajesh Katkar
Re: spark streaming and kinesis integration
Mich Talebzadeh
RE: spark streaming and kinesis integration
Jonske, Kurt
Re: spark streaming and kinesis integration
Rajesh Katkar
Re: spark streaming and kinesis integration
Mich Talebzadeh
Re: spark streaming and kinesis integration
Mich Talebzadeh
Re: Re: spark streaming and kinesis integration
Lingzhe Sun
Re: Re: spark streaming and kinesis integration
Rajesh Katkar
Re: Re: spark streaming and kinesis integration
Yi Huang
Re: Re: spark streaming and kinesis integration
孙令哲
Re: Re: spark streaming and kinesis integration
Mich Talebzadeh
Raise exception whilst casting instead of defaulting to null
Yeachan Park
Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application
me
Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application
me
Re: Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application
Mich Talebzadeh
Re: Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application
Andrew Redd
Potability of dockers built on different cloud platforms
ashok34...@yahoo.com.INVALID
Re: Potability of dockers built on different cloud platforms
Ken Peng
Re: Potability of dockers built on different cloud platforms
Mich Talebzadeh
Looping through a series of telephone numbers
Philippe de Rochambeau
Re: Looping through a series of telephone numbers
Mich Talebzadeh
Re: Looping through a series of telephone numbers
Philippe de Rochambeau
Re: Looping through a series of telephone numbers
Sean Owen
Re: Looping through a series of telephone numbers
Gera Shegalov
Re: Looping through a series of telephone numbers
Mich Talebzadeh
Re: Looping through a series of telephone numbers
Bjørn Jørgensen
Re: Looping through a series of telephone numbers
Philippe de Rochambeau
Re: Looping through a series of telephone numbers
Mich Talebzadeh
Re: Looping through a series of telephone numbers
Anastasios Zouzias
Re: Looping through a series of telephone numbers
Philippe de Rochambeau
Re: Help me learn about JOB TASK and DAG in Apache Spark
Mich Talebzadeh
Re: Help me learn about JOB TASK and DAG in Apache Spark
AN-TRUONG Tran Phan
Re: Help me learn about JOB TASK and DAG in Apache Spark
Mich Talebzadeh
Re: Help me learn about JOB TASK and DAG in Apache Spark
Khalid Mammadov
Re: Help me learn about JOB TASK and DAG in Apache Spark
Mich Talebzadeh
Creating InMemory relations with data in ColumnarBatches
praveen sinha
Re: Creating InMemory relations with data in ColumnarBatches
Mich Talebzadeh
Re: Creating InMemory relations with data in ColumnarBatches
praveen sinha
Re: Creating InMemory relations with data in ColumnarBatches
Bobby Evans
[ANNOUNCE] Apache Celeborn(incubating) 0.2.1 available
rexxiong
spark.catalog.listFunctions type signatures
Guillaume Masse
Re: spark.catalog.listFunctions type signatures
Jacek Laskowski
Re: spark.catalog.listFunctions type signatures
Guillaume Masse
What is the range of the PageRank value of graphx
lee
Re: What is the range of the PageRank value of graphx
Sean Owen
Re: What is the range of the PageRank value of graphx
lee
Slack for PySpark users
keen
Re: Slack for PySpark users
Hyukjin Kwon
Earlier messages
Later messages