user  

Messages by Thread

• • Re: How to read excel file in PySpark Bjørn Jørgensen
  • Re: How to read excel file in PySpark Mich Talebzadeh
  • Re: How to read excel file in PySpark Sean Owen
  • Re: How to read excel file in PySpark Mich Talebzadeh
  • Re: How to read excel file in PySpark Bjørn Jørgensen
  • Re: How to read excel file in PySpark Mich Talebzadeh
• implement a distribution without shuffle like RDD.coalesce for DataSource V2 write Pengfei Li
  • Re: implement a distribution without shuffle like RDD.coalesce for DataSource V2 write Mich Talebzadeh
  • Re: implement a distribution without shuffle like RDD.coalesce for DataSource V2 write Mich Talebzadeh
• TAC Applications for Community Over Code North America and Asia now open Gavin McDonald
• Fwd: iceberg queries Gaurav Agarwal
• Spark using iceberg Gaurav Agarwal
  • Re: Spark using iceberg Gaurav Agarwal
• Announcing the Community Over Code 2023 Streaming Track James Hughes
• Apache Spark not reading UTC timestamp from MongoDB correctly karan alang
  • Re: Apache Spark not reading UTC timestamp from MongoDB correctly Sean Owen
  • Re: Apache Spark not reading UTC timestamp from MongoDB correctly Enrico Minack
• Getting SparkRuntimeException: Unexpected value for length in function slice: length must be greater than or equal to 0 Bariudin, Daniel
• [Feature Request] create *permanent* Spark View from DataFrame via PySpark keen
  • Re: [Feature Request] create *permanent* Spark View from DataFrame via PySpark keen
  • Re: [Feature Request] create *permanent* Spark View from DataFrame via PySpark Mich Talebzadeh
  • Re: [Feature Request] create *permanent* Spark View from DataFrame via PySpark Wenchen Fan
• Comparison of Trino, Spark, and Hive-MR3 Sungwoo Park
• Viewing UI for spark jobs running on K8s Nikhil Goyal
  • Re: Viewing UI for spark jobs running on K8s Qian Sun
• ChatGPT and prediction of Spark future Mich Talebzadeh
  • Re: ChatGPT and prediction of Spark future Winston Lai
  • Re: ChatGPT and prediction of Spark future Mich Talebzadeh
• Structured streaming append mode picture question Hill Liu
• JDK version support information Poorna Murali
  • Re: JDK version support information Aironman DirtDiver
  • Re: JDK version support information Poorna Murali
  • Re: JDK version support information Sean Owen
• Re: maven with Spark 3.4.0 fails compilation Bjørn Jørgensen
  • Re: maven with Spark 3.4.0 fails compilation Mich Talebzadeh
  • Re: Re: maven with Spark 3.4.0 fails compilation Lingzhe Sun
  • Re: Re: maven with Spark 3.4.0 fails compilation Bjørn Jørgensen
  • Re: Re: maven with Spark 3.4.0 fails compilation Mich Talebzadeh
  • Re: Re: maven with Spark 3.4.0 fails compilation Bjørn Jørgensen
• [Spark Structured Streaming]: Dynamic Scaling of Executors Aishwarya Panicker
  • Re: [Spark Structured Streaming]: Dynamic Scaling of Executors Mich Talebzadeh
  • Re: [Spark Structured Streaming]: Dynamic Scaling of Executors Aishwarya Panicker
• Re: [MLlib] how-to find implementation of Decision Tree Regressor fit function Sean Owen
• Dynamic value as the offset of lag() function Nipuna Shantha
• Incremental Value dependents on another column of Data frame Spark Nipuna Shantha
  • Re: Incremental Value dependents on another column of Data frame Spark Raghavendra Ganesh
  • Re: Incremental Value dependents on another column of Data frame Spark Enrico Minack
• cannot load model using pyspark second_co...@yahoo.com.INVALID
• Data Stream Processing applications testing Alexandre Strapacao Guedes Vianna
• Understanding Spark S3 Read Performance Shashank Rao
  • RE: Understanding Spark S3 Read Performance info
• Spark shuffle and inevitability of writing to Disk Mich Talebzadeh
  • Re: Spark shuffle and inevitability of writing to Disk Mich Talebzadeh
• [spark-core] Can executors recover/reuse shuffle files upon failure? Faiz Halde
  • Re: [spark-core] Can executors recover/reuse shuffle files upon failure? Mich Talebzadeh
  • Re: [spark-core] Can executors recover/reuse shuffle files upon failure? vaquar khan
  • RE: Re: [spark-core] Can executors recover/reuse shuffle files upon failure? Maksym M
  • Re: Re: [spark-core] Can executors recover/reuse shuffle files upon failure? Mich Talebzadeh
  • Re: Re: [spark-core] Can executors recover/reuse shuffle files upon failure? Mich Talebzadeh
• Pyspark cluster mode on standalone deployment خالد القحطاني
• Shuffle with Window().partitionBy(<column>) ashok34...@yahoo.com.INVALID
  • Re: Shuffle with Window().partitionBy(<column>) Rauf Khan
  • Re: Shuffle with Window().partitionBy(<column>) ashok34...@yahoo.com.INVALID
• Error while merge in delta table Karthick Nk
  • Re: Error while merge in delta table Jacek Laskowski
  • Re: Error while merge in delta table Farhan Misarwala
  • Re: Error while merge in delta table Karthick Nk
  • Re: Error while merge in delta table Farhan Misarwala
• Re: Does spark read the same file twice, if two stages are using the same DataFrame? Winston Lai
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Nitin Siwach
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Mich Talebzadeh
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Nitin Siwach
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Mich Talebzadeh
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Nitin Siwach
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Mich Talebzadeh
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Nitin Siwach
  • Re: Does spark read the same file twice, if two stages are using the same DataFrame? Mich Talebzadeh
  • Does spark read the same file twice, if two stages are using the same DataFrame? Vijay B
• Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count? Yong Zhang
  • Re: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count? Mich Talebzadeh
  • Re: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count? Yong Zhang
  • Re: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count? Yong Zhang
  • RE: Can Spark SQL (not DataFrame or Dataset) aggregate array into map of element of count? Vijay B
• Write DataFrame with Partition and choose Filename in PySpark Marco Costantini
  • Re: Write DataFrame with Partition and choose Filename in PySpark Mich Talebzadeh
  • Re: Write DataFrame with Partition and choose Filename in PySpark Marco Costantini
  • Re: Write DataFrame with Partition and choose Filename in PySpark Mich Talebzadeh
  • Re: Write DataFrame with Partition and choose Filename in PySpark Marco Costantini
  • Re: Write DataFrame with Partition and choose Filename in PySpark Mich Talebzadeh
• Write custom JSON from DataFrame in PySpark Marco Costantini
  • Re: Write custom JSON from DataFrame in PySpark Enrico Minack
  • Re: Write custom JSON from DataFrame in PySpark Marco Costantini
• CVE-2023-32007: Apache Spark: Shell command injection via Spark UI Arnout Engelen
• Change column values using several when conditions marc nicole
  • Re: Change column values using several when conditions Bjørn Jørgensen
• How to change column values using several when conditions ? marc nicole
• Any experience with K8s Remote Shuffling Service at scale? Andrey Gourine
• How to read text files with GBK encoding in the spark core lianyou1...@126.com
• Tensorflow on Spark CPU second_co...@yahoo.com.INVALID
  • Re: Tensorflow on Spark CPU Sean Owen
  • Re: Tensorflow on Spark CPU second_co...@yahoo.com.INVALID
  • Re: Tensorflow on Spark CPU Sean Owen
• driver and executors shared same Kubernetes PVC second_co...@yahoo.com.INVALID
• ***pyspark.sql.functions.monotonically_increasing_id()*** Karthick Nk
  • Re: ***pyspark.sql.functions.monotonically_increasing_id()*** Winston Lai
• config: minOffsetsPerTrigger not working Abhishek Singla
  • Re: config: minOffsetsPerTrigger not working Mich Talebzadeh
  • Re: config: minOffsetsPerTrigger not working Abhishek Singla
• What is the best way to organize a join within a foreach? Marco Costantini
  • What is the best way to organize a join within a foreach? Marco Costantini
  • Re: What is the best way to organize a join within a foreach? Mich Talebzadeh
  • Re: What is the best way to organize a join within a foreach? Marco Costantini
  • Re: What is the best way to organize a join within a foreach? Mich Talebzadeh
  • Re: What is the best way to organize a join within a foreach? Marco Costantini
  • Re: What is the best way to organize a join within a foreach? Mich Talebzadeh
  • Re: What is the best way to organize a join within a foreach? Marco Costantini
  • Re: What is the best way to organize a join within a foreach? Mich Talebzadeh
  • Re: What is the best way to organize a join within a foreach? ayan guha
  • Re: What is the best way to organize a join within a foreach? Mich Talebzadeh
  • Re: What is the best way to organize a join within a foreach? Marco Costantini
  • Re: What is the best way to organize a join within a foreach? Mich Talebzadeh
  • Re: What is the best way to organize a join within a foreach? Amit Joshi
• Reg: create spark using virtual machine through chef sunkara akhil sai teja
  • Unsubcribing phiroc
• Use Spark Aggregator in PySpark Thomas Wang
  • Re: Use Spark Aggregator in PySpark Enrico Minack
• Spark Aggregator with ARRAY<BOOLEAN> input and ARRAY<LONG> output Thomas Wang
  • Re: Spark Aggregator with ARRAY<BOOLEAN> input and ARRAY<LONG> output Raghavendra Ganesh
  • Re: Spark Aggregator with ARRAY<BOOLEAN> input and ARRAY<LONG> output Thomas Wang
• State of GraphX and GraphFrames g
• Dependency injection for spark executors Deepak Patankar
• Partition by on dataframe causing a Sort Nikhil Goyal
  • Re: Partition by on dataframe causing a Sort Nikhil Goyal
• [Spark on SBT] Executor just keeps running Dhruv Singla
  • Re: [Spark on SBT] Executor just keeps running Dhruv Singla
• Spark Log Shipper to Cloud Bucket Jayabindu Singh
• Re: Spark Multiple Hive Metastore Catalog Support Ankit Gupta
  • Re: Spark Multiple Hive Metastore Catalog Support Elliot West
  • Re: Spark Multiple Hive Metastore Catalog Support Cheng Pan
  • Re: Spark Multiple Hive Metastore Catalog Support Ankit Gupta
• CVE-2023-22946: Apache Spark proxy-user privilege escalation from malicious configuration class Sean R. Owen
• Scala commands syntax shortcuts(alias) Ankit Singla
• Spark Kubernetes Operator Yuval Itzchakov
  • Re: Spark Kubernetes Operator Mich Talebzadeh
  • Re: Spark Kubernetes Operator Yuval Itzchakov
  • RE: Spark Kubernetes Operator Aldo Culquicondor
• How to create spark udf use functioncatalog? ??????
  • Re: How to create spark udf use functioncatalog? Jacek Laskowski
  • How to create spark udf use functioncatalog? tzxxh
• [ANNOUNCE] Apache Spark 3.2.4 released Dongjoon Hyun
• _spark_metadata path issue with S3 lifecycle policy Yuval Itzchakov
  • Re: _spark_metadata path issue with S3 lifecycle policy Yuri Oleynikov (‫יורי אולייניקוב‬‎)
  • Re: _spark_metadata path issue with S3 lifecycle policy Yuval Itzchakov
  • Re: _spark_metadata path issue with S3 lifecycle policy Yuri Oleynikov (‫יורי אולייניקוב‬‎)
  • Re: _spark_metadata path issue with S3 lifecycle policy Yuval Itzchakov
• Accessing python runner file in AWS EKS kubernetes cluster as in local:// Mich Talebzadeh
  • Re: Accessing python runner file in AWS EKS kubernetes cluster as in local:// Bjørn Jørgensen
  • Re: Accessing python runner file in AWS EKS kubernetes cluster as in local:// Mich Talebzadeh
  • Re: Accessing python runner file in AWS EKS kubernetes cluster as in local:// Mich Talebzadeh
• Re: How to determine the function of tasks on each stage in an Apache Spark application? Jacek Laskowski
  • Re: How to determine the function of tasks on each stage in an Apache Spark application? Maytas Monsereenusorn
  • Re: How to determine the function of tasks on each stage in an Apache Spark application? Trường Trần Phan An
  • Re: How to determine the function of tasks on each stage in an Apache Spark application? Jacek Laskowski
  • Re: How to determine the function of tasks on each stage in an Apache Spark application? Trường Trần Phan An
• PySpark tests are failed with the java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.sources.FakeSourceOne not found Ranga Reddy
• Non string type partitions Charles vinodh
  • Re: Non string type partitions Chitral Verma
  • Re: Non string type partitions Charles vinodh
  • Re: Non string type partitions Charles vinodh
  • Re: Non string type partitions Bjørn Jørgensen
• [ANNOUNCE] Apache Uniffle(Incubating) 0.7.0 available Junfan Zhang
• [SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI Chenghao Lyu
  • Re: [SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI Chitral Verma
  • Re: [SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI Jacek Laskowski
• spark streaming and kinesis integration Rajesh Katkar
  • Re: spark streaming and kinesis integration Mich Talebzadeh
  • Re: spark streaming and kinesis integration Rajesh Katkar
  • Re: spark streaming and kinesis integration Mich Talebzadeh
  • RE: spark streaming and kinesis integration Jonske, Kurt
  • Re: spark streaming and kinesis integration Rajesh Katkar
  • Re: spark streaming and kinesis integration Mich Talebzadeh
  • Re: spark streaming and kinesis integration Mich Talebzadeh
  • Re: Re: spark streaming and kinesis integration Lingzhe Sun
  • Re: Re: spark streaming and kinesis integration Rajesh Katkar
  • Re: Re: spark streaming and kinesis integration Yi Huang
  • Re: Re: spark streaming and kinesis integration 孙令哲
  • Re: Re: spark streaming and kinesis integration Mich Talebzadeh
• Raise exception whilst casting instead of defaulting to null Yeachan Park
• Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application me
  • Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application me
  • Re: Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application Mich Talebzadeh
  • Re: Troubleshooting ArrayIndexOutOfBoundsException in long running Spark application Andrew Redd
• Potability of dockers built on different cloud platforms ashok34...@yahoo.com.INVALID
  • Re: Potability of dockers built on different cloud platforms Ken Peng
  • Re: Potability of dockers built on different cloud platforms Mich Talebzadeh
• Looping through a series of telephone numbers Philippe de Rochambeau
  • Re: Looping through a series of telephone numbers Mich Talebzadeh
  • Re: Looping through a series of telephone numbers Philippe de Rochambeau
  • Re: Looping through a series of telephone numbers Sean Owen
  • Re: Looping through a series of telephone numbers Gera Shegalov

• Earlier messages
• Later messages