Unsubscribe
DF.write.format("csv")
.option("timestampFormat", "/MM/dd HH:mm:ss ZZ")
.mode(SaveMode.Overwrite)
.save("ftp://ftp:ftp@127.0.0.1:21/sparkftp/write/";)
I get a error
21/08/10 08:32:04 WARN FileOutputCommitter: Could not delete
ftp://ftpuser:ftpuser@10.3.87.51:21/sparkftp/
Hi,
I have a basic question to ask.
I am running a Google k8s cluster (AKA GKE) with three nodes each having
configuration below
e2-standard-2 (2 vCPUs, 8 GB memory)
spark-submit is launched from another node (actually a data proc single
node that I have just upgraded to e2-custom (4 vCPUs, 8
unsubscribe
Regards,
Vijay Gharge
The doc says that the minimum supported pandas version is 0.23.2 which is
only supported in python2.
IIRC, python2 is not supported in pyspark a long time ago. Can any one
confirm whether the doc is wrong and what is the right version of pandas
and pyarrow ?
https://spark.apache.org/docs/latest/ap
We have solved it using an orchestrator, which copied data from FTP to
HDFS. But of course, you can just use a Java FTP client to just read files,
put them somewhere, and then read with Spark.
пн, 9 авг. 2021 г. в 06:39, Sean Owen :
> FTP is definitely not supported. Read the files to distributed
