Re: PySpark saveAsTextFile gzip

You can use the saveAsNewAPIHadoop file. You can use it for compressing your output, here's a sample code to use the AP

PySpark saveAsTextFile gzip

Hi, I've searched but can't seem to find a PySpark example. How do I write compressed text file output to S3 using PySpark saveAsTextFile? Thanks, Tom