Re: Cross Region Apache Spark Setup

2020-04-23 Thread Stone Zhong
t. > > Just my 2 cents > > --- > Cheers, > -z > > > From: Stone Zhong > Sent: Wednesday, April 15, 2020 4:31 > To: user@spark.apache.org > Subject: Cross Region Apache Spark Setup > > Hi, > > I am trying to setup a cross regi

Re: Cross Region Apache Spark Setup

2020-04-20 Thread ZHANG Wei
--- Cheers, -z From: Stone Zhong Sent: Wednesday, April 15, 2020 4:31 To: user@spark.apache.org Subject: Cross Region Apache Spark Setup Hi, I am trying to setup a cross region Apache Spark cluster. All my data are stored in Amazon S3 and well partitioned

Cross Region Apache Spark Setup

2020-04-14 Thread Stone Zhong
Hi, I am trying to setup a cross region Apache Spark cluster. All my data are stored in Amazon S3 and well partitioned by region. For example, I have parquet file at S3://mybucket/sales_fact.parquet/us-west S3://mybucket/sales_fact.parquet/us-east S3://mybucket/sales_fact.parquet/uk