Hi Alexey,
Thanks we will give it a try.
From: Alexey Romanenko
Reply-To: "user@beam.apache.org"
Date: Thursday, June 10, 2021 at 5:14 AM
To: "user@beam.apache.org"
Subject: Re: How to specify a spark config with Beam spark runner
Hi Tao,
"Limited spark options”
Hi Tao,
"Limited spark options”, that you mentioned, are Beam's application arguments
and if you run your job via "spark-submit" you should still be able to
configure Spark application via normal spark-submit “--conf key=value” CLI
option.
Doesn’t it work for you?
—
Alexey
> On 10 Jun 2021,
Hi Beam community,
We are trying to specify a spark config
“spark.hadoop.fs.s3a.canned.acl=BucketOwnerFullControl” in the spark-submit
command for a beam app. I only see limited spark options supported according to
this doc: https://beam.apache.org/documentation/runners/spark/
How can we speci
