Hi there, I am trying to configure memory for spark using AWS CLI. However, I got the following message:
*A client error (ValidationException) occurred when calling the RunJobFlow operation: Cannot specify args for application 'Spark' when release label is used.* In the aws 'create-cluster' command, I have '--release-label emr-4.0.0 --applications Name=Hadoop Name=Spark,Args=[-d,num-executors=4,spark.executor.memory=3000M,spark.driver.memory=4000M]' and it seems like I cannot specify args when there is '--release-label'. How do I get around this? I also tried using a JSON configuration file saved in a S3 bucket and add "--configurations http://path/bucket/config.json"; to the command but it gave me an 403 error (access denied). But when I did "aws s3 ls (s3://bucket)" I could see that bucket and the config.json in the bucket. Please advise. Thank you very much. Best Regards, Vivian
