Have you looked at http://spark.apache.org/docs/latest/ec2-scripts.html ?
There is description on setting AWS_SECRET_ACCESS_KEY. On Sun, Jun 19, 2016 at 4:46 AM, Mohamed Taher AlRefaie <m.ta...@msn.com> wrote: > Hello all: > > I have an application that requires accessing DynamoDB tables. Each worker > establishes a connection with the database on its own. > > I have added both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to both > master's and workers `spark-env.sh` file. I have also run the file using > `sh` to make sure the variables are exported. > > When the code runs, I always get the error: > > Caused by: com.amazonaws.AmazonClientException: Unable to load AWS > credentials from any provider in the chain > at > com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:131) > at > com.amazonaws.http.AmazonHttpClient.getCredentialsFromContext(AmazonHttpClient.java:774) > at > com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:800) > at > com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:695) > at > com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:447) > at > com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:409) > at > com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:358) > at > com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:2051) > at > com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:2021) > at > com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.describeTable(AmazonDynamoDBClient.java:1299) > at > com.amazon.titan.diskstorage.dynamodb.DynamoDBDelegate.describeTable(DynamoDBDelegate.java:635) > ... 27 more > > It seems that the AWS SDK has failed to load the credentials even though > they're exported. When I do export command, it returns the credentials. > What type of solution should I try? > > Thanks, > Mohamed Taher Alrefaie. >
