Hi, It seems that you've set the "fs.hdfs.hadoopconf" configuration parameter to a file. I think you have to set it the directory containing the configuration. Sorry, I know that's not very intuitive, but in Hadoop the settings for in different files (hdfs|yarn|core)-site.xml.
On Sat, Nov 21, 2015 at 12:48 PM, Konstantin Knauf < konstantin.kn...@tngtech.com> wrote: > Hi Ufuk, > > sorry for not getting back to you for so long, and thanks for your > answer. The problem persists unfortunately. Running the job from the IDE > works (with core-site.xml on classpath), running it in local standalone > mode does not. AccessKeyID and SecretAccesKey are not found. > > Attached the jobmanager log on DEBUG level. The core-site.xml is > definitely at the configured location. > > I am now on version 0.10.0 and using the binaries for Hadoop 1.2.1 to > run the jar in local mode. Do I have to use the Hadoop 2.x version for > this to work? I have put hadoop-common-2.3.jar into the flink lib folder. > > I don't know if it is relevant (but it seems to be related), when I run > the job from my IDE I get the warning: > > 2015-11-21 12:43:11 WARN NativeCodeLoader:62 - Unable to load > native-hadoop library for your platform... using builtin-java classes > where applicable > > Cheers and thank you, > > Konstantin > > > On 14.10.2015 11:44, Ufuk Celebi wrote: > > > >> On 10 Oct 2015, at 22:59, snntr <konstantin.kn...@tngtech.com> wrote: > >> > >> Hey everyone, > >> > >> I was having the same problem with S3 and found this thread very useful. > >> Everything works fine now, when I start Flink from my IDE, but when I > run > >> the jar in local mode I keep getting > >> > >> java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access > Key > >> must be specified as the username or password (respectively) of a s3n > URL, > >> or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey > >> properties (respectively). > >> > >> I have set fs.hdfs.hadoopconf to point to a core-site.xml on my local > >> machine with the required properties. What am I missing? > >> > >> Any advice is highly appreciated ;) > > > > This looks like a problem with picking up the Hadoop config. Can you > look into the logs to check whether the configuration is picked up? Change > the log settings to DEBUG in log/log4j.properties for this. And can you > provide the complete stack trace? > > > > – Ufuk > > > > > > -- > Konstantin Knauf * konstantin.kn...@tngtech.com * +49-174-3413182 > TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke > Sitz: Unterföhring * Amtsgericht München * HRB 135082 >
