Hey Chesnay,
Thanks for the thorough answer, much appreciated.
Sorry for the "requesting []...", it was a loss in translation, that passed
my second reading check, the correct verb should have been "asking" :). It
was no request to the community at all, sorry again for that.
The solution to imple
> if we want to use two sets of credentials, for example to access two
different AWS buckets, that would not be feasible at the moment?
That is correct.
> As it seems that this limitation is quite an important one, is there
a place where we can find this documented?
I don't think it is expli
Hello Chesnay,
Thanks for the reply.
I wonder something based on your reply, if we want to use two sets of
credentials, for example to access two different AWS buckets, that would
not be feasible at the moment?
One example I have in mind would be to separate the credentials for
accessing data vs
The FileSystem class is essentially one big singleton, with only 1
instance of each FileSystem implementation being loaded, shared across
all jobs.
For that reason we do not support job-specific FileSystem configurations.
Note that we generally also don't really support configuring the
FileSyst
