TLDR: I will buy your coffee if you can help me understand to securely
configure Azure credentials (doc
page<https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/azure/>
for reference).
I am a junior developer tasked with being the first person to learn the Apache
Flink framework. I know that storing secrets in flink-conf.yaml in a container
is a bad idea. I’ve tried exposing Azure storage keys as env vars and using
`config.setString`, but those properties seem to get overridden. I plan on
using Flink operator, so if you can show me in that context that’d be ideal.
Thanks, and sorry for bothering everyone. I’ve just exhausted myself and am
hopeful someone will have mercy for me. I really will Venmo you $5 for coffee
if you want.
Thanks,
Ivan
Larger code examples:
Setting dynamic properties before executing the job doesn’t work because the
values seem to get overridden or never forwarded.
```
val config = new Configuration()
config.setString("fs.azure.account.key.mystore1.blob.core.windows.net",
System.getenv("KEY_1"))
config.setString("fs.azure.account.key.mystore2.blob.core.windows.net",
System.getenv("KEY_2"))
config.setString("fs.azure.account.key.mystore3.blob.core.windows.net",
System.getenv("KEY_3"))
val env =
environment.StreamExecutionEnvironment.getExecutionEnvironment(config)
```
In Flink operator configuration fields can be provided as follows, but then I
can’t commit the file with a secret inside. Ideally there would be a way to
reference a secret but the values must be literal strings.
```
spec:
flinkConfiguration:
fs.azure.account.key.mystore1.blob.core.windows.net: SECRET_STRING
fs.azure.account.key.mystore2.blob.core.windows.net: SECRET_STRING
fs.azure.account.key.mystore3.blob.core.windows.net: SECRET_STRING
```
The last possible solution I can think that I’ll be trying is putting the
entire flink-conf.yaml into a secret, or having a different container that adds
secrets to the flink-operator-job.yaml and then does the `kubectl create -f
flink-operator-job.yaml` (if that’s even possible).
