Hi Fred, Thanks for opening the discussion. There are probably multiple ways to solve it, you could also think about addressing this problem in a catalogue and hide any password/keywords in there. But then again, I do think your solution could also work and is straightforward.
If there are no people who have a better alternative or think otherwise, I think it should be fine to open a Jira ticket and if you're willing to make a contribution, that would be great! Best regards, Martijn Visser https://twitter.com/MartijnVisser82 https://github.com/MartijnVisser On Wed, 30 Mar 2022 at 10:25, Teunissen, F.G.J. (Fred) <fred.teunis...@ing.com.invalid> wrote: > Hi devs, > > Some SQL Table properties contain sensitive data, like passwords that we > do not want to expose in the VVP ui to other users. Also, having them clear > text in a SQL statement is not secure. For example, > > CREATE TABLE Orders ( > `user` BIGINT, > product STRING, > order_time TIMESTAMP(3) > ) WITH ( > 'connector' = 'kafka', > > 'properties.bootstrap.servers' = 'kafka-host-1:9093,kafka-host-2:9093', > 'properties.security.protocol' = 'SSL', > 'properties.ssl.key.password' = 'should-be-a-secret', > 'properties.ssl.keystore.location' = '/tmp/secrets/my-keystore.jks', > 'properties.ssl.keystore.password' = 'should-also-be-a-secret', > 'properties.ssl.truststore.location' = > '/tmp/secrets/my-truststore.jks', > 'properties.ssl.truststore.password' = 'should-again-be-a-secret', > 'scan.startup.mode' = 'earliest-offset' > ); > > I would like to bring up for a discussion a proposal to provide these > secrets values via environment variables since these can be populated from > a K8s configMap or secrets. > > For implementing the SQL Table properties, the ConfigOption<T> class is > used in connectors and formatters. This class could be extended that it > checks whether the config-value contains certain tokens, like > ‘${env-var-name}’. If it does, it could fetch the value from the > environment variable and use that to replace that token in the config-value. > > The above SQL statement would then look like, > > CREATE TABLE Orders ( > `user` BIGINT, > product STRING, > order_time TIMESTAMP(3) > ) WITH ( > 'connector' = 'kafka', > > 'properties.bootstrap.servers' = 'kafka-host-1:9093,kafka-host-2:9093', > 'properties.security.protocol' = 'SSL', > 'properties.ssl.key.password' = '${secret_kafka_ssl_key_password}', > 'properties.ssl.keystore.location' = '/tmp/secrets/my-keystore.jks', > 'properties.ssl.keystore.password' = > '${secret_kafka_ssl_keystore_password}', > 'properties.ssl.truststore.location' = > '/tmp/secrets/my-truststore.jks', > 'properties.ssl.truststore.password' = > '${secret_kafka_ssl_truststore_password}', > 'scan.startup.mode' = 'earliest-offset' > ); > > For the purpose of secrets I don’t think you need any complex processing > of tokens but perhaps there are other usages as well. For instance, > > 'properties.bootstrap.servers' = > 'kafka-${otap_env}-1:9093,kafka-${otap_env}-2:9093', > > Because it is possible that (but I think unlikely) someone wants a > property value like ‘${not-an-env-var}’ you need to be able to escape this > ’$’ token like ‘$${not-an-env-var}’. This also means that in theory it > would break compatibility. > > Looking forward for your feedback! > > Best, > Fred Teunissen > > ----------------------------------------------------------------- > ATTENTION: > The information in this e-mail is confidential and only meant for the > intended recipient. If you are not the intended recipient, don't use or > disclose it in any way. Please let the sender know and delete the message > immediately. > ----------------------------------------------------------------- >
