[ https://issues.apache.org/jira/browse/SOLR-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645735#comment-17645735 ]
Haythem Khiri commented on SOLR-15857: -------------------------------------- This seems to be solved > Add Secret Manager support for ZK ACL credentials > ------------------------------------------------- > > Key: SOLR-15857 > URL: https://issues.apache.org/jira/browse/SOLR-15857 > Project: Solr > Issue Type: Improvement > Reporter: Lamine > Priority: Minor > Attachments: ZooKeeper_Access_Control.pdf > > Time Spent: 6.5h > Remaining Estimate: 0h > > _{*}Note{*}: Attached a copy of a ref guide describing this feature in more > details and with usage examples._ > *Problem* > Solr uses a list of credentials to connect to Zookeeper and to handle ACLs. > - 1- In the current implementation, the credentials are passed through > command line (system props) or read from a clear text file stored in all > cluster hosts. Needless to say this is not safe enough. > - 2- On the other hand, the same code to load the credentials is called > twice, first by _ZkCredentialsProvider_ to connect to Zookeeper and a second > time by _ZkACLProvider_ to create ACLs. The code is also duplicated, although > it's only reading from system props. > Adding a custom pair of {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} to > load the credentials from another source (ex a Secret Manager) would also > require duplicate the code and make repetitive calls to extract the same > credentials. > - 3- There is no way to customize how credentials are passed without > recompiling. > > *Proposed solution* > Let’s start with problem 2). > *Problem 2* > - Refactor the way how the credentials are injected by passing them as a > dependency. One code, called once and injected into the client class. Here > the client classes are _ZkCredentialsProvider_ and {_}ZkACLProvider{_}. > - Favor composition over inheritance to inject custom credentials loaders > without changing the composing (container) class. > - Add a third interface _ZkCredentialsInjector_ whose implementations load > ZK credentials from a credentials source to be injected into > _ZkCredentialsProvider_ and _ZkACLProvider_ > - The workflow is: Credentials source —> _ZkCredentialsInjector_ --> > {_}ZkCredentialsProvider{_}/{_}ZkACLProvider{_} --> Zookeeper > - The _ZkCredentialsInjector_ gets the creds from an external source which > get injected into zkCredentialsProvider and zkACLProvider. The "{_}external > source{_}" here can be system props, a file, a Secret Manager, or any other > local or remote source. > > {code:java} > public interface ZkCredentialsInjector{ > List<ZkCredential> getZkCredentials(); > ... > } > {code} > > > - Any class implementing _ZkCredentialsInjector_ can be injected via system > props in {_}solr.ini.sh/cmd{_}. > In the below example _VMParamsZkCredentialsInjector_ is injected. > Note: _VMParamsAllAndReadonlyDigestZkACLProvider_ and > _VMParamsSingleSetCredentialsDigestZkCredentialsProvider_ would be deprecated > and replaced with a combination of > {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} and > {_}VMParamsZkCredentialsInjector{_}. > > {code:java} > SOLR_ZK_CREDS_AND_ACLS=“ > -DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider \ > > -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider > \ > > -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.VMParamsZkCredentialsInjector > \ > -DzkDigestUsername=admin-user -DzkDigestPassword=CHANGEME-ADMIN-PASSWORD > \ > -DzkDigestReadonlyUsername=readonly-user > -DzkDigestReadonlyPassword=CHANGEME-READONLY-PASSWORD" > SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS"{code} > > - Add {_}DigestZkACLProvider{_}/{_}DigestZkCredentialsProvider{_} classes > to support _Digest_ based scheme ZK authentication/authorization > > {code:java} > Class DigestZkACLProvider implements ZkACLProvider { CredentialsInjector > credentialsInjector; > ... > } > Class DigestZkCredentialsProvider implements ZkCredentialsProvider{ > CredentialsInjector credentialsInjector; > ... > }{code} > > This concept can be generalized to non-digest schemes (a kind of Strategy > pattern) but that would require more refactoring, it can be achieved in a > future contribution if this one is accepted. > Now apply this new feature and add a custom injector to solve problem 1). > *Problem 1* > - Store the credentials in a Secret Manager to have Solr pull them out at > startup. > - Add _SecretCredentialInjector_ class that contains a dependency interface > ({_}SecretCredentialsProvider{_}) whose implementation pulls zk credentials > from a Secret Manager and delegate the _getZkCredentials_ call. > {code:java} > public class SecretCredentialInjector implements ZkCredentialsInjector { > ... > private SecretCredentialsProvider secretCredentialProvider; > > public List<ZkCredential> getZkCredentials() { > ... > return secretCredentialProvider.getZkCredentials(secretName); > } > ... > } > {code} > > - In this contribution the offered implementating class is > _AWSSecretCredentialsProvider_ that gets zk credentials from AWS Secret > Manager. Tu support any other Secret Manager provider all you need to do is > add a class implementing _SecretCredentialsProvider_ and pass it through > system props (-{_}DzkSecretCredentialsProvider{_}) > > > {code:java} > SOLR_ZK_CREDS_AND_ACLS="-DzkACLProvider=org.apache.solr.common.cloud.acl.DigestZkACLProvider > \ > > -DzkCredentialsProvider=org.apache.solr.common.cloud.acl.DigestZkCredentialsProvider > \ > > -DzkCredentialsInjector=org.apache.solr.common.cloud.acl.SecretCredentialInjector > \ > > -DzkSecretCredentialsProvider=org.apache.solr.secret.zk.AWSSecretCredentialsProvider > \ > -DzkSecretCredentialSecretName=zkSecret \ > -DzkCredentialsAWSSecretRegion=us-west-2" > SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS" > > {code} > > *Problem 3* > A new _contrib_ module ({_}secret-provider{_}) is added where > _SecretCredentialsProvider_ implementing classes can be added without the > need to add a new dependency to Solr core. All one needs to do after adding a > new class is to pass it through system props via _solr.ini.sh/cmd_ file. > This module can be used in the future for other secrets injections, not > specifically related to zk. > > Thank you in advance for your time and your comments. > > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org