gerlowskija commented on code in PR #5306: URL: https://github.com/apache/solr/pull/5306#discussion_r4527710256
########## solr/bin/solr: ########## @@ -636,6 +637,15 @@ fi # Everything below here is to support start, stop and restart. +if [[ "$SCRIPT_CMD" == "start" ]]; then + for example_arg in "$@"; do + if [[ "$example_arg" == "-e" || "$example_arg" == "--example" ]]; then Review Comment: [-0] Rather than iterating through all the arguments to detect if we're running an example or not, why don't we instead put just the `export ...` line [right here](https://github.com/apache/solr/blob/main/solr/bin/solr#L918) where we've already done that "am I running an example" check? ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { + solrHomeDir = exampleDir.resolve(exampleType); if (!Files.isDirectory(solrHomeDir)) Files.createDirectory(solrHomeDir); } else { solrHomeDir = serverDir.resolve("solr"); } } + if ("secure".equals(exampleType)) Files.createDirectories(solrHomeDir); if (!Files.isDirectory(solrHomeDir)) throw new IllegalArgumentException( "Value of --solr-home option is invalid! " + solrHomeDir.toAbsolutePath() + " is not a directory!"); } + /** Runs the equivalent steps of the Ref Guide security tutorial */ + void runSecureExample(CloudExampleParams params, boolean userManaged) throws Exception { + StartSolrParams start = params.start(); + if (userManaged || params.zkHost() != null) { + throw new IllegalArgumentException( + "The secure example requires its own embedded ZooKeeper in SolrCloud mode."); + } + if (start.host() != null + || start.jvmOpts() != null + || !start.extraArgs().isEmpty() + || params.scriptInputs() != null + || System.getenv("SOLR_DATA_HOME") != null) { + throw new IllegalArgumentException( + "Unsupported secure example option. Use --solr-home, -p, --no-prompt, -m, or --force."); Review Comment: ```suggestion "Unsupported argument provided. 'secure' example currently only supports: --solr-home, -p, --no-prompt, -m, or --force."); ``` ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { Review Comment: [0] It's nothing inherent to your PR or that I expect you to fix, but all of these `if (certainExample)` conditionals could go away if we had a better OOP design here. Ideally that'd make it easier for us to gather new examples as well... ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { + solrHomeDir = exampleDir.resolve(exampleType); if (!Files.isDirectory(solrHomeDir)) Files.createDirectory(solrHomeDir); } else { solrHomeDir = serverDir.resolve("solr"); } } + if ("secure".equals(exampleType)) Files.createDirectories(solrHomeDir); if (!Files.isDirectory(solrHomeDir)) throw new IllegalArgumentException( "Value of --solr-home option is invalid! " + solrHomeDir.toAbsolutePath() + " is not a directory!"); } + /** Runs the equivalent steps of the Ref Guide security tutorial */ + void runSecureExample(CloudExampleParams params, boolean userManaged) throws Exception { + StartSolrParams start = params.start(); + if (userManaged || params.zkHost() != null) { + throw new IllegalArgumentException( + "The secure example requires its own embedded ZooKeeper in SolrCloud mode."); + } + if (start.host() != null + || start.jvmOpts() != null + || !start.extraArgs().isEmpty() + || params.scriptInputs() != null + || System.getenv("SOLR_DATA_HOME") != null) { + throw new IllegalArgumentException( + "Unsupported secure example option. Use --solr-home, -p, --no-prompt, -m, or --force."); + } + int port = params.basePort(); + Path home = solrHomeDir.toAbsolutePath().normalize(); + echo( + "Commands below use paths relative to " + + serverDir.getParent().toAbsolutePath().normalize()); + Path tls = home.resolve("tls"); + Path include = tls.resolve("solr.in.sh"); + String zkHost = "localhost:" + (port + 1000); + Map<String, String> environment = new HashMap<>(EnvironmentUtils.getProcEnvironment()); + environment.put("SOLR_INCLUDE", include.toString()); + environment.put("SOLR_ULIMIT_CHECKS", "false"); + environment.put("SOLR_PID_DIR", home.toString()); + environment.put("SOLR_HOME", tls.toString()); + environment.remove("SOLR_ZK_CREDS_AND_ACLS"); + environment.remove("ZK_HOST"); + echo("This example uses 'SolrRocks' for its keystore and demonstration accounts."); Review Comment: ```suggestion echo("This example uses 'SolrRocks' as the password for Solr basic-auth accounts and the keystore password."); ``` ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { + solrHomeDir = exampleDir.resolve(exampleType); if (!Files.isDirectory(solrHomeDir)) Files.createDirectory(solrHomeDir); } else { solrHomeDir = serverDir.resolve("solr"); } } + if ("secure".equals(exampleType)) Files.createDirectories(solrHomeDir); if (!Files.isDirectory(solrHomeDir)) throw new IllegalArgumentException( "Value of --solr-home option is invalid! " + solrHomeDir.toAbsolutePath() + " is not a directory!"); } + /** Runs the equivalent steps of the Ref Guide security tutorial */ + void runSecureExample(CloudExampleParams params, boolean userManaged) throws Exception { + StartSolrParams start = params.start(); + if (userManaged || params.zkHost() != null) { + throw new IllegalArgumentException( + "The secure example requires its own embedded ZooKeeper in SolrCloud mode."); + } + if (start.host() != null + || start.jvmOpts() != null + || !start.extraArgs().isEmpty() + || params.scriptInputs() != null + || System.getenv("SOLR_DATA_HOME") != null) { + throw new IllegalArgumentException( + "Unsupported secure example option. Use --solr-home, -p, --no-prompt, -m, or --force."); + } + int port = params.basePort(); + Path home = solrHomeDir.toAbsolutePath().normalize(); + echo( + "Commands below use paths relative to " + + serverDir.getParent().toAbsolutePath().normalize()); + Path tls = home.resolve("tls"); + Path include = tls.resolve("solr.in.sh"); + String zkHost = "localhost:" + (port + 1000); + Map<String, String> environment = new HashMap<>(EnvironmentUtils.getProcEnvironment()); + environment.put("SOLR_INCLUDE", include.toString()); + environment.put("SOLR_ULIMIT_CHECKS", "false"); + environment.put("SOLR_PID_DIR", home.toString()); + environment.put("SOLR_HOME", tls.toString()); + environment.remove("SOLR_ZK_CREDS_AND_ACLS"); + environment.remove("ZK_HOST"); + echo("This example uses 'SolrRocks' for its keystore and demonstration accounts."); + echo("ZooKeeper is protected by digest ACLs; its transport remains plaintext on loopback."); + if (!params.noPrompt()) { + prompt( + new Scanner(userInput, StandardCharsets.UTF_8), Review Comment: [0] `runCloudExample` already creates its own Scanner. Could we pull that Scanner-creation code into somewhere that'd be reusable? ########## solr/packaging/test/test_example.bats: ########## @@ -50,3 +50,17 @@ teardown() { run curl "http://localhost:${SOLR2_PORT}/solr/admin/info/properties"; assert_output --partial 'helloworld' } + +@test "examples suppress resource limit warnings while ordinary starts retain them" { Review Comment: [0] Why? Is it just about output prettiness? ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { + solrHomeDir = exampleDir.resolve(exampleType); if (!Files.isDirectory(solrHomeDir)) Files.createDirectory(solrHomeDir); } else { solrHomeDir = serverDir.resolve("solr"); } } + if ("secure".equals(exampleType)) Files.createDirectories(solrHomeDir); if (!Files.isDirectory(solrHomeDir)) throw new IllegalArgumentException( "Value of --solr-home option is invalid! " + solrHomeDir.toAbsolutePath() + " is not a directory!"); } + /** Runs the equivalent steps of the Ref Guide security tutorial */ + void runSecureExample(CloudExampleParams params, boolean userManaged) throws Exception { + StartSolrParams start = params.start(); + if (userManaged || params.zkHost() != null) { + throw new IllegalArgumentException( + "The secure example requires its own embedded ZooKeeper in SolrCloud mode."); + } + if (start.host() != null + || start.jvmOpts() != null + || !start.extraArgs().isEmpty() + || params.scriptInputs() != null + || System.getenv("SOLR_DATA_HOME") != null) { + throw new IllegalArgumentException( + "Unsupported secure example option. Use --solr-home, -p, --no-prompt, -m, or --force."); + } + int port = params.basePort(); + Path home = solrHomeDir.toAbsolutePath().normalize(); + echo( + "Commands below use paths relative to " + + serverDir.getParent().toAbsolutePath().normalize()); + Path tls = home.resolve("tls"); + Path include = tls.resolve("solr.in.sh"); + String zkHost = "localhost:" + (port + 1000); + Map<String, String> environment = new HashMap<>(EnvironmentUtils.getProcEnvironment()); + environment.put("SOLR_INCLUDE", include.toString()); + environment.put("SOLR_ULIMIT_CHECKS", "false"); + environment.put("SOLR_PID_DIR", home.toString()); + environment.put("SOLR_HOME", tls.toString()); + environment.remove("SOLR_ZK_CREDS_AND_ACLS"); + environment.remove("ZK_HOST"); + echo("This example uses 'SolrRocks' for its keystore and demonstration accounts."); + echo("ZooKeeper is protected by digest ACLs; its transport remains plaintext on loopback."); + if (!params.noPrompt()) { + prompt( + new Scanner(userInput, StandardCharsets.UTF_8), + "Press Enter to delete and recreate " + + secureExampleDisplayArgument(home.toString()) + + ", or Ctrl-C to cancel: ", + ""); + } + resetSecureExampleHome(home, environment); + Files.setPosixFilePermissions(home, PosixFilePermissions.fromString("rwx------")); + Files.createDirectories(tls); + Files.setPosixFilePermissions(tls, PosixFilePermissions.fromString("rwx------")); + for (String node : List.of("node1", "node2")) { + Path nodeHome = Files.createDirectories(home.resolve(node).resolve("solr")); + for (String file : List.of("solr.xml", "zoo.cfg")) { + Files.copy(serverDir.resolve("solr").resolve(file), nodeHome.resolve(file)); + } + } + writeSecureExampleFile( + include, + String.format( + Locale.ROOT, + """ + SOLR_ULIMIT_CHECKS=false + SOLR_HOST_BIND=127.0.0.1 + SOLR_HOST_ADVERTISE=127.0.0.1 + SOLR_PID_DIR=%s + SOLR_OPTS=%s + """, + secureExampleShellQuote(home.toString()), + secureExampleShellQuote("-Dsolr.security.allow.paths=" + home))); + boolean ready = false; + try { + echo("\n"); + echo("1. Start both nodes and configure the shared SolrCloud URL scheme to https."); + startSecureNodes(params, home, environment, zkHost); + for (int nodePort : List.of(port, port + 1)) { + waitForSecureExample( + environment, tls, null, "http://localhost:"; + nodePort + "/api/node/health"); + } + runSecureCommand( + environment, + script, + "cluster", + "--property", + "urlScheme", + "--value", + "https", + "-z", + zkHost); + stopSecureNodes(port, environment); + + echo("\n"); + echo("2. Generate a self-signed certificate and PKCS12 key and trust stores."); + String keytool = Path.of(EnvUtils.getProperty("java.home"), "bin", "keytool").toString(); + runSecureCommand( + environment, + keytool, + "-genkeypair", + "-alias", + "solr-ssl", + "-keyalg", + "RSA", + "-keysize", + "2048", + "-validity", + "365", + "-dname", + "CN=localhost", + "-ext", + "SAN=DNS:localhost,IP:127.0.0.1", + "-storetype", + "PKCS12", + "-keystore", + tls.resolve("solr-ssl.keystore.p12").toString(), + "-storepass", + "SolrRocks", + "-keypass", + "SolrRocks"); + runSecureCommand( + environment, + keytool, + "-exportcert", + "-rfc", + "-alias", + "solr-ssl", + "-keystore", + tls.resolve("solr-ssl.keystore.p12").toString(), + "-storetype", + "PKCS12", + "-storepass", + "SolrRocks", + "-file", + tls.resolve("cert.pem").toString()); + runSecureCommand( + environment, + keytool, + "-importcert", + "-noprompt", + "-alias", + "solr-ssl", + "-file", + tls.resolve("cert.pem").toString(), + "-storetype", + "PKCS12", + "-keystore", + tls.resolve("solr-ssl.truststore.p12").toString(), + "-storepass", + "SolrRocks"); + runSecureCommand( + environment, + "openssl", + "x509", + "-in", + tls.resolve("cert.pem").toString(), + "-noout", + "-subject", + "-dates"); + Files.writeString( + include, + String.format( + Locale.ROOT, + """ + SOLR_SSL_ENABLED=true + SOLR_SSL_KEY_STORE=%s + SOLR_SSL_KEY_STORE_TYPE=PKCS12 + SOLR_SSL_KEY_STORE_PASSWORD=SolrRocks + SOLR_SSL_TRUST_STORE=%s + SOLR_SSL_TRUST_STORE_TYPE=PKCS12 + SOLR_SSL_TRUST_STORE_PASSWORD=SolrRocks + SOLR_SSL_CLIENT_TRUST_STORE=${SOLR_SSL_CLIENT_TRUST_STORE:-%s} + SOLR_SSL_CLIENT_TRUST_STORE_TYPE=PKCS12 + SOLR_SSL_CLIENT_TRUST_STORE_PASSWORD=SolrRocks + SOLR_SSL_NEED_CLIENT_AUTH=false + SOLR_SSL_WANT_CLIENT_AUTH=false + SOLR_SSL_CHECK_PEER_NAME=true + """, + secureExampleShellQuote(tls.resolve("solr-ssl.keystore.p12").toString()), + secureExampleShellQuote(tls.resolve("solr-ssl.truststore.p12").toString()), + secureExampleShellQuote(tls.resolve("solr-ssl.truststore.p12").toString())), + StandardOpenOption.APPEND); + for (String file : List.of("solr-ssl.keystore.p12", "solr-ssl.truststore.p12")) { + Files.setPosixFilePermissions( + tls.resolve(file), PosixFilePermissions.fromString("rw-------")); + } + + echo("\n"); + echo("3. Restart both nodes with HTTPS and verify the certificate."); + startSecureNodes(params, home, environment, zkHost); + for (int nodePort : List.of(port, port + 1)) { + waitForSecureExample( + environment, + tls, + "admin:SolrRocks", + "https://localhost:"; + nodePort + "/api/node/health"); + } + echo("\n"); + echo("\n"); + echo( + "4. Enable Basic Authentication and configure administrator, indexing, and search accounts."); + runSecureCommand( + environment, + script, + "auth", + "enable", + "--type", + "basicAuth", + "--credentials", + "admin:SolrRocks", + "--block-unknown", + "true", + "-z", + zkHost); + waitForSecureExample( + environment, tls, "admin:SolrRocks", "https://localhost:"; + port + "/api/node/health"); + secureExampleRequest( + environment, + tls, + port, + "/api/cluster/security/authentication", + """ + { + "set-user": { + "index": "SolrRocks", + "search": "SolrRocks" + } + } + """); + secureExampleRequest( + environment, + tls, + port, + "/api/cluster/security/authorization", + """ + { + "update-permission": {"index": 9, "role": ["admin", "search"]}, Review Comment: [0] As much as I love multi-line Java strings - at a certain point we might just want to make this a resource file or something that gets loaded rather than a larger string literal. Also, is there a reason you're making this an API call as opposed to copying a security.json into ZK directly? Might be more concise to upload the security.json directly 🤷 ########## solr/packaging/build.gradle: ########## @@ -365,13 +365,23 @@ abstract class BatsTask extends Exec { @Optional String filterTags = null + // Whether testFiles was populated via the --tests CLI option, as opposed to being + // preconfigured by the build script (e.g. integrationTestsPicocli's curated list). + // Explicitly requesting tests by name means the nightly tag filter should not apply. + // TODO: once the commons-cli/picocli split goes away (integrationTestsPicocli and its + // hardcoded testFiles list are removed), this field is redundant with `!testFiles.empty` + // and can be deleted in favor of that simpler check. + @Input + boolean testsSelectedViaCli = false Review Comment: [Q] This rule makes sense, but I'm surprised to only see it in a single gradle file. Is it already elsewhere and we're just including it here now to fix a gap in parity? Or is there something special about `packaging` that it requires this? ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { + solrHomeDir = exampleDir.resolve(exampleType); if (!Files.isDirectory(solrHomeDir)) Files.createDirectory(solrHomeDir); } else { solrHomeDir = serverDir.resolve("solr"); } } + if ("secure".equals(exampleType)) Files.createDirectories(solrHomeDir); if (!Files.isDirectory(solrHomeDir)) throw new IllegalArgumentException( "Value of --solr-home option is invalid! " + solrHomeDir.toAbsolutePath() + " is not a directory!"); } + /** Runs the equivalent steps of the Ref Guide security tutorial */ + void runSecureExample(CloudExampleParams params, boolean userManaged) throws Exception { + StartSolrParams start = params.start(); + if (userManaged || params.zkHost() != null) { + throw new IllegalArgumentException( + "The secure example requires its own embedded ZooKeeper in SolrCloud mode."); + } + if (start.host() != null + || start.jvmOpts() != null + || !start.extraArgs().isEmpty() + || params.scriptInputs() != null + || System.getenv("SOLR_DATA_HOME") != null) { + throw new IllegalArgumentException( + "Unsupported secure example option. Use --solr-home, -p, --no-prompt, -m, or --force."); + } + int port = params.basePort(); + Path home = solrHomeDir.toAbsolutePath().normalize(); + echo( + "Commands below use paths relative to " + + serverDir.getParent().toAbsolutePath().normalize()); + Path tls = home.resolve("tls"); + Path include = tls.resolve("solr.in.sh"); + String zkHost = "localhost:" + (port + 1000); + Map<String, String> environment = new HashMap<>(EnvironmentUtils.getProcEnvironment()); + environment.put("SOLR_INCLUDE", include.toString()); + environment.put("SOLR_ULIMIT_CHECKS", "false"); + environment.put("SOLR_PID_DIR", home.toString()); + environment.put("SOLR_HOME", tls.toString()); + environment.remove("SOLR_ZK_CREDS_AND_ACLS"); + environment.remove("ZK_HOST"); + echo("This example uses 'SolrRocks' for its keystore and demonstration accounts."); + echo("ZooKeeper is protected by digest ACLs; its transport remains plaintext on loopback."); + if (!params.noPrompt()) { + prompt( + new Scanner(userInput, StandardCharsets.UTF_8), + "Press Enter to delete and recreate " + + secureExampleDisplayArgument(home.toString()) + + ", or Ctrl-C to cancel: ", + ""); + } + resetSecureExampleHome(home, environment); + Files.setPosixFilePermissions(home, PosixFilePermissions.fromString("rwx------")); + Files.createDirectories(tls); + Files.setPosixFilePermissions(tls, PosixFilePermissions.fromString("rwx------")); + for (String node : List.of("node1", "node2")) { + Path nodeHome = Files.createDirectories(home.resolve(node).resolve("solr")); + for (String file : List.of("solr.xml", "zoo.cfg")) { + Files.copy(serverDir.resolve("solr").resolve(file), nodeHome.resolve(file)); + } + } + writeSecureExampleFile( + include, + String.format( + Locale.ROOT, + """ + SOLR_ULIMIT_CHECKS=false + SOLR_HOST_BIND=127.0.0.1 + SOLR_HOST_ADVERTISE=127.0.0.1 + SOLR_PID_DIR=%s + SOLR_OPTS=%s + """, + secureExampleShellQuote(home.toString()), + secureExampleShellQuote("-Dsolr.security.allow.paths=" + home))); + boolean ready = false; + try { + echo("\n"); + echo("1. Start both nodes and configure the shared SolrCloud URL scheme to https."); Review Comment: [Q] Most of our examples stand up Solr(s) configured in a certain way. And a few print out the specific commands they're running so that users have a sense what's going on. But AFAIK none of our examples today actually print out a tutorial/walkthrough as they're going. Can you explain your thinking there a bit please? I'm not opposed to it necessarily but it feels a little redundant with the really good tutorial docs you've also included in this PR. ########## solr/solr-ref-guide/modules/getting-started/pages/tutorial-securing-solr.adoc: ########## @@ -0,0 +1,511 @@ += Exercise 7: Securing Solr +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +This exercise starts a two node Solr cluster, enables HTTPS, assigns separate administration, indexing, and search accounts, and protects ZooKeeper data with access control lists (ACLs). +You will verify HTTPS for both client requests and requests between Solr nodes. +Each step uses the tools included in the Solr distribution, JDK `keytool`, OpenSSL, and `curl`. + +Use a fresh full Solr installation on Unix or macOS, with ports 8983, 8984, and 9983 available. +Run the commands in the same Bash-compatible shell, from the installation directory, using an installation path without spaces. +Do not run these commands against an existing cluster. + +IMPORTANT: This is a local learning exercise with demonstration passwords and a shared self-signed certificate. +Keep the cluster on loopback. +ZooKeeper authentication and ACLs do not encrypt its connection; this exercise leaves ZooKeeper transport on loopback without TLS. +For deployment guidance, see xref:deployment-guide:securing-solr.adoc[], xref:deployment-guide:enabling-ssl.adoc[], and xref:deployment-guide:zookeeper-access-control.adoc[]. + +== Start Two Nodes + +Prepare two isolated Solr homes by copying the distribution's configuration files. +Then start the first node with the ordinary `bin/solr start` command: + +[source,bash] +---- +umask 077 # Makes new files and directories accessible only to your user +export TUTORIAL_HOME="$PWD/tutorial/secure" +mkdir -p "$TUTORIAL_HOME/node1/solr" "$TUTORIAL_HOME/node2/solr" "$TUTORIAL_HOME/tls" +for node in node1 node2; do + cp server/solr/solr.xml "$TUTORIAL_HOME/$node/solr/" + cp server/solr/zoo.cfg "$TUTORIAL_HOME/$node/solr/" +done +cat > "$TUTORIAL_HOME/tls/solr.in.sh" <<EOF +SOLR_ULIMIT_CHECKS=false +SOLR_HOST_BIND=127.0.0.1 +SOLR_HOST_ADVERTISE=127.0.0.1 +SOLR_PID_DIR="$TUTORIAL_HOME" +SOLR_OPTS="-Dsolr.security.allow.paths=$TUTORIAL_HOME" +EOF +export SOLR_INCLUDE="$TUTORIAL_HOME/tls/solr.in.sh" +SOLR_LOGS_DIR="$TUTORIAL_HOME/node1/logs" \ + bin/solr start --solr-home "$TUTORIAL_HOME/node1/solr" +---- + +`SOLR_ULIMIT_CHECKS=false` suppresses resource-limit warnings for this local exercise. + +`SOLR_LOGS_DIR` gives each node its own log directory. + +The first SolrCloud node listens on `localhost:8983` and starts embedded ZooKeeper on `localhost:9983`. +The embedded ZooKeeper is suitable for this exercise; production clusters should use an external ensemble. + +Check that it responds over HTTP, then start the second node: + +[source,bash] +---- +curl --fail 'http://localhost:8983/api/node/health' +SOLR_LOGS_DIR="$TUTORIAL_HOME/node2/logs" \ + bin/solr start -p 8984 --solr-home "$TUTORIAL_HOME/node2/solr" -z localhost:9983 +curl --fail 'http://localhost:8984/api/node/health' +curl --fail --silent --show-error \ + 'http://localhost:8983/api/cluster/nodes' +---- + +Wait for each health request to succeed before proceeding; repeat the corresponding `curl` command if the node is still initializing. +A listening TCP port alone does not mean Solr has finished starting. + +Both nodes use ZooKeeper's root namespace. +The first node runs embedded ZooKeeper; the second joins it with `-z localhost:9983`. +Each home starts with only configuration files and stores its own data. + +The response's `nodes` array must contain two nodes, with ports 8983 and 8984. +If only one node appears, repeat the `curl` command after the second node finishes initializing; continue when both are present. +Both nodes share the first node's ZooKeeper, rather than forming separate clusters. +The working directory holds both nodes' data, certificates, and a private configuration file. +Node logs are under `tutorial/secure/node1/logs` and `tutorial/secure/node2/logs`. + +== Create a Certificate and PKCS12 Stores + +Generate a private key and self-signed certificate directly in a PKCS12 keystore. +The subject alternative names cover both names used in this exercise: Review Comment: [0] "The subject alternative names" I'm having trouble understanding what this means. I'd guess it's referring to the `-subject` flag given to the final openssl command below in some way, but I don't know what "alternative names" means since that flag isn't taking an argument. Maybe some wording cleanup would be worthwhile here to make this clearer. ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { + solrHomeDir = exampleDir.resolve(exampleType); if (!Files.isDirectory(solrHomeDir)) Files.createDirectory(solrHomeDir); } else { solrHomeDir = serverDir.resolve("solr"); } } + if ("secure".equals(exampleType)) Files.createDirectories(solrHomeDir); if (!Files.isDirectory(solrHomeDir)) throw new IllegalArgumentException( "Value of --solr-home option is invalid! " + solrHomeDir.toAbsolutePath() + " is not a directory!"); } + /** Runs the equivalent steps of the Ref Guide security tutorial */ + void runSecureExample(CloudExampleParams params, boolean userManaged) throws Exception { + StartSolrParams start = params.start(); + if (userManaged || params.zkHost() != null) { + throw new IllegalArgumentException( + "The secure example requires its own embedded ZooKeeper in SolrCloud mode."); + } + if (start.host() != null + || start.jvmOpts() != null + || !start.extraArgs().isEmpty() + || params.scriptInputs() != null + || System.getenv("SOLR_DATA_HOME") != null) { + throw new IllegalArgumentException( + "Unsupported secure example option. Use --solr-home, -p, --no-prompt, -m, or --force."); + } + int port = params.basePort(); + Path home = solrHomeDir.toAbsolutePath().normalize(); + echo( + "Commands below use paths relative to " + + serverDir.getParent().toAbsolutePath().normalize()); + Path tls = home.resolve("tls"); + Path include = tls.resolve("solr.in.sh"); + String zkHost = "localhost:" + (port + 1000); + Map<String, String> environment = new HashMap<>(EnvironmentUtils.getProcEnvironment()); + environment.put("SOLR_INCLUDE", include.toString()); + environment.put("SOLR_ULIMIT_CHECKS", "false"); + environment.put("SOLR_PID_DIR", home.toString()); + environment.put("SOLR_HOME", tls.toString()); + environment.remove("SOLR_ZK_CREDS_AND_ACLS"); + environment.remove("ZK_HOST"); + echo("This example uses 'SolrRocks' for its keystore and demonstration accounts."); + echo("ZooKeeper is protected by digest ACLs; its transport remains plaintext on loopback."); + if (!params.noPrompt()) { + prompt( + new Scanner(userInput, StandardCharsets.UTF_8), + "Press Enter to delete and recreate " + + secureExampleDisplayArgument(home.toString()) + + ", or Ctrl-C to cancel: ", + ""); + } + resetSecureExampleHome(home, environment); + Files.setPosixFilePermissions(home, PosixFilePermissions.fromString("rwx------")); + Files.createDirectories(tls); + Files.setPosixFilePermissions(tls, PosixFilePermissions.fromString("rwx------")); + for (String node : List.of("node1", "node2")) { + Path nodeHome = Files.createDirectories(home.resolve(node).resolve("solr")); + for (String file : List.of("solr.xml", "zoo.cfg")) { + Files.copy(serverDir.resolve("solr").resolve(file), nodeHome.resolve(file)); + } + } + writeSecureExampleFile( + include, + String.format( + Locale.ROOT, + """ + SOLR_ULIMIT_CHECKS=false + SOLR_HOST_BIND=127.0.0.1 + SOLR_HOST_ADVERTISE=127.0.0.1 + SOLR_PID_DIR=%s + SOLR_OPTS=%s + """, + secureExampleShellQuote(home.toString()), + secureExampleShellQuote("-Dsolr.security.allow.paths=" + home))); + boolean ready = false; + try { + echo("\n"); + echo("1. Start both nodes and configure the shared SolrCloud URL scheme to https."); + startSecureNodes(params, home, environment, zkHost); + for (int nodePort : List.of(port, port + 1)) { + waitForSecureExample( + environment, tls, null, "http://localhost:"; + nodePort + "/api/node/health"); + } + runSecureCommand( + environment, + script, + "cluster", + "--property", + "urlScheme", + "--value", + "https", + "-z", + zkHost); + stopSecureNodes(port, environment); + + echo("\n"); + echo("2. Generate a self-signed certificate and PKCS12 key and trust stores."); + String keytool = Path.of(EnvUtils.getProperty("java.home"), "bin", "keytool").toString(); + runSecureCommand( + environment, + keytool, + "-genkeypair", + "-alias", + "solr-ssl", + "-keyalg", + "RSA", + "-keysize", + "2048", + "-validity", + "365", + "-dname", + "CN=localhost", + "-ext", + "SAN=DNS:localhost,IP:127.0.0.1", + "-storetype", + "PKCS12", + "-keystore", + tls.resolve("solr-ssl.keystore.p12").toString(), + "-storepass", + "SolrRocks", + "-keypass", + "SolrRocks"); + runSecureCommand( + environment, + keytool, + "-exportcert", + "-rfc", + "-alias", + "solr-ssl", + "-keystore", + tls.resolve("solr-ssl.keystore.p12").toString(), + "-storetype", + "PKCS12", + "-storepass", + "SolrRocks", + "-file", + tls.resolve("cert.pem").toString()); + runSecureCommand( + environment, + keytool, + "-importcert", + "-noprompt", + "-alias", + "solr-ssl", + "-file", + tls.resolve("cert.pem").toString(), + "-storetype", + "PKCS12", + "-keystore", + tls.resolve("solr-ssl.truststore.p12").toString(), + "-storepass", + "SolrRocks"); + runSecureCommand( + environment, + "openssl", + "x509", + "-in", + tls.resolve("cert.pem").toString(), + "-noout", + "-subject", + "-dates"); + Files.writeString( + include, + String.format( + Locale.ROOT, + """ + SOLR_SSL_ENABLED=true + SOLR_SSL_KEY_STORE=%s + SOLR_SSL_KEY_STORE_TYPE=PKCS12 + SOLR_SSL_KEY_STORE_PASSWORD=SolrRocks + SOLR_SSL_TRUST_STORE=%s + SOLR_SSL_TRUST_STORE_TYPE=PKCS12 + SOLR_SSL_TRUST_STORE_PASSWORD=SolrRocks + SOLR_SSL_CLIENT_TRUST_STORE=${SOLR_SSL_CLIENT_TRUST_STORE:-%s} + SOLR_SSL_CLIENT_TRUST_STORE_TYPE=PKCS12 + SOLR_SSL_CLIENT_TRUST_STORE_PASSWORD=SolrRocks + SOLR_SSL_NEED_CLIENT_AUTH=false + SOLR_SSL_WANT_CLIENT_AUTH=false + SOLR_SSL_CHECK_PEER_NAME=true + """, + secureExampleShellQuote(tls.resolve("solr-ssl.keystore.p12").toString()), + secureExampleShellQuote(tls.resolve("solr-ssl.truststore.p12").toString()), + secureExampleShellQuote(tls.resolve("solr-ssl.truststore.p12").toString())), + StandardOpenOption.APPEND); + for (String file : List.of("solr-ssl.keystore.p12", "solr-ssl.truststore.p12")) { + Files.setPosixFilePermissions( + tls.resolve(file), PosixFilePermissions.fromString("rw-------")); + } + + echo("\n"); + echo("3. Restart both nodes with HTTPS and verify the certificate."); + startSecureNodes(params, home, environment, zkHost); + for (int nodePort : List.of(port, port + 1)) { + waitForSecureExample( + environment, + tls, + "admin:SolrRocks", + "https://localhost:"; + nodePort + "/api/node/health"); + } + echo("\n"); + echo("\n"); + echo( + "4. Enable Basic Authentication and configure administrator, indexing, and search accounts."); + runSecureCommand( + environment, + script, + "auth", + "enable", + "--type", + "basicAuth", + "--credentials", + "admin:SolrRocks", + "--block-unknown", + "true", + "-z", + zkHost); + waitForSecureExample( + environment, tls, "admin:SolrRocks", "https://localhost:"; + port + "/api/node/health"); + secureExampleRequest( + environment, + tls, + port, + "/api/cluster/security/authentication", + """ + { + "set-user": { + "index": "SolrRocks", + "search": "SolrRocks" + } + } + """); + secureExampleRequest( + environment, + tls, + port, + "/api/cluster/security/authorization", + """ + { + "update-permission": {"index": 9, "role": ["admin", "search"]}, + "set-user-role": { + "admin": ["superadmin", "admin"], + "index": ["index", "search"], + "search": ["search"] + } + } + """); + + echo("\n"); + echo("\n"); + echo( + "5. Configure ZooKeeper digest accounts and migrate existing znodes to restricted ACLs."); + Path credentials = tls.resolve("zk-credentials.properties"); + writeSecureExampleFile( + credentials, + """ + zkDigestUsername=zk-admin + zkDigestPassword=SolrRocks Review Comment: [-0] There's something jarring and disheartening about seeing a bunch of password hardcoding in our "secure" example. I guess both the fact that the password is the same on each and every run, and that the password appears as plaintext in this PR. Wdyt about generating a new password on each new run and printing it out for the user before writing it to these files? ########## solr/solr-ref-guide/modules/getting-started/pages/tutorial-securing-solr.adoc: ########## Review Comment: [0] This page has some duplication with our `enabling-ssl.adoc` page, which also shares some very similar commands for turning on SSL, generating certs, etc. I guess some duplication is inevitable when taking reference content and turning it into a more descriptive, readable README. But if you haven't yet it might be worth taking a look at the two pages to make sure there's not too much duplication... ########## solr/solr-ref-guide/modules/getting-started/pages/tutorial-securing-solr.adoc: ########## @@ -0,0 +1,511 @@ += Exercise 7: Securing Solr +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +This exercise starts a two node Solr cluster, enables HTTPS, assigns separate administration, indexing, and search accounts, and protects ZooKeeper data with access control lists (ACLs). +You will verify HTTPS for both client requests and requests between Solr nodes. +Each step uses the tools included in the Solr distribution, JDK `keytool`, OpenSSL, and `curl`. + +Use a fresh full Solr installation on Unix or macOS, with ports 8983, 8984, and 9983 available. +Run the commands in the same Bash-compatible shell, from the installation directory, using an installation path without spaces. +Do not run these commands against an existing cluster. + +IMPORTANT: This is a local learning exercise with demonstration passwords and a shared self-signed certificate. +Keep the cluster on loopback. +ZooKeeper authentication and ACLs do not encrypt its connection; this exercise leaves ZooKeeper transport on loopback without TLS. +For deployment guidance, see xref:deployment-guide:securing-solr.adoc[], xref:deployment-guide:enabling-ssl.adoc[], and xref:deployment-guide:zookeeper-access-control.adoc[]. + +== Start Two Nodes + +Prepare two isolated Solr homes by copying the distribution's configuration files. +Then start the first node with the ordinary `bin/solr start` command: + +[source,bash] +---- +umask 077 # Makes new files and directories accessible only to your user +export TUTORIAL_HOME="$PWD/tutorial/secure" +mkdir -p "$TUTORIAL_HOME/node1/solr" "$TUTORIAL_HOME/node2/solr" "$TUTORIAL_HOME/tls" +for node in node1 node2; do + cp server/solr/solr.xml "$TUTORIAL_HOME/$node/solr/" + cp server/solr/zoo.cfg "$TUTORIAL_HOME/$node/solr/" +done +cat > "$TUTORIAL_HOME/tls/solr.in.sh" <<EOF +SOLR_ULIMIT_CHECKS=false +SOLR_HOST_BIND=127.0.0.1 +SOLR_HOST_ADVERTISE=127.0.0.1 +SOLR_PID_DIR="$TUTORIAL_HOME" +SOLR_OPTS="-Dsolr.security.allow.paths=$TUTORIAL_HOME" +EOF +export SOLR_INCLUDE="$TUTORIAL_HOME/tls/solr.in.sh" +SOLR_LOGS_DIR="$TUTORIAL_HOME/node1/logs" \ + bin/solr start --solr-home "$TUTORIAL_HOME/node1/solr" +---- + +`SOLR_ULIMIT_CHECKS=false` suppresses resource-limit warnings for this local exercise. + +`SOLR_LOGS_DIR` gives each node its own log directory. + +The first SolrCloud node listens on `localhost:8983` and starts embedded ZooKeeper on `localhost:9983`. +The embedded ZooKeeper is suitable for this exercise; production clusters should use an external ensemble. + +Check that it responds over HTTP, then start the second node: + +[source,bash] +---- +curl --fail 'http://localhost:8983/api/node/health' +SOLR_LOGS_DIR="$TUTORIAL_HOME/node2/logs" \ + bin/solr start -p 8984 --solr-home "$TUTORIAL_HOME/node2/solr" -z localhost:9983 +curl --fail 'http://localhost:8984/api/node/health' +curl --fail --silent --show-error \ + 'http://localhost:8983/api/cluster/nodes' +---- + +Wait for each health request to succeed before proceeding; repeat the corresponding `curl` command if the node is still initializing. +A listening TCP port alone does not mean Solr has finished starting. + +Both nodes use ZooKeeper's root namespace. +The first node runs embedded ZooKeeper; the second joins it with `-z localhost:9983`. +Each home starts with only configuration files and stores its own data. + +The response's `nodes` array must contain two nodes, with ports 8983 and 8984. +If only one node appears, repeat the `curl` command after the second node finishes initializing; continue when both are present. +Both nodes share the first node's ZooKeeper, rather than forming separate clusters. +The working directory holds both nodes' data, certificates, and a private configuration file. +Node logs are under `tutorial/secure/node1/logs` and `tutorial/secure/node2/logs`. + +== Create a Certificate and PKCS12 Stores + +Generate a private key and self-signed certificate directly in a PKCS12 keystore. +The subject alternative names cover both names used in this exercise: + +[source,bash] +---- +keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 \ + -validity 365 -dname 'CN=localhost' \ + -ext 'SAN=DNS:localhost,IP:127.0.0.1' \ + -storetype PKCS12 -keystore "$TUTORIAL_HOME/tls/solr-ssl.keystore.p12" \ + -storepass SolrRocks -keypass SolrRocks +keytool -exportcert -alias solr-ssl -rfc \ + -keystore "$TUTORIAL_HOME/tls/solr-ssl.keystore.p12" -storetype PKCS12 \ + -storepass SolrRocks -file "$TUTORIAL_HOME/tls/cert.pem" +keytool -importcert -noprompt -alias solr-ssl \ + -file "$TUTORIAL_HOME/tls/cert.pem" \ + -keystore "$TUTORIAL_HOME/tls/solr-ssl.truststore.p12" -storetype PKCS12 \ + -storepass SolrRocks +openssl x509 -in "$TUTORIAL_HOME/tls/cert.pem" -noout -subject -dates -text +---- + +The keystore contains the private key and certificate presented by each server. +The truststore contains the certificate trusted by Solr's outbound HTTPS clients. +The PEM certificate lets `curl` verify the server without disabling certificate checks. +Using the same keystore on both nodes simplifies the exercise; real deployments should manage certificates and private keys separately for each node. + +== Enable HTTPS on Both Nodes + +Write an include file used only for this exercise. +The unquoted heredoc expands the absolute working-directory path now; escaped dollar signs preserve variables for later launches: + +[source,bash] +---- +cat > "$TUTORIAL_HOME/tls/solr.in.sh" <<EOF +SOLR_ULIMIT_CHECKS=false +SOLR_HOST_BIND=127.0.0.1 +SOLR_HOST_ADVERTISE=127.0.0.1 +SOLR_PID_DIR="$TUTORIAL_HOME" +SOLR_OPTS="-Dsolr.security.allow.paths=$TUTORIAL_HOME" +SOLR_SSL_ENABLED=true +SOLR_SSL_KEY_STORE="$TUTORIAL_HOME/tls/solr-ssl.keystore.p12" +SOLR_SSL_KEY_STORE_PASSWORD=SolrRocks +SOLR_SSL_KEY_STORE_TYPE=PKCS12 +SOLR_SSL_TRUST_STORE="$TUTORIAL_HOME/tls/solr-ssl.truststore.p12" +SOLR_SSL_TRUST_STORE_PASSWORD=SolrRocks +SOLR_SSL_TRUST_STORE_TYPE=PKCS12 +SOLR_SSL_CLIENT_TRUST_STORE="\${SOLR_SSL_CLIENT_TRUST_STORE:-$TUTORIAL_HOME/tls/solr-ssl.truststore.p12}" +SOLR_SSL_CLIENT_TRUST_STORE_PASSWORD=SolrRocks +SOLR_SSL_CLIENT_TRUST_STORE_TYPE=PKCS12 +SOLR_SSL_NEED_CLIENT_AUTH=false +SOLR_SSL_WANT_CLIENT_AUTH=false +SOLR_SSL_CHECK_PEER_NAME=true +EOF +chmod 600 "$TUTORIAL_HOME/tls/solr.in.sh" +---- + +The server's truststore and the outbound client's truststore are explicit. +Nodes advertise the IPv4 loopback address `127.0.0.1`; client URLs can also use `localhost`, because the certificate covers both. +Hostname verification remains enabled. +Client certificates are not required; later steps authenticate users with Basic Authentication over HTTPS. + +Before restarting, set the cluster's advertised URL scheme while ZooKeeper is still running. Review Comment: [0] "Before restarting" is the first mention that the we actually have to restart our nodes in order to pick up HTTPS. I imagine this'll be confusing and take some explaining for folks not already very familiar with Solr, who are likely to have the question: "Why didn't I just pick these settings before starting Solr the first time?" [Q] Also, do we really need the `bin/solr clutster --property urlScheme` bit below? Maybe our docs are incorrect, but I don't see it in the [Enabling SSL](https://solr.apache.org/guide/solr/latest/deployment-guide/enabling-ssl.html#starting-solr-after-enabling-ssl) mini-tutorial. Are we sure this is really needed? ########## solr/core/src/java/org/apache/solr/cli/RunExampleTool.java: ########## @@ -597,20 +628,501 @@ void initDirs( String solrHomeProp = EnvUtils.getProperty("solr.home"); if (solrHomeProp != null && !solrHomeProp.isEmpty()) { solrHomeDir = Path.of(solrHomeProp); - } else if ("cloud".equals(exampleType)) { - solrHomeDir = exampleDir.resolve("cloud"); + } else if ("cloud".equals(exampleType) || "secure".equals(exampleType)) { + solrHomeDir = exampleDir.resolve(exampleType); if (!Files.isDirectory(solrHomeDir)) Files.createDirectory(solrHomeDir); } else { solrHomeDir = serverDir.resolve("solr"); } } + if ("secure".equals(exampleType)) Files.createDirectories(solrHomeDir); if (!Files.isDirectory(solrHomeDir)) throw new IllegalArgumentException( "Value of --solr-home option is invalid! " + solrHomeDir.toAbsolutePath() + " is not a directory!"); } + /** Runs the equivalent steps of the Ref Guide security tutorial */ + void runSecureExample(CloudExampleParams params, boolean userManaged) throws Exception { + StartSolrParams start = params.start(); + if (userManaged || params.zkHost() != null) { + throw new IllegalArgumentException( + "The secure example requires its own embedded ZooKeeper in SolrCloud mode."); + } + if (start.host() != null + || start.jvmOpts() != null + || !start.extraArgs().isEmpty() + || params.scriptInputs() != null + || System.getenv("SOLR_DATA_HOME") != null) { + throw new IllegalArgumentException( + "Unsupported secure example option. Use --solr-home, -p, --no-prompt, -m, or --force."); + } + int port = params.basePort(); + Path home = solrHomeDir.toAbsolutePath().normalize(); + echo( + "Commands below use paths relative to " + + serverDir.getParent().toAbsolutePath().normalize()); + Path tls = home.resolve("tls"); + Path include = tls.resolve("solr.in.sh"); + String zkHost = "localhost:" + (port + 1000); + Map<String, String> environment = new HashMap<>(EnvironmentUtils.getProcEnvironment()); + environment.put("SOLR_INCLUDE", include.toString()); + environment.put("SOLR_ULIMIT_CHECKS", "false"); + environment.put("SOLR_PID_DIR", home.toString()); + environment.put("SOLR_HOME", tls.toString()); + environment.remove("SOLR_ZK_CREDS_AND_ACLS"); + environment.remove("ZK_HOST"); + echo("This example uses 'SolrRocks' for its keystore and demonstration accounts."); + echo("ZooKeeper is protected by digest ACLs; its transport remains plaintext on loopback."); + if (!params.noPrompt()) { + prompt( + new Scanner(userInput, StandardCharsets.UTF_8), + "Press Enter to delete and recreate " + + secureExampleDisplayArgument(home.toString()) + + ", or Ctrl-C to cancel: ", + ""); + } + resetSecureExampleHome(home, environment); + Files.setPosixFilePermissions(home, PosixFilePermissions.fromString("rwx------")); + Files.createDirectories(tls); + Files.setPosixFilePermissions(tls, PosixFilePermissions.fromString("rwx------")); + for (String node : List.of("node1", "node2")) { + Path nodeHome = Files.createDirectories(home.resolve(node).resolve("solr")); + for (String file : List.of("solr.xml", "zoo.cfg")) { + Files.copy(serverDir.resolve("solr").resolve(file), nodeHome.resolve(file)); + } + } + writeSecureExampleFile( + include, + String.format( + Locale.ROOT, + """ Review Comment: [+1] Yay Java multi-line Strings 😛 ########## solr/solr-ref-guide/modules/getting-started/pages/tutorial-securing-solr.adoc: ########## @@ -0,0 +1,511 @@ += Exercise 7: Securing Solr +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +This exercise starts a two node Solr cluster, enables HTTPS, assigns separate administration, indexing, and search accounts, and protects ZooKeeper data with access control lists (ACLs). +You will verify HTTPS for both client requests and requests between Solr nodes. +Each step uses the tools included in the Solr distribution, JDK `keytool`, OpenSSL, and `curl`. + +Use a fresh full Solr installation on Unix or macOS, with ports 8983, 8984, and 9983 available. +Run the commands in the same Bash-compatible shell, from the installation directory, using an installation path without spaces. +Do not run these commands against an existing cluster. + +IMPORTANT: This is a local learning exercise with demonstration passwords and a shared self-signed certificate. +Keep the cluster on loopback. +ZooKeeper authentication and ACLs do not encrypt its connection; this exercise leaves ZooKeeper transport on loopback without TLS. +For deployment guidance, see xref:deployment-guide:securing-solr.adoc[], xref:deployment-guide:enabling-ssl.adoc[], and xref:deployment-guide:zookeeper-access-control.adoc[]. + +== Start Two Nodes + +Prepare two isolated Solr homes by copying the distribution's configuration files. +Then start the first node with the ordinary `bin/solr start` command: + +[source,bash] +---- +umask 077 # Makes new files and directories accessible only to your user +export TUTORIAL_HOME="$PWD/tutorial/secure" +mkdir -p "$TUTORIAL_HOME/node1/solr" "$TUTORIAL_HOME/node2/solr" "$TUTORIAL_HOME/tls" +for node in node1 node2; do + cp server/solr/solr.xml "$TUTORIAL_HOME/$node/solr/" + cp server/solr/zoo.cfg "$TUTORIAL_HOME/$node/solr/" +done +cat > "$TUTORIAL_HOME/tls/solr.in.sh" <<EOF +SOLR_ULIMIT_CHECKS=false +SOLR_HOST_BIND=127.0.0.1 +SOLR_HOST_ADVERTISE=127.0.0.1 +SOLR_PID_DIR="$TUTORIAL_HOME" +SOLR_OPTS="-Dsolr.security.allow.paths=$TUTORIAL_HOME" +EOF +export SOLR_INCLUDE="$TUTORIAL_HOME/tls/solr.in.sh" +SOLR_LOGS_DIR="$TUTORIAL_HOME/node1/logs" \ + bin/solr start --solr-home "$TUTORIAL_HOME/node1/solr" +---- + +`SOLR_ULIMIT_CHECKS=false` suppresses resource-limit warnings for this local exercise. + +`SOLR_LOGS_DIR` gives each node its own log directory. + +The first SolrCloud node listens on `localhost:8983` and starts embedded ZooKeeper on `localhost:9983`. +The embedded ZooKeeper is suitable for this exercise; production clusters should use an external ensemble. + +Check that it responds over HTTP, then start the second node: + +[source,bash] +---- +curl --fail 'http://localhost:8983/api/node/health' +SOLR_LOGS_DIR="$TUTORIAL_HOME/node2/logs" \ + bin/solr start -p 8984 --solr-home "$TUTORIAL_HOME/node2/solr" -z localhost:9983 +curl --fail 'http://localhost:8984/api/node/health' +curl --fail --silent --show-error \ + 'http://localhost:8983/api/cluster/nodes' +---- + +Wait for each health request to succeed before proceeding; repeat the corresponding `curl` command if the node is still initializing. +A listening TCP port alone does not mean Solr has finished starting. + +Both nodes use ZooKeeper's root namespace. +The first node runs embedded ZooKeeper; the second joins it with `-z localhost:9983`. +Each home starts with only configuration files and stores its own data. + +The response's `nodes` array must contain two nodes, with ports 8983 and 8984. +If only one node appears, repeat the `curl` command after the second node finishes initializing; continue when both are present. +Both nodes share the first node's ZooKeeper, rather than forming separate clusters. +The working directory holds both nodes' data, certificates, and a private configuration file. +Node logs are under `tutorial/secure/node1/logs` and `tutorial/secure/node2/logs`. + +== Create a Certificate and PKCS12 Stores + +Generate a private key and self-signed certificate directly in a PKCS12 keystore. +The subject alternative names cover both names used in this exercise: + +[source,bash] +---- +keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 \ + -validity 365 -dname 'CN=localhost' \ + -ext 'SAN=DNS:localhost,IP:127.0.0.1' \ + -storetype PKCS12 -keystore "$TUTORIAL_HOME/tls/solr-ssl.keystore.p12" \ + -storepass SolrRocks -keypass SolrRocks +keytool -exportcert -alias solr-ssl -rfc \ + -keystore "$TUTORIAL_HOME/tls/solr-ssl.keystore.p12" -storetype PKCS12 \ + -storepass SolrRocks -file "$TUTORIAL_HOME/tls/cert.pem" +keytool -importcert -noprompt -alias solr-ssl \ + -file "$TUTORIAL_HOME/tls/cert.pem" \ + -keystore "$TUTORIAL_HOME/tls/solr-ssl.truststore.p12" -storetype PKCS12 \ + -storepass SolrRocks +openssl x509 -in "$TUTORIAL_HOME/tls/cert.pem" -noout -subject -dates -text +---- + +The keystore contains the private key and certificate presented by each server. +The truststore contains the certificate trusted by Solr's outbound HTTPS clients. +The PEM certificate lets `curl` verify the server without disabling certificate checks. +Using the same keystore on both nodes simplifies the exercise; real deployments should manage certificates and private keys separately for each node. + +== Enable HTTPS on Both Nodes + +Write an include file used only for this exercise. +The unquoted heredoc expands the absolute working-directory path now; escaped dollar signs preserve variables for later launches: Review Comment: [0] Might be simpler if we just give them the file content to copy/paste, rather than giving them a heredoc syntax and having to explain how various quotes are escaped or not escaped 😛 ########## solr/solr-ref-guide/modules/getting-started/pages/tutorial-securing-solr.adoc: ########## @@ -0,0 +1,511 @@ += Exercise 7: Securing Solr +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +This exercise starts a two node Solr cluster, enables HTTPS, assigns separate administration, indexing, and search accounts, and protects ZooKeeper data with access control lists (ACLs). +You will verify HTTPS for both client requests and requests between Solr nodes. +Each step uses the tools included in the Solr distribution, JDK `keytool`, OpenSSL, and `curl`. + +Use a fresh full Solr installation on Unix or macOS, with ports 8983, 8984, and 9983 available. +Run the commands in the same Bash-compatible shell, from the installation directory, using an installation path without spaces. +Do not run these commands against an existing cluster. + +IMPORTANT: This is a local learning exercise with demonstration passwords and a shared self-signed certificate. +Keep the cluster on loopback. Review Comment: [0] "Keep the cluster on loopback" implies it, but IMO we should really say explicitly here: "this exact set up is not ready or appropriate for production" ########## solr/bin/solr: ########## @@ -636,6 +637,15 @@ fi # Everything below here is to support start, stop and restart. +if [[ "$SCRIPT_CMD" == "start" ]]; then + for example_arg in "$@"; do + if [[ "$example_arg" == "-e" || "$example_arg" == "--example" ]]; then Review Comment: Also, what's wrong with running the ulimit check? Is the thought that we only want that check run for folks going into production or something? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
