Copilot commented on code in PR #5109: URL: https://github.com/apache/solr/pull/5109#discussion_r4251164809
########## solr/monitoring/dev/docker-compose.yml: ########## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# ============================================================================ +# DEVELOPER CONVENIENCE ONLY — NOT FOR PRODUCTION USE +# ============================================================================ +# +# This docker-compose stack is provided solely to make it easy for contributors +# to view and test changes to the two generated monitoring artifacts: +# +# ../grafana-solr-dashboard.json — Grafana dashboard +# ../prometheus-solr-alerts.yml — Prometheus alert rules +# +# It is NOT intended as a reference deployment or a recommended production setup. +# For integrating the artifacts into your own Prometheus + Grafana installation, +# see the Solr Reference Guide (monitoring-with-prometheus-and-grafana.adoc). +# +# Quick start: +# cd solr/monitoring/dev +# ./stack.sh # Full stack with bundled Solr nodes +# ./stack.sh --local-solr # Monitoring only — point at your locally-running Solr +# ./stack.sh --help # All options +# +# Services (full stack): +# solr1 http://localhost:8983 (SolrCloud node 1, embedded ZooKeeper) +# solr2 http://localhost:8984 (SolrCloud node 2) +# Prometheus http://localhost:9090 +# Grafana http://localhost:3000 (admin / admin) +# Alertmanager http://localhost:9093 +# trafficgen (no port — continuously indexes/searches the 'test' collection) +# +# Environment variables (set by stack.sh, or override manually): +# PROMETHEUS_CONFIG Path to the prometheus config to mount (default: ./prometheus/prometheus.yml) +# SOLR_URL Solr base URL for trafficgen (default: http://solr1:8983/solr) +# SOLR_IMAGE Solr image for the bundled nodes (default: solr:10) +# PROMETHEUS_IMAGE Prometheus image (default: pinned, see below) +# GRAFANA_IMAGE Grafana image (default: pinned, see below) +# ALERTMANAGER_IMAGE Alertmanager image (default: pinned, see below) +# +# The monitoring images are pinned so a stack that worked yesterday still works +# today; override any of them to test against a newer release. +# ============================================================================ + +services: + solr1: + image: ${SOLR_IMAGE:-solr:10} + profiles: [bundled-solr] + ports: + - "8983:8983" + - "9983:9983" # embedded ZooKeeper (Solr 10 default: ZK_PORT = SOLR_PORT + 1000) Review Comment: Docker's short port syntax binds these unauthenticated Solr and ZooKeeper ports on every host interface, not just localhost. Running the advertised local stack on a shared network therefore exposes admin APIs and ZooKeeper, contrary to Solr's required network-isolated posture. Bind the host side explicitly to loopback. This issue also appears on line 74 of the same file. ########## solr/monitoring/mixin/dashboards/dashboards.libsonnet: ########## @@ -0,0 +1,96 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// dashboards.libsonnet — placeholder dashboard. +// +// Two panels, enough to show that the mixin build produces a dashboard Grafana +// can load and that config.libsonnet overrides reach the generated queries. +// The real dashboard replaces this wholesale; see SOLR-18147. + +local config = import '../config.libsonnet'; +local g = import 'github.com/grafana/grafonnet/gen/grafonnet-latest/main.libsonnet'; + +local d = g.dashboard; +local p = g.panel; +local q = g.query.prometheus; +local v = g.dashboard.variable; +local cfg = config._config; + +local datasourceVar = + v.datasource.new('datasource', 'prometheus') + + v.datasource.generalOptions.withLabel('Data Source'); + +local intervalVar = + v.interval.new('interval', ['1m', '5m', '10m', '30m', '1h']) + + v.interval.generalOptions.withCurrent(cfg.rateInterval) + + v.interval.generalOptions.withLabel('Interval'); + +local prom(expr, legend='{{instance}}') = + q.new('$datasource', expr) + + q.withLegendFormat(legend); + +local ts(title, exprs, unit='short', desc='') = + p.timeSeries.new(title) + + p.timeSeries.queryOptions.withTargets(exprs) + + p.timeSeries.standardOptions.withUnit(unit) + + p.timeSeries.panelOptions.withDescription(desc) + + p.timeSeries.options.legend.withDisplayMode('list') + + p.timeSeries.options.tooltip.withMode('multi'); + +local panels = [ + ts( + 'Request Rate', + [prom('sum by (instance)(rate(solr_core_requests_total{%s}[$interval]))' % cfg.solrSelector)], + unit='reqps', + desc='Requests per second per node, across all handlers and cores.' + ) + { gridPos: { x: 0, y: 0, w: 12, h: 8 } }, + + ts( + 'Heap Used', + [prom('max by (instance)(jvm_memory_used_bytes{%s,jvm_memory_type="heap"})' % cfg.solrSelector)], + unit='bytes', + desc='JVM heap in use per node. The max aggregation collapses the OTel instrumentation scopes that report this metric, which differ by Solr version.' + ) + { gridPos: { x: 12, y: 0, w: 12, h: 8 } }, Review Comment: `jvm.memory.used` emits a separate series for each memory pool (`jvm_memory_pool_name`). Aggregating with `max by (instance)` therefore keeps only the largest heap pool, so this panel understates total heap usage on generational collectors. Sum the heap pools per instance instead, de-duplicating instrumentation scopes first only if a supported version can emit duplicate scopes. ########## solr/monitoring/dev/prometheus/prometheus-local.yml: ########## @@ -0,0 +1,41 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# prometheus-local.yml — scrape config for a Solr you run yourself. +# +# Used by: ./stack.sh --local-solr +# +# 'host.docker.internal' resolves to the host machine from inside Docker (Mac, +# Windows, and Linux with Docker 20.10+). docker-compose.yml adds the matching +# extra_hosts entry so this also works on Linux. +# +# A single node on :8983 is scraped by default. Add further ports to the targets +# list to match a multi-node local SolrCloud setup. + +global: + scrape_interval: 15s + evaluation_interval: 15s + +rule_files: + - /etc/prometheus/rules/solr-alerts.yml Review Comment: Local-Solr mode also loads the rules without configuring an Alertmanager target, so its alerts remain only in Prometheus and never appear in the bundled Alertmanager. Add the compose service as an alerting target here as well. ########## solr/monitoring/dev/prometheus/prometheus.yml: ########## @@ -0,0 +1,36 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# prometheus.yml — scrape config for the bundled Solr nodes in the dev stack. +# +# Used by docker-compose.yml in this directory. The job name must match the +# mixin's solrSelector (job="solr" by default), or the generated dashboard and +# alert rules will not match any series. + +global: + scrape_interval: 15s + evaluation_interval: 15s + +rule_files: + - /etc/prometheus/rules/solr-alerts.yml Review Comment: The rule file is loaded and evaluated, but Prometheus has no Alertmanager target configured, so firing `SolrNodeDown` alerts will never reach the Alertmanager service in this stack. Add the compose service as an alerting target. ########## solr/monitoring/mixin/make.sh: ########## @@ -0,0 +1,60 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# make.sh — Docker-based wrapper around the mixin Makefile. +# +# Usage: +# ./make.sh # equivalent to: make all +# ./make.sh install all # equivalent to: make install all +# ./make.sh check # equivalent to: make check +# ./make.sh --rebuild # force-rebuild the Docker image, then make all +# +# The Docker image (solr-mixin-make:latest) is built automatically on first +# use and reused on subsequent runs. Pass --rebuild to force a fresh build +# (e.g. after updating the Dockerfile or tool versions). + +set -euo pipefail + +IMAGE="solr-mixin-make:latest" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +MONITORING_DIR="$(dirname "$SCRIPT_DIR")" + +# --- parse flags ------------------------------------------------------------- +REBUILD=false +MAKE_ARGS=() +for arg in "$@"; do + case "$arg" in + --rebuild) REBUILD=true ;; + *) MAKE_ARGS+=("$arg") ;; + esac +done + +# --- build image if missing or explicitly requested -------------------------- +if ! docker image inspect "$IMAGE" >/dev/null 2>&1 || [ "$REBUILD" = true ]; then + echo "[make.sh] Building Docker image $IMAGE ..." + docker build -t "$IMAGE" "$SCRIPT_DIR" +fi + +# --- run make inside the container ------------------------------------------- +# Mount the monitoring/ directory as /work so that the Makefile's relative +# paths (../grafana-solr-dashboard.json, ../prometheus-solr-alerts.yml) resolve +# correctly to the host filesystem. +TTY_FLAG="" +[ -t 0 ] && TTY_FLAG="-t" + +docker run --rm -i $TTY_FLAG \ + -v "$MONITORING_DIR:/work" \ + "$IMAGE" "${MAKE_ARGS[@]:-all}" Review Comment: On native Linux, this container runs as root and atomically replaces the bind-mounted generated artifacts (and creates `vendor/`) as root-owned files. After using the advertised wrapper, contributors may be unable to edit, regenerate locally, or clean those files without elevated permissions. Run the build with the invoking UID/GID and a writable home directory. ########## changelog/unreleased/SOLR-18533-monitoring-dev-tooling.yml: ########## @@ -0,0 +1,13 @@ +# See https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc + +title: > + Add tooling for building Solr's Grafana dashboard and Prometheus alert rules from jsonnet sources, + under `solr/monitoring`, together with a docker-compose stack for testing them locally. The + dashboard and alert rules shipped so far are placeholders; the real ones follow in SOLR-18147. Review Comment: This points readers to the original SOLR-18147 POC, while this PR defines SOLR-18534 and SOLR-18535 as the dashboard and alert-rule follow-ups. Name those two follow-up issues so the release note directs users to the work that will actually replace each placeholder. ########## solr/monitoring/mixin/mixin.libsonnet: ########## @@ -0,0 +1,40 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// mixin.libsonnet — root entry point for the Solr monitoring mixin. +// +// Exposes: +// grafanaDashboards:: — map of filename -> Grafana dashboard JSON object +// prometheusAlerts:: — Prometheus alert groups object (for -S / YAML output) +// +// Usage: +// jsonnet -J vendor -e '(import "mixin.libsonnet").grafanaDashboards["grafana-solr-dashboard.json"]' \ +// > ../grafana-solr-dashboard.json +// jsonnet -J vendor -S -e '(import "mixin.libsonnet").prometheusAlerts' \ +// | gojsontoyaml > ../prometheus-solr-alerts.yml + +local alerts = import 'alerts/alerts.libsonnet'; +local config = import 'config.libsonnet'; +local dashboards = import 'dashboards/dashboards.libsonnet'; + +config { + // grafanaDashboards is a map from output filename to dashboard JSON. + // The Makefile uses the key "grafana-solr-dashboard.json" to generate the output file. + grafanaDashboards:: dashboards, + + // prometheusAlerts is the Prometheus alert rule groups structure. + // Pipe through gojsontoyaml to produce the YAML file. + prometheusAlerts:: alerts, Review Comment: The documented external override API does not affect either output. `dashboards` and `alerts` are lexically imported before this object is extended, and each imports its own standalone `config.libsonnet`, so `(import 'mixin.libsonnet') + { _config+:: ... }` changes only the root field while generated selectors remain `job="solr"`. Thread the root configuration into dashboard and alert constructors, or derive both outputs from `self._config`. ########## solr/monitoring/README.md: ########## @@ -0,0 +1,203 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + the "License"); you may not use this file except in compliance with Review Comment: The Apache License header is missing the opening parenthesis before the defined term. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
