Gerrrr commented on code in PR #170: URL: https://github.com/apache/otava/pull/170#discussion_r3840852265
########## tests/influxdb_e2e_test.py: ########## @@ -0,0 +1,109 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import subprocess +from datetime import datetime, timezone +from pathlib import Path + +import pytest +from e2e_test_utils import container + +from otava.config import load_config_from_file +from otava.data_selector import DataSelector +from otava.importer import InfluxDBImporter +from otava.influxdb import InfluxDB + +INFLUXDB_IMAGE = "influxdb:3.11.2-core" +INFLUXDB_PORT = 8181 +INFLUXDB_TOKEN = "apiv3_otava_example_admin_token_2026" +EXAMPLE_DIR = Path("examples/influxdb").resolve() + + +def test_influxdb_sql_and_influxql_return_identical_seeded_data(): + with container( + INFLUXDB_IMAGE, + command=[ + "influxdb3", + "serve", + "--node-id=otava-e2e", + "--object-store=memory", + "--admin-token-file=/example/admin-token.json", + ], + ports=[INFLUXDB_PORT], + volumes={str(EXAMPLE_DIR): "/example:ro"}, + ) as (container_id, port_map): + seed = subprocess.run( + [ + "docker", + "exec", + "--env", + f"INFLUXDB3_HOST_URL=http://127.0.0.1:{INFLUXDB_PORT}";, + "--env", + f"INFLUXDB3_AUTH_TOKEN={INFLUXDB_TOKEN}", + "--env", + "INFLUXDB3_DATABASE_NAME=performance", + container_id, + "/bin/sh", + "/example/seed.sh", + ], + capture_output=True, + text=True, + timeout=120, + ) + if seed.returncode != 0: + pytest.fail( + "InfluxDB seed command returned non-zero exit code.\n\n" + f"Command: {seed.args!r}\n" + f"Exit code: {seed.returncode}\n\n" + f"Stdout:\n{seed.stdout}\n\n" + f"Stderr:\n{seed.stderr}\n" + ) + + host = f"http://localhost:{port_map[INFLUXDB_PORT]}"; + config = load_config_from_file( + str(EXAMPLE_DIR / "otava.yaml"), + arg_overrides=[ + "--influxdb-host", + host, + "--influxdb-token", + INFLUXDB_TOKEN, + ], + ) + importer = InfluxDBImporter(InfluxDB(config.influxdb)) Review Comment: This test starts a real InfluxDB server, but it bypasses Otava’s public execution path by constructing InfluxDBImporter directly. As a result, it would still pass if the CLI options, configuration wiring, `Importers.get()` registration, analysis, or reporting were broken. Could this invoke `otava analyze`, as the PostgreSQL and Graphite E2E tests do, and assert the resulting output? ########## docs/INFLUXDB.md: ########## @@ -0,0 +1,99 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + --> + +# Importing results from InfluxDB 3 + +Otava imports query results from InfluxDB 3 Core or Enterprise through the +[`influxdb3-python`](https://docs.influxdata.com/influxdb3/core/reference/client-libraries/v3/python/) +client. SQL is the default query language; set `query_language: influxql` for +InfluxQL queries. + +## Connection + +```yaml +influxdb: + host: http://localhost:8181 + database: performance + token: ${INFLUXDB_TOKEN} +``` + +The same settings are available through `INFLUXDB_HOST`, `INFLUXDB_DATABASE`, +and `INFLUXDB_TOKEN`, or the `--influxdb-host`, `--influxdb-database`, and +`--influxdb-token` command-line options. Command-line values take precedence +over environment variables, which take precedence over YAML. + +## Reproducible example + +The bundled example starts InfluxDB 3 Core with authenticated, in-memory +storage, seeds deterministic latency data, and runs Otava against it: + +```bash +docker build -t apache/otava:latest . +docker compose -f examples/influxdb/docker-compose.yaml run --rm otava \ + analyze api_latency_sql --branch main --since 2025-01-01 +docker compose -f examples/influxdb/docker-compose.yaml down +``` + +Run `api_latency_influxql` instead to query the same data with InfluxQL. + +The admin token committed under `examples/influxdb/` is a fixed test +credential, and the server discards its in-memory data when stopped. Both are +for this local demonstration only. Use a securely generated token and durable +object storage for production deployments. + +## Test configuration + +```yaml +tests: + api_latency: + type: influxdb + query_language: sql + query: | + SELECT time, branch, p95_ms, commit + FROM api_latency + WHERE branch = %{BRANCH} + ORDER BY time + time_column: time + attributes: [branch, commit] + metrics: + p95: + column: p95_ms + direction: -1 + scale: 1 + + legacy_api_latency: + type: influxdb + query_language: influxql + query: SELECT time, branch, p95_ms FROM api_latency WHERE branch = %{BRANCH} + attributes: [branch] + metrics: [p95_ms] +``` + +Metric definitions use `column`, `direction`, and `scale` as with the other +SQL-backed importers. `%{BRANCH}` is replaced with an escaped string literal +when `--branch` is supplied. + +Run the analysis with: + +```bash +otava analyze api_latency_sql --branch main --last 100 Review Comment: The configuration above defines the test as `api_latency`, but this command runs `api_latency_sql`. Let's rename `api_latency` above into `api_latency_sql` to make it consistent with the rest of the doc and the example. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
