weiqingy opened a new pull request, #702:
URL: https://github.com/apache/flink-agents/pull/702
Linked issue: #161
### Purpose of change
Splits the Python unit-test job from the integration-test job using a new
`@pytest.mark.integration` marker.
Today, 10 tests under `python/flink_agents/integrations/*/tests/` need live
external services (Ollama, DashScope, OpenAI, Azure, Anthropic, Chroma) or
optional Python deps (mem0), but they live in the unit-test job and silently
skip in 5 of 6 matrix cells — only the Python 3.10 cell auto-pulls Ollama. When
Ollama regresses, exactly one cell reports failure, which looks like
environmental noise.
After this PR:
- **`python/pyproject.toml`** — new `[tool.pytest.ini_options]` block
registers the `integration` marker and enables `strict_markers = true`.
- **10 test files under `python/flink_agents/integrations/*/tests/`** —
module-level `pytestmark = pytest.mark.integration` (list form for
`test_mem0_vector_store.py` to preserve its existing `skipif`).
- **`tools/ut.sh`** — both `if $run_e2e; then ... else ... fi` blocks (uv
branch + pip fallback) updated:
- `ut-python` (`run_e2e=false`): appends `-m "not integration"` to the
existing `-k "not e2e_tests"` filter. Drops 70 tests from the unit-test job
(was 518, now 448).
- `it-python` (`run_e2e=true`): runs two sequential pytest invocations —
existing `-k "e2e_tests_integration"` (27 tests, unchanged) and new `-m
"integration"` (70 tests) — with aggregated exit codes and an exit-5 trap on
the integration arm to defend against `-m` selector typos that
`--strict-markers` cannot catch.
No `.github/workflows/ci.yml` change required.
`integrations/mcp/tests/test_mcp.py` is intentionally not tagged: it
self-hosts an MCP server via `multiprocessing` and runs deterministically
without external deps.
### Tests
Locally verified empirical contract (`pytest --collect-only -q`):
| Selector | Tests | Notes |
|----------|-------|-------|
| `pytest -m "not integration" -k "not e2e_tests"` | **448** | New
`ut-python` arm (was 518) |
| `pytest -m "integration"` | **70** | New `it-python` arm 2 |
| `pytest -k "e2e_tests_integration"` | **27** | `it-python` arm 1,
unchanged |
| no selector | 552 | Unchanged |
Overlap between `-m "integration"` and `-k "e2e_tests_integration"` is **0**
— disjoint by directory structure.
Additional verification:
- `./tools/lint.sh -c` passes
- `./tools/check-license.sh` passes
- `./tools/ut.sh -p` passes locally (448 passed, 104 deselected, ~28s)
- `pytest --markers` shows the registered `integration` marker with
description
- Typo'd `@pytest.mark.intergration` triggers a collection error
(`strict_markers = true` enforcement is active)
### API
No public API changes. This PR is CI plumbing plus a pytest marker added to
existing test files.
### Documentation
- [ ] `doc-needed`
- [x] `doc-not-needed`
- [ ] `doc-included`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
