rosemarYuan opened a new pull request, #937: URL: https://github.com/apache/flink-agents/pull/937
<!-- * Thank you very much for contributing to Flink Agents. * Please add the relevant components in the PR title. E.g., [api], [runtime], [java], [python], [hotfix], etc. --> <!-- Please link the PR to the relevant issue(s). Hotfix doesn't need this. --> Linked issue: #754 > **Stacked PR** — this PR is based on #821. Please review the [incremental diff](https://github.com/apache/flink-agents/compare/5eaff2d6...fe7c2577), not the full diff against `main`. ### Purpose of change This is the third PR in the four-PR stack tracked by #754: 1. #756 - API changes (under review) 2. #821 - Java CEL runtime 3. **This PR** — Python CEL 4. Documentation (planned) It adds `trigger_conditions` to the Python `@action` decorator, `Agent.add_action`, and YAML. Python keeps the conditions in their original order as raw strings in `AgentPlan` JSON. This implementation does not add a Python CEL library. `AgentPlan.from_agent()` checks only that each Action has a non-empty list of non-blank strings. For a remote job, `apply()` sends the full Plan JSON to `AgentPlanPreflight` through the existing PyFlink JVM. Java then checks the CEL rules before Python builds the output `DataStream`. An invalid Plan raises `ValueError`, while a JVM or bridge failure raises `RuntimeError`. Java remains responsible for CEL validation, compilation, and Action matching. Python collects the raw conditions and runs only the Python Actions selected by Java. ### Tests - Python API, Plan, and remote `apply()` tests: They cover `@action`, `Agent.add_action`, YAML aliases, raw condition order, basic data checks, Plan JSON, calls to Java preflight, user errors, bridge errors, and retry after a failed `apply()`. - Java Plan tests: They cover valid Plans, invalid CEL with Action and condition details, invalid Plan shape, JSON deserialization, and Python/Java Plan snapshots. - Cross-language tests: They cover real Java preflight from Python, a Python Agent running a Java Action, and a Java Agent running a Python Action. The same Action runs only once when two conditions both match. ### API - Python Action declarations and YAML now support ordered `trigger_conditions`. - Remote `apply()` can raise `ValueError` when the Plan is invalid, before the Flink job is build. ### Notes We also considered two ways to validate CEL fully in Python, but neither is a good fit for this PR. | Option | Why not selected | | --- | --- | | Use the community `cel-python` library with Lark | The Lark tree contains only the parsed structure, not the full CEL rules. We would need to write and maintain every CEL validation rule ourselves, which could get out of sync with Java. | | Use the official `cel-expr-python` library backed by CEL C++ | It does not support Python 3.10 yet, [our question](https://github.com/cel-expr/cel-python/issues/96) about support has not received a reply, and some APIs needed by this project are not exposed. | Calling Java from Python Plan validation adds a JVM dependency for detailed CEL checks, but this is acceptable because supported Python jobs already run through PyFlink and use the same JVM. ### Documentation <!-- Do not remove this section. Check the proper box only. --> - [ ] `doc-needed` <!-- Your PR changes impact docs --> - [x] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [ ] `doc-included` <!-- Your PR already contains the necessary documentation updates --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
