weiqingy opened a new pull request, #930: URL: https://github.com/apache/flink-agents/pull/930
Linked issue: #280 ### Purpose of change The Azure OpenAI connections reject a non-null `output_schema`, so every caller with a schema stays on the prompt-engineering fallback even when the deployment's backing model supports native structured output. This applies the native path for them, in Java and Python together, on top of the mechanism added in #843. Two things differ from the OpenAI connection, both forced by how Azure works rather than by preference. Capability is keyed on `model_of_azure_deployment`, not `model`. On Azure the `model` parameter is the deployment name, an arbitrary string the user chose, so matching an allowlist against it is wrong in both directions: a deployment called `prod-chat` backed by a capable model would report not-capable, and a deployment called `gpt-4o` backed by anything would report capable. `model_of_azure_deployment` is the only input that names the model actually behind the deployment. When it is unset, capability reports false and the caller stays on the prompt path, which is the safe direction. The native path additionally requires an `api_version` at or above `2024-08-01`. This is a safety gate rather than an optimization. Azure documents `2024-08-01-preview` as the first version supporting structured outputs, but I could not determine whether an older version returns an error or silently ignores `response_format`, and neither SDK does any client-side gating. Never sending below the floor makes both possible behaviors safe: if the service would reject it, the call is never made; if it would silently ignore it, the native path is never taken so the prompt fallback still applies. This matters because the docs currently recommend `2024-02-15-preview` and `2024-02-01`, both below the floor. A bare `gpt-4o` is deliberately absent from the allowlist. Azure reports a model and its version as separate properties, so a bare `gpt-4o` may be the `2024-05-13` build, which does not support the feature, and no available input disambiguates it. This matches the same exclusion made for the OpenAI connection. Building the Java request inline left no seam to observe, so `chat()` is split into a 4-arg override, a private `doChat`, and package-private `buildRequest` and `toResponse`. The split turns a single read of `model_of_azure_deployment` into two reads that must agree; `toResponse` makes that testable. Reading the key after the call is equivalent to reading it before because the parameter map is built fresh per call and never retained. One behavior worth calling out: if a caller supplies their own `response_format` and the native path is about to write one, the connection now raises instead of silently replacing it. Before this change that combination was unreachable; it produced a raw SDK error on one channel and dropped the caller's value on the other. Passing `response_format` without a schema is unaffected on every non-native path. Azure AI Inference (the `azureai` module) is **not** included. Native structured output cannot work there at the pinned `azure-ai-inference:1.0.0-beta.5`: `json_schema` is rejected below api-version `2024-08-01-preview`, but raising `ModelServiceVersion` to that value makes the endpoint return 404, which is why the SDK pins its own default down. I have recorded the details on #280 so nobody repeats the investigation. ### Tests Both languages pin the same set of behaviors: native applied for a capable model with a POJO/BaseModel schema and an adequate api-version; not applied for an incapable, unknown, or bare-`gpt-4o` model; not applied for a `RowTypeInfo` schema or a null schema; not applied below the api-version floor; the capability predicate's true and false cases; the collision raising; and a caller-supplied `response_format` passing through untouched on every non-native path. Python: 54 tests pass in `flink_agents/integrations/chat_models` (non-integration), 447 across `api` and `plan`. The new tests are in their own non-integration file, since the existing Azure test module is integration-marked and would be skipped in the unit run. Java: 58 tests pass in the `openai` module, `spotless:check` clean, no compiler warnings. The five pre-existing tests in `AzureOpenAIChatModelConnectionTest` are unmodified and still pass, including the ones guarding the exception behavior around the refactored region. The capable-model list and the api-version floor are identical in both languages. ### API No new public type, no new configuration field, and no signature change to an existing method. Java adds a 4-arg `chat` override of the overload introduced in #843; Python's `output_schema` parameter already existed and is now honored rather than rejected. A caller that passes no `output_schema` sees unchanged behavior. No dependency change in either language, and no build file is touched. ### Documentation - [ ] `doc-needed` - [x] `doc-not-needed` - [ ] `doc-included` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
