goutamadwant opened a new issue, #13178:
URL: https://github.com/apache/seatunnel/issues/13178

   ### Search before asking
   
   - [x] I searched existing public issues and PRs and found no same-purpose 
diagnostic-hardening fix. #8604 is historical LLM error-handling background, 
not a functional failure this change claims to resolve.
   
   ### What happened
   
   LLM inference errors include the complete input row and retain unsafe nested
   provider/parser exceptions. All four HTTP adapters also put non-200 response 
bodies
   into IOExceptions. As a result, printed failure stacks can include input or 
provider
   content even though neither is needed to diagnose the HTTP status/error 
category.
   
   With synthetic markers and an HTTP stub returning 503, the new regression 
fails
   on unchanged dev for OPENAI, MICROSOFT, KIMIAI, CUSTOM, DEEPSEEK, DOUBAO and 
ZHIPU.
   Malformed 200 responses and numeric conversion failures are additional 
affected paths.
   
   Expected: omit rows, prompts, provider payloads and unsafe throwable text 
from LLM
   initialization/inference diagnostics, retaining safe 
provider/model/status/category
   metadata. Preserve successful outputs, options, serialization and invocation 
attempts.
   
   This is diagnostic hardening. The reproduced destinations are task logs and
   administrative job diagnostics; no unauthorized access or authorization 
bypass is asserted.
   
   ### SeaTunnel Version
   
   3.1.0-SNAPSHOT, dev commit `f54bb5876bd282b88181b6d93c9f59ec5dd38621`.
   
   ### SeaTunnel Config
   
   The regression constructs an LLM transform with `model=test-model`,
   `api_key=test-key`, `output_data_type=STRING`, a loopback HTTP-stub 
endpoint, and
   one input row containing `sensitive-row-value`. CUSTOM additionally supplies 
its
   existing request-header/body/JSONPath configuration. The stub returns HTTP 
503
   with body `private-provider-response`. All values are synthetic.
   
   The complete reproducible configuration and stub are in
   
[LLMBooleanOutputTest](https://github.com/goutamadwant/seatunnel/blob/567b0321eefa300f6316ca44c28eafa50a69f70e/seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/llm/LLMBooleanOutputTest.java).
   
   ### Running Command
   
   Use a fresh checkout and bring only the regression test onto the unchanged 
base,
   not the production fix:
   
   ```shell
   git clone https://github.com/apache/seatunnel.git 
seatunnel-llm-diagnostics-repro
   cd seatunnel-llm-diagnostics-repro
   git fetch https://github.com/goutamadwant/seatunnel.git 
567b0321eefa300f6316ca44c28eafa50a69f70e
   git switch --detach f54bb5876bd282b88181b6d93c9f59ec5dd38621
   git restore --source FETCH_HEAD -- 
seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/llm/LLMBooleanOutputTest.java
   mvn -B -pl seatunnel-transforms-v2 
'-Dtest=LLMBooleanOutputTest#inferenceFailureDoesNotExposeRowOrProviderBody' 
-Dskip.spotless=true test
   ```
   
   Set `JAVA_HOME` to JDK 11 or 17. Expected baseline result: 7 tests, 7 
assertion
   failures because the input marker appears in the printed exception stack. 
The same
   test passes with the production fix. No provider account is needed.
   
   ### Error Exception
   
   Relevant synthetic baseline messages (row metadata abbreviated):
   
   ```text
   java.lang.RuntimeException: Failed to inference model with row 
SeaTunnelRow{... fields=[sensitive-row-value]}
   Caused by: java.io.IOException: Failed to chat with model, response: 
private-provider-response
   ```
   
   ### Zeta or Flink or Spark Version
   
   Zeta from the same 3.1.0-SNAPSHOT checkout. The unit reproducer directly 
exercises
   the LLM transform and HTTP adapters; it does not depend on an engine version.
   
   ### Java or Scala Version
   
   JDK 11 and JDK 17. No Scala-specific changes.
   
   ### Screenshots
   
   Not required; textual synthetic regression evidence is provided above.
   
   ### Are you willing to submit PR?
   
   - [x] Yes, I am willing to submit a PR.
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to