zxs1633079383 commented on PR #870: URL: https://github.com/apache/flink-agents/pull/870#issuecomment-4884814604
Updated in e023346. Design change: - Removed the Java `ThreadLocal` and Python `ContextVar` token-usage side channel. - Added a result-plus-usage path: Java `EmbeddingResult<T>` / `embedWithUsage(...)`, Python `EmbeddingResult` / `embed_with_usage(...)`. - Kept existing public `embed(...)` return types unchanged; default wrappers return embeddings with no usage. - `BaseEmbeddingModelSetup` now records metrics only from a successful returned result. - Bedrock batch embedding now merges token usage from worker results on the caller path. Regression coverage added/updated: - Java/Python usage recorded and no-usage no-op cases. - Java/Python provider-failure-then-success cases, verifying no stale usage leaks into the next success. - Bedrock batch size > 1 usage aggregation. - OpenAI/Tongyi usage extraction remains covered. Local verification: - `uv run --python 3.12 --extra test pytest flink_agents/api/embedding_models/tests/test_token_metrics.py flink_agents/integrations/embedding_models/tests/test_openai_embedding_model.py flink_agents/integrations/embedding_models/tests/test_tongyi_embedding_model.py -q` - `uv run --python 3.12 --extra lint ruff format --check ... && uv run --python 3.12 --extra lint ruff check ...` - `mvn --batch-mode --no-transfer-progress -pl api -Dtest=BaseEmbeddingModelSetupTokenMetricsTest test` - `mvn --batch-mode --no-transfer-progress -pl integrations/embedding-models/bedrock -am -Dtest=BedrockEmbeddingModelTest -Dsurefire.failIfNoSpecifiedTests=false test` - `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
