tvalentyn commented on code in PR #37725:
URL: https://github.com/apache/beam/pull/37725#discussion_r2886657856
##########
sdks/python/tox.ini:
##########
@@ -515,13 +515,14 @@ commands =
[testenv:py{310,311}-transformers-{428,447,448,latest}]
deps =
# Environment dependencies are defined in the `setenv` section and installed
in the `commands` section.
-extras = test,gcp,ml_test
+pip_pre = False
+extras = test
setenv =
COMMON_DEPS = tensorflow==2.12.0 protobuf==4.25.5 pip==25.0.1
# sentence-transformers 2.2.2 is the latest version that supports
transformers 4.28.x
428: DEPS = sentence-transformers==2.2.2 'transformers>=4.28.0,<4.29.0'
'torch>=1.9.0,<1.14.0'
447: DEPS = 'transformers>=4.47.0,<4.48.0' 'torch>=1.9.0,<1.14.0'
- 455: DEPS = 'transformers>=4.55.0,<4.56.0' 'torch>=2.0.0,<2.1.0'
+ 448: DEPS = 'transformers>=4.48.0,<4.49.0' 'torch>=1.9.0,<1.14.0'
Review Comment:
the point of this dependency workflow is to run test scenarios using
multiple versions of a dependency. We only do this for dependencies we care
about and for versions we care about.
For example, for pyarrow, we have an extensive test suite that exercises
every version. It is an outlier. In the vast majority of tests, we only test
the lastest supported versions of a dependency, in some cases we test the
earliest version we support, and the latest version we support.
Looking at this change it is not obvious to me why we making a change to
test transformers 4.48.x instead of transformers 4.55.x.
Looking at
https://github.com/apache/beam/blob/e7a97c26893e9d25fb0224c3c5b3bcdd5ffe569a/sdks/python/container/ml/py312/gpu_image_requirements.txt#L290
we do use transformers 4.55.x
Hence I asked the question about the motivation of your change since it
isn't obvious.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
