This approach looks good to me, and tames the somewhat ad-hoc approach we've had so far.
On Wed, Jan 29, 2025 at 9:14 AM Kenneth Knowles <k...@apache.org> wrote: > Looks good to me. > > I've been told in the past that it is more "pythonic" to have one large > module with codepaths that are always there but don't work unless you > installed with extras, versus having multiple modules that are fully > functional without extras. Is that still the prevailing view for Beam? > By "module" to you mean package (e.g. that one would install with "pip install foo"? I would say the notion of extras is already a bit off the beaten path, generally one tries to be batteries-included. WIth these (very heavy-weight) ML dependencies I don't think there's another option though. Generally it still seems that extras are preferable to separate foo-X, foo-Y, etc. if the versions are tightly coupled (e.g. foo-X version N requires exactly foo version N) and/or they live in the same namespace. It seems like the combinatorial complexity of testing is getting pretty > extraordinary. > We certainly can't test all possible combinations, but hopefully in this particular case testing each in isolation is good enough. > On Mon, Jan 27, 2025 at 3:35 PM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >> Hey everyone, I put together a mini-doc on bundling some more Beam >> Python/ML extras so that we have a better strategy for making sure that >> users can use ML (or other Python) dependencies which are well tested with >> their Beam version. It is mostly in line with how we handle our other >> dependencies and only expands that scope a bit with some new extras, but it >> should give us a well-defined strategy moving forward which we can >> reference back to as needed. >> >> Please take a look here if you're interested - >> https://docs.google.com/document/d/1c84Gc-cZRCfrU8f7kWGsNR2o8oSRjCM-dGHO9KvPWPw/edit?usp=sharing >> >> Thanks, >> Danny >> >
