+1 On Wed, Jan 29, 2025 at 12:31 PM Robert Bradshaw via dev < dev@beam.apache.org> wrote:
> This approach looks good to me, and tames the somewhat ad-hoc approach > we've had so far. > > On Wed, Jan 29, 2025 at 9:14 AM Kenneth Knowles <k...@apache.org> wrote: > >> Looks good to me. >> >> I've been told in the past that it is more "pythonic" to have one large >> module with codepaths that are always there but don't work unless you >> installed with extras, versus having multiple modules that are fully >> functional without extras. Is that still the prevailing view for Beam? >> > > By "module" to you mean package (e.g. that one would install with "pip > install foo"? > Yes, thanks for the correction. > I would say the notion of extras is already a bit off the beaten path, > generally one tries to be batteries-included. WIth these (very > heavy-weight) ML dependencies I don't think there's another option though. > > Generally it still seems that extras are preferable to separate foo-X, > foo-Y, etc. if the versions are tightly coupled (e.g. foo-X version N > requires exactly foo version N) and/or they live in the same namespace. > Makes sense. > It seems like the combinatorial complexity of testing is getting pretty >> extraordinary. >> > > We certainly can't test all possible combinations, but hopefully in this > particular case testing each in isolation is good enough. > Are there dep conflicts between the extras? Kenn > > >> On Mon, Jan 27, 2025 at 3:35 PM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> Hey everyone, I put together a mini-doc on bundling some more Beam >>> Python/ML extras so that we have a better strategy for making sure that >>> users can use ML (or other Python) dependencies which are well tested with >>> their Beam version. It is mostly in line with how we handle our other >>> dependencies and only expands that scope a bit with some new extras, but it >>> should give us a well-defined strategy moving forward which we can >>> reference back to as needed. >>> >>> Please take a look here if you're interested - >>> https://docs.google.com/document/d/1c84Gc-cZRCfrU8f7kWGsNR2o8oSRjCM-dGHO9KvPWPw/edit?usp=sharing >>> >>> Thanks, >>> Danny >>> >>
