2. Make use of the current PreCommit and PostCommit test suite and modify it so that it installs pre-released dependencies.
> Leads to noisy test signals if the pre-release candidate is unstable. I am favor of option 2 since it's a simple solution that is easy to implement and try out. The disadvantage rests on an assumption that pre-released candidates would be unstable, which may not be the case. We could try this and pivot if we find this create too much noise. @Jarek Potiuk <ja...@potiuk.com> - curious, from your experience with Airflow dependency management and testing, which option do you use (if you have a similar scenario)? On Wed, Apr 12, 2023 at 7:45 AM Danny McCormick via dev <dev@beam.apache.org> wrote: > Thanks for doing this Anand, I'm +1 on option 1 as well - I think having > the clear signal of the normal suite succeeding and the prerelease one > failing would be helpful and there shouldn't be too much additional code > necessary. That makes it really easy to treat the prerelease suite as a (at > least temporary) signal on needing upper bounds on our dependencies. > > Thanks, > Danny > > On Wed, Apr 12, 2023 at 12:36 AM Anand Inguva via dev <dev@beam.apache.org> > wrote: > >> Hi all, >> >> For Apache Beam Python we are considering using pre-released dependencies >> for unit testing by using the --pre flag to install pre-released >> dependencies of packages. >> >> We believe that using pre-released dependencies may help us to identify >> and resolve bugs more quickly, and to take advantage of new features or bug >> fixes that are not yet available in stable releases. However, we also >> understand that using pre-released dependencies may introduce new risks and >> challenges, including potential code duplication and stability issues. >> >> Before proceeding, we wanted to get your feedback on this approach. >> >> 1. Create a new PreCommit test suite and a PostCommit test suite that >> runs tests by installing pre-released dependencies. >> >> Pros: >> >> - stable and pre-released test suites are separate and it will be >> easier to debug if the pre-released test suite fails. >> >> Cons: >> >> - More test infra code to maintain. More tests to monitor. >> >> >> 2. Make use of the current PreCommit and PostCommit test suite and modify >> it so that it installs pre-released dependencies. >> >> Pros: >> >> - Less infra code and less tests to monitor. >> >> Cons: >> >> - Leads to noisy test signals if the pre-release candidate is >> unstable. >> >> I am in favor of approach 1 since this approach would ensure that any >> issues encountered during pre-release testing do not impact the stable >> release environment, and vice versa. >> >> If you have experience or done any testing work using pre-released >> dependencies, please let me know if you took any different approaches. It >> will be really helpful. >> >> Thanks, >> Anand >> >
