I asked to Airflow folks about this. See [1] for the full response and a link to one of their RC emails. To summarize their position (specifically for pypi) is: Unless a user does something explicit (such as using a flag, or explicitly requesting an rc release), pip install will not serve RC binaries. And that is compatible with RC section of http://www.apache.org/legal/release-policy.html#release-types
Ahmet [1] https://lists.apache.org/thread.html/f1f342332c1e180f57d60285bebe614ffa77bb53c4f74c4cbc049096@%3Cdev.airflow.apache.org%3E On Fri, Apr 26, 2019 at 3:38 PM Ahmet Altay <al...@google.com> wrote: > The incremental value of publishing python artifacts to a separate place > but not to actual pypi listing will be low. Users can already download RC > artifacts, or even pip install from http location directly. I think the > incremental value will be low, because for a user or a downstream library > to test with Beam RCs using their usual ways will still require them to get > other dependencies from the regular pypi listing. That would mean they need > to change their setup to test with beam rcs, which is the same state as > today. There will be some incremental value of putting them in more obvious > places (e.g. pypi test repository). I would rather not complicate the > release process for doing this. > > > > On Thu, Apr 25, 2019 at 2:25 PM Kenneth Knowles <k...@apache.org> wrote: > >> Pip is also able to be pointed at any raw hosted directory for the >> install, right? So we could publish RCs or snapshots somewhere with more >> obvious caveats and not interfere with the pypi list of actual releases. >> Much like the Java snapshots are stored in a separate opt-in repository. >> >> Kenn >> >> On Thu, Apr 25, 2019 at 5:39 AM Maximilian Michels <m...@apache.org> >> wrote: >> >>> > wouldn't that be in conflict with Apache release policy [1] ? >>> > [1] http://www.apache.org/legal/release-policy.html >>> >>> Indeed, advertising pre-release artifacts is against ASF rules. For >>> example, Flink was asked to remove a link to the Maven snapshot >>> repository from their download page. >>> >>> However, that does not mean we cannot publish Python artifacts. We just >>> have to clearly mark them for developers only and not advertise them >>> alongside with the official releases. >>> >>> -Max >>> >>> On 25.04.19 10:23, Robert Bradshaw wrote: >>> > Don't we push java artifacts to maven repositories as part of the RC >>> > process? And completely unvetted snapshots? (Or is this OK because >>> > they are special opt-in apache-only ones?) >>> > >>> > I am generally in favor of the idea, but would like to avoid increased >>> > toil on the release manager. >>> > >>> > One potential hitch I see is that current release process updates the >>> > versions to x.y.z (no RC or other pre-release indicator in the version >>> > number) whereas pypi (and other systems) typically expect distinct >>> > (recognizable) version numbers for each attempt, and only the actual >>> > final result has the actual final release version. >>> > >>> > On Thu, Apr 25, 2019 at 6:38 AM Ahmet Altay <al...@google.com> wrote: >>> >> >>> >> I do not know the answer.I believe this will be similar to sharing >>> the RC artifacts for validation purposes and would not be a formal release >>> by itself. But I am not an expert and I hope others will share their >>> opinions. >>> >> >>> >> I quickly searched pypi for apache projects and found at least >>> airflow [1] and libcloud [2] are publishing rc artifacts to pypi. We can >>> reach out to those communities and learn about their processes. >>> >> >>> >> Ahmet >>> >> >>> >> [1] https://pypi.org/project/apache-airflow/#history >>> >> [2] https://pypi.org/project/apache-libcloud/#history >>> >> >>> >> On Wed, Apr 24, 2019 at 6:15 PM Michael Luckey <adude3...@gmail.com> >>> wrote: >>> >>> >>> >>> Hi, >>> >>> >>> >>> wouldn't that be in conflict with Apache release policy [1] ? >>> >>> >>> >>> [1] http://www.apache.org/legal/release-policy.html >>> >>> >>> >>> On Thu, Apr 25, 2019 at 1:35 AM Alan Myrvold <amyrv...@google.com> >>> wrote: >>> >>>> >>> >>>> Great idea. I like the RC candidates to follow as much as the >>> release artifact process as possible. >>> >>>> >>> >>>> On Wed, Apr 24, 2019 at 3:27 PM Ahmet Altay <al...@google.com> >>> wrote: >>> >>>>> >>> >>>>> To clarify my proposal, I am proposing publishing to the >>> production pypi repository with an rc tag in the version. And in turn allow >>> users to depend on beam's rc version + all the other regular dependencies >>> users would have directly from pypi. >>> >>>>> >>> >>>>> Publishing to test pypi repo would also be helpful if test pypi >>> repo also mirrors other packages that exist in the production pypi >>> repository. >>> >>>>> >>> >>>>> On Wed, Apr 24, 2019 at 3:12 PM Pablo Estrada <pabl...@google.com> >>> wrote: >>> >>>>>> >>> >>>>>> I think this is a great idea. A way of doing it for python would >>> be by using the test repository for PyPi[1], and that way we would not have >>> to do an official PyPi release, but still would be able to install it with >>> pip (by passing an extra flag), and test. >>> >>>>>> >>> >>>>>> In fact, there are some Beam artifacts already in there[2]. At >>> some point I looked into this, but couldn't figure out who has access/the >>> password for it. >>> >>>>> >>> >>>>> >>> >>>>> I also don't know who owns beam package in test pypi repo. Does >>> anybody know? >>> >>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> In short: +1, and I would suggest using the test PyPi repo to >>> avoid publishing to the main PyPi repo. >>> >>>>>> Best >>> >>>>>> -P. >>> >>>>>> >>> >>>>>> [1] https://test.pypi.org/ >>> >>>>>> [2] https://test.pypi.org/project/apache-beam/ >>> >>>>>> >>> >>>>>> On Wed, Apr 24, 2019 at 3:04 PM Ahmet Altay <al...@google.com> >>> wrote: >>> >>>>>>> >>> >>>>>>> Hi all, >>> >>>>>>> >>> >>>>>>> What do you think about the idea of publishing pre-release >>> artifacts as part of the RC emails? >>> >>>>>>> >>> >>>>>>> For Python this would translate into publishing the same >>> artifacts from RC email with a version like "2.X.0rcY" to pypi. I do not >>> know, but I am guessing we can do a similar thing with Maven central for >>> Java artifacts as well. >>> >>>>>>> >>> >>>>>>> Advantages would be: >>> >>>>>>> - Allow end users to validate RCs for their own purposes using >>> the same exact process they will normally use. >>> >>>>>>> - Enable early-adaptors to start using RC releases early on in >>> the release cycle if that is what they would like to do. This will in turn >>> reduce time pressure on some releases. Especially for cases like someone >>> needs a release to be finalized for an upcoming event. >>> >>>>>>> >>> >>>>>>> There will also be disadvantages, some I could think of: >>> >>>>>>> - Users could request support for RC artifacts. Hopefully in the >>> form of feedback for us to improve the release. But it could also be in the >>> form of folks using RC artifacts for production for a long time. >>> >>>>>>> - It will add toil to the current release process, there will be >>> one more step for each RC. I think for python this will be a small step but >>> nevertheless it will be additional work. >>> >>>>>>> >>> >>>>>>> For an example of this, you can take a look at tensorflow >>> releases. For 1.13 there were 3 pre-releases [1]. >>> >>>>>>> >>> >>>>>>> Ahmet >>> >>>>>>> >>> >>>>>>> [1] https://pypi.org/project/tensorflow/#history >>> >>
